community and household resilience in rural somaliland...6 introduction this report provides an...
TRANSCRIPT
Community and Household Resilience in Rural Somaliland
End of project evaluation
Final report
November 2016
Elizabeth Wood Research and Evaluation Services
2
Acknowledgements
Many thanks to the individuals below. Specific areas of work include; participating in key informant
interviews and evaluation workshops, providing information and advice about the project and the
context, conducting focus group discussions and key informant interviews, providing project
monitoring data, translations from English to Somali and logistics/administrative support.
Seth ‘Yuusuf’ Clarkson WRG Country Representative
Stephen Mwalo Programme Manager, Somaliland, Tearfund
Mohamed Abdirahman Hassen Programme Manager
Mowlid Abdi Hassan Support Office Manager
Liban Abdirahman Mohamed Team Supervisor
Abdikadir Mohamed Ahmed M&E Officer
Yusuf Sh. Ahmed Ali Community Mobiliser
Hodo Abdilahi Hussein Community Mobiliser
Abdirizak Osman Askar Community Mobiliser
Amal Ahmed Mohamed Logistics Assistant
Ashwaq Abdi Hashi Finance Assistant/Community Mobiliser
Mohamed Hassan Abdulahi Community Mobiliser
Umalkhayr Mohamed Ali Community Mobiliser
And also thanks to;
Myron Jespersen WRG International Office
Andrew Benckert WRG International Office
And the communities of Galoole, Raybadka, Balimataan, Lebisagaala and Ina Igarre.
Cover image: Ina Igarre, Somaliland. October 2016. Lizzie Wood.
3
Contents
Introduction ............................................................................................................................................ 6
Project context ........................................................................................................................................ 7
Summarised methodology ...................................................................................................................... 8
Scoring of OECD DAC evaluation criteria by project staff ................................................................... 9
Summary of data collection plan and sources .................................................................................. 10
Definitions of key terms ........................................................................................................................ 11
Evaluation findings ................................................................................................................................ 12
1. Impact ........................................................................................................................................... 12
1.1. How many people have been affected? .................................................................................... 12
1.2. What has happened as a result of the project? ......................................................................... 14
Scoring of the indicator by WRG project staff .............................................................................. 14
Impact indicator ranking with communities ................................................................................. 15
Seasonal timelines with communities .......................................................................................... 16
Challenges and hazards ................................................................................................................. 19
Balance of decision making power between men and women .................................................... 19
1.3. What real difference has the activity made to the beneficiaries? ............................................. 20
Tangible evidence of impact ......................................................................................................... 20
Subjective evidence of impact ...................................................................................................... 21
2. Effectiveness ................................................................................................................................. 21
2.1. To what extent were the objectives achieved? ......................................................................... 21
Community Resilience Management Plan (CRMP) ....................................................................... 22
Community Animal Health Workers (CAHWs) .............................................................................. 25
Saving for Life/Self Help Groups (SFL/SHG) and vocational training ............................................ 26
2.2. What were the major factors influencing the achievement or non-achievement of the
objectives? ........................................................................................................................................ 28
3. Cohesion........................................................................................................................................ 30
3.1. Why was coherence present or lacking? ................................................................................... 30
Cohesion within the communities ................................................................................................ 30
Internal coordination within WRG and with stakeholders ........................................................... 31
Communication between WRG and the partner villages: ............................................................ 31
Recommendations for beneficiary accountability systems .......................................................... 32
4. Efficiency ....................................................................................................................................... 34
4.1. Were the objectives achieved on time? .................................................................................... 34
Scoring of the indicator by WRG project staff .............................................................................. 34
4
5. Sustainability ................................................................................................................................. 35
5.1. What were the major factors which influenced the achievement or non-achievement of
sustainability of the project? ............................................................................................................ 35
Scoring of the indicator by WRG project staff .............................................................................. 35
5.2. To what extent did the benefits of a programme or project continue after donor funding
ceased or are likely to continue without external funding? ............................................................. 37
Additional feedback from the villages to WRG ............................................................................. 38
Livelihood diversification .............................................................................................................. 39
Complementary activities ............................................................................................................. 41
6. Relevance ...................................................................................................................................... 42
6.1. To what extent are the objectives of the programme still valid? .............................................. 42
6.2. Are the activities and outputs of the programme consistent with the intended impacts and
effects? .............................................................................................................................................. 42
External factors influencing results ............................................................................................... 43
Conclusion ............................................................................................................................................. 44
What has happened as a result of the project? ................................................................................ 44
What real difference has the project made to the project beneficiaries? ....................................... 45
The extent to which objectives have been achieved ........................................................................ 46
Contribution of the project to impact .............................................................................................. 46
Factors contributing to the achievement and non-achievement of objectives ............................... 47
Sustainability ..................................................................................................................................... 48
Measuring impact and theory of change .......................................................................................... 49
Monitoring and Evaluation ............................................................................................................... 51
Recommendations ................................................................................................................................ 51
References ............................................................................................................................................ 58
Tables and figures
Figure 1. Project outcomes, purpose and long term objective .............................................................. 6
Table 1: Summary of data collection plan and sources ........................................................................ 10
Table 2. Individually counted project beneficiaries (includes boys and girls indirectly benefitting from
the project) ........................................................................................................................................... 12
Table 3. Estimated population of each village ...................................................................................... 13
Table 4. Number of direct project beneficiaries in planned and unplanned outputs and activities
(Across the two tables. Some individuals participated in more than one activity) .............................. 13
Table 5: Summary of impact indicator ranking with evaluation focus group discussions .................... 16
Table 6. FGD groups that completed a seasonal timeline (14 in total; 9 female and 5 male) ............. 17
5
Table 7: Summary of the main findings and trends from the seasonal timelines (covering the period
October 2015 to August 2016).............................................................................................................. 18
Table 7: Percentage of village populations contributing to Community Funds ................................... 24
Table 8. Final feedback from the partner villages to the project ......................................................... 38
Figure 2. Suggested overall theory of change flow chart ..................................................................... 50
Annexes (attached)
Annex A Full list of project outputs (achievements, enabling, constraining & sustainability factors)
Annex B Full evaluation methodology
Annex C Household survey results
Annex D Household survey clean, raw dataset
Annex E Full seasonal timelines results (FGDs)
Annex F Focus group discussions and impact indicator ranking data
Annex G WRG staff and stakeholder interview data
Annex H Theory of change edits
Annex I Household survey questionnaire
Annex J Impact indicator ranking tool (FGDs)
Annex K Focus Group Discussion tool - communities
Annex L Focus Group Discussion tool - CBOs
Annex M Key Informant Interview tool - CAWHs
Annex N Key Informant Interview tool – Forest Guards
Annex O Key Informant Interview tool – vulnerable households
Annex P Data collection plan
Annex Q CLEARINGS tool for reference
Abbreviations
CRMP Community Risk Management Plan EWS Early Warning System FFS Farmer Field School FGD Focus Group Discussion KII Key Informant Interview SHG/SFLG Self Help Group/Saving for Life Group MoERD Ministry of Environment and Rural Development MOA Ministry of Agriculture
6
Introduction
This report provides an independent evaluation of the project ‘Community and Household Resilience in Rural Somaliland’ implemented by World Relief Germany and Tearfund. The evaluation focuses on the project outcomes, which are presented in the figure below with the project purpose and goal.
Figure 1. Project outcomes, purpose and long term objective
The evaluation has focused on a series of evaluation questions, which also represent quality standards. The questions fall into the following categories: impact, sustainability, relevance, effectiveness, efficiency and coherence1 (the complete list of evaluation questions can be seen below in the summarised methodology section). In addition, this report provides an analysis of the enabling factors to impact, the challenges still faced by the communities and an analysis of sustainability. At the end of the report are a series of recommendations to facilitate future resilience building work in Somaliland. There are a set of annexes that contain the findings by each output and activity, the full methodology, household survey results report, all the data sets and the data collection tools.
1 DAC Criteria for Evaluating Development Assistance <www.oecd.org/dac/evaluation/daccriteriaforevaluatingdevelopmentassistance.htm> consulted 15 Sept 2016.
Outcome 1: by September
2016, intensive planning,
preparedness and
cooperation among women
and men has been
undertaken for DRR and
continues to be coordinated
from within the 5 project
villages and their respective
households.
Outcome 2: by September
2016 each of the 5 project
villages has demonstrated
that they are growing in
their ability, with
government support, to
protect and restore
sustainability improved
livelihoods.
Purpose: by
September 2016,
the five project
villages are better
equipped to cope
with and adapt to
drought and
seasonal changes.
Long-term objective: to
develop an
environment,
communities and
households that are
resilient to seasonal
changes, drought, and
climate change by 2020.
7
Project context
The majority people living in Africa’s dryland regions make their living as pastoralists or agro-
pastoralistsi. At the same time, the greater frequency of droughts and more erratic nature of rains in
many countries, including the countries in the Horn of Africa, combined with underlying economic,
social and environmental vulnerabilities have meant that droughts have an increasingly destructive
impact on at-risk populationsii. Drought is among the most devastating of natural hazards – crippling
food production, depleting pastures (potentially leading to conflict over resources), disrupting
markets, and, at its most extreme, causing widespread human and animal deathsiii. As World Relief
Germany (WRG) and Tearfund have has identified, droughts can also lead to increased migration from
rural to urban areas, placing additional pressures on declining food production. In addition, herders
are often forced to seek alternative sources of food and water for their animals.
Further to this, the international context has directly impacted upon these rural communities. With conflict in Yemen and Saudi Arabia’s reduced purchasing power after the fall of oil prices, Somaliland is left with almost no options for exporting livestock; project partners report there is no market, at any price, for drought-weakened animals.iv
WRG has developed partnerships with the five villages of Balimataan, Galoole, Ina Igarre, Lebisagaala and Raybadka. In previous years, populations of the partner villages coped by relying on a combination of debt and livestock sale to purchase basic food supplies to get through the dry season. However, without a livestock market, project participants are at risk of exhausting credit options, greater levels of debt. In recent years, they have at times had to rely more significantly of food aid and charity.
For these reason, it is important to consider what can be done to make current, traditional livelihood
activities more productive, more stable and more sustainable, ideally through investments in
knowledge, skills, management practices and financial inputs. These investments are also most
effective when they are supported by rural communities, village leadership, as well as authorities at
the district and national (ministerial) levelv.
WRG’s and Tearfund’s project (October 2014 – September 2016) has focused on these areas, with the
aim of reducing sensitivity to shocks such as drought and increasing the ability of the communities to
manage. Activities have included on strengthening the CBOs to provide training in the activities within
the Community Risk Management Plans (CRMPs). The CRMPs have included Farmer Field Schools
(FFS), fodder production, Community Funds, Xeers to prohibit illegal deforestation, Saving for Life
(SFL)/Self Help Groups (SHG) and, later, soil conservation through Earthworks. Other key activities
included training individuals to become Community Animal Health Workers (CAHWs) and vocational
training to support livelihood diversification.
The overall strategy of the project is to implement activities in partnership with the villages, promoting
knowledge, skills, train-the-trainer initiatives and general collaboration, where possible led by
community institutions such as the CBOs or delivering through community-based facilitators.
Relatively low-cost inputs facilitate the startup and implementation of some activities, enhancing a
sustainable and participatory approach. The project has also sought to develop connections between
the villages and other stakeholders who can support trainings and activities in the longer-term, such
as FAO, district offices and government ministries. Additionally, the project emphasizes the need to
empower women in decision making processes and to raise awareness about the social, economic and
environmental challenges faced because of qat addiction.
8
The project also complements the objectives stated in the Somaliland National Development Plan
(2012-2016) in the Rural Development section (the objectives are described in the references section
of this report)vi.
Summarised methodology
This section gives the main elements of the methodology in a summarised form. Please note that the
full methodology is presented in Annex B, with the full data collection plan in Annex P.
Objective of the evaluation
The objective of the evaluation is to evaluate the project outcomes, which are listed in the section above. These will be the focus of this report, as well as being informed by a series of evaluation questions that fall into the following categories: impact, sustainability, relevance, effectiveness, efficiency and coherence2 (the list of sub-questions are provided in the full methodology in Annex B).
The evaluation will also review the project purpose and develop recommendations to support progress towards the project goal ‘to develop an environment, communities and households that are resilient to seasonal changes, drought and climate change by 2020’.
Main components of the evaluation
Following the initial scoping and inception phase, this evaluation focuses on two components: (1) Data collection and analysis and (2) Discussion, sharing and reporting findings. For the data collection, both quantitative and qualitative data has been collected, including:
Secondary data review of project documents and external reports and information.
Primary data collection during September and October 2016 including; focus group discussions, participatory seasonal timelines, impact indicator ranking, key informant interviews and a household survey.
Focus group discussions
Following the development of impact indicators, focus group discussions (FGDs) will take place with the groups listed in table 1 below in each village. The FGDs incorporate the following elements:
The FGDs questions are harmonised with the evaluation questions.
Impact indicator ranking, to further understanding about the impact of the project, from the point of view of the communities.
Mapping a seasonal timeline, giving the FGD participants a further opportunity to explain their activities to prepare and manage in the dry season, what helped them and any challenges.
Key informant interviews
Key informant interviews (KII) will provide a further source of information with stakeholders that have been directly involved in the project, either as a community member or project staff, or have important information about the project such as a government ministry representative.
2 DAC Criteria for Evaluating Development Assistance <www.oecd.org/dac/evaluation/daccriteriaforevaluatingdevelopmentassistance.htm> consulted 15 Sept 2016.
9
Household survey
A household survey has been discussed during the scoping and inception phase that would aim to measure the resilience of households according to Tearfund’s CLEARING3 tool (referenced in Annex Q). The aim of this is to provide findings about the status of resilience at the household level, and the perceptions of households about the level of resilience of their community. The findings also support the findings of the main evaluation throughout the report. The aim is also to be able to generalise findings within each of the five villages with statistical significance – and allow statistically significant comparisons to be drawn between each village. Overall, the sample for the household survey will be large enough to allow for a 90% confidence level and a 10% margin of error per village. Please see the full results of the household survey in Annex C (and the complete clean, raw dataset in Annex D).
Scoring of OECD DAC evaluation criteria by project staff In addition to analysing the secondary and primary data collected during the evaluation, during the
final evaluation the project staff scored some of the DAC evaluation criteria, based on their own
experiences and insights. The results of this are included throughout the evaluation report.
Data collection and analysis
The data collection and analysis phase will focus on gathering secondary and primary data and evidence to inform the evaluation questions. The elements of data collection will include, focus group discussions, impact indicator ranking, use of seasonal timelines, key informant interviews and a household survey.
A different perspective on impact
There have been two ways of considering impact in this evaluation. Whilst tangible measurements of impact have been considered and included in this report (such as agent-led interviews and surveys), more subjective methods of assessing impact have also been reported. Subjective impact evaluation relates to the communities perceptions of changes to their own resiliencevii. Subjective evaluation is a new approach without fully established methodologies in the sector. Both approaches are considered valuable, together they give a holistic evaluation of impact.
Gender desegregation
The main way that gender disaggregation will take place is through the qualitative data collection (focus group discussions of which nine out of 20 groups were female), as well as the use of seasonal timelines. The household survey, which randomly selected households, did not contain enough female respondents to statistically generalise the findings to females specifically.
Discussing, sharing and reporting findings
The following report has been produced based on the analysed data from the three main groups of stakeholders (communities, WRG staff and local authorities), with clear, applicable recommendations and conclusions regarding the impact, implementation and quality of the project. The initial key findings were shared and discussed in a workshop with the project staff in October 2016, during which they shared valuable input into the conclusions of the evaluation.
3 Tearfund. CLEARING tool for measuring resilience
10
In addition to WRG staff and Tearfund, it is also recommended that the key findings be shared with the communities in the five partner villages, as well as other stakeholders such as ministries and other officials.
Limitations
The data was collected by the project staff, which may impact or reduce the way the communities
speak freely about the project. Also, in the time available, as far as possible the data collection tools
were translated from English to Somali and then back-translated. These tools were for the household
survey, the seasonal timelines and the impact indicator ranking. However, the tools for the FGDs and
KIIs were not translated into Somali before they were implemented by the WRG staff. However, the
staff wrote several of the questions for these tools and were very familiar with them. There was also
additional training and verification provided on these tools.
Summary of data collection plan and sources
Table 1: Summary of data collection plan and sources
Type of data Type of data collection Total Sources
Secondary data
Document review Project progress reports, monitoring reports, monitoring data, most significant change stories, photographs, external reports, studies and articles.
Primary data (in all five partner villages)
Focus group discussions, including seasonal timelines and impact indicator ranking
20 FGDs (11 male groups, 9 female groups)
CBOs, FFS, Earthworks, SGH/SFL, vocational training participants, men and women.
Community key informant interviews
7 interviews
Vulnerable households, CAHWs, Forest Guards
WRG staff key informant interviews (in person)
6 interviews
Country Representative, Programme Manager, Support Officer Manager, M&E Officer, Community Mobilisers.
WRG staff key informant interviews (by email)
1 interview
Team Supervisor
Government ministry key informant interviews
2 interviews
Ministry of Environment and Rural Development and Ministry of Agriculture
Household survey 276 household surveys
Heads of household in the five partner villages. The sample size allows for 90% confidence level and 10% margin of error in each village. The survey provides some findings for the evaluation and is a baseline for future work.
Please note that the full list data collection tools and data can be seen in attached annexes. The
household survey results are summarised in full in Annex C.
11
Definitions of key terms
Some of the key terms used in this evaluation and their definitions are provided below.
Drought:
The definition of drought used by FAO is; drought is generally defined as an extended period (a season,
a year, or several years) of deficient precipitation compared to the statistical multi-year average for a
region that results in water shortage for some activity, group or environmental sector (NDMC, 2008).
Resilience:
The ability to deal with shocks, stresses and uncertainty. By ‘deal with’ we mean: withstand, absorb,
recover from or adapt to in a timely fashion without losing core capacities or fundamental identity
Agro-pastoralist communities: The project focuses on two types of village community, agro-pastoralist and pastoralist. Agro-pastoralist are found in mainly semi-arid zones (Aridity Index 0.2 - 0.5), agro-pastoralist systems are systems in which livestock-keepers derive half or more of their agricultural income from crop farming – and crop residues make an important share of animal feed (more than 10%). Livestock perform multiple roles such as producing meat and milk, provide man power and manure. Agro-pastoral systems are a complex form of natural resource management that allow efficient exploitation of a limited and variable resource baseviii. Pastoralist communities Found mainly in more arid zones (Aridity Index 0.05 – 0.2) these are systems in which livestock-keepers derive the majority of their income from animals that graze natural vegetation. The nutritional value and availability of natural vegetation depends on several factors including the variability and intensity of annual rains. In pastoral zones, where the potential for crop growth is limited by moisture availability, raising livestock is often the only viable form of agriculture. Livestock are moved around to take advantage of patchy seasonal vegetation. The pastoral system is a complex form of natural resource management in resource-scarce, climatically marginal and variable conditions. It is also a relationship between local ecology, livestock and peopleix. Of note is that the pastoralists are increasing engaging in planting of small plots as a diversification strategy to complement their livestock production activities. The distinction between agro-pastoralists and pastoralists is becoming increasingly blurredx.
The project has approached resilience by considering the following elements:
Exposure: the degree to which people are subject to droughts and other shocks, which depends on
where they live.
Sensitivity: the degree to which people are affected by droughts and other shocks, which is
determined by the composition of their income sources and assets.
Coping Capacity: the ability of people to reduce the impact of droughts and other shocks after they
occur. This could be by utilizing their own resources, or support from friends, relatives or the
government.
12
Evaluation findings
1. Impact 1.1. How many people have been affected? 1.2. What has happened as a result of the programme or project? 1.3. What real difference has the activity made to the beneficiaries?
Please also note the following:
The full list of project outputs and an overview of what was achieved in each output area is presented in Annex A, with enabling, constraining and sustainability factors per output.
Some of the outputs are included in the main report in the findings section. This is to highlight finding of note, support the conclusions reached and provide context for some of the recommendations made.
All recommendations developed during the evaluation are summarised in the section towards the end of this report.
1.1. How many people have been affected?
In terms of the number of people affected by the project, the direct and indirect beneficiary numbers are provided in the introduction section of this report, with the beneficiary numbers by output. Of an estimated population of between 14,000 and 15,000 individuals in the five partner villages, there were 2,186 direct beneficiaries (some of had benefited from more than one activity) and up to 15,000 indirect beneficiaries (including both adults and children). The percentage of direct beneficiaries is estimated as being 15% to 16% across the five communities.
Table 2. Individually counted project beneficiaries (includes boys and girls indirectly benefitting from
the project)4
Category Individual beneficiaries Percentage of direct
beneficiaries Direct Indirect5
Men (18 years and older)
661 3,000 30%, 20%
Women (18 years and older)
661 4,500 30%, 30%
Boys ( up to 18 years old)
432 3,750 20%, 25%
Girls (up to 18 years old) 432 3,750 20%, 25%
TOTAL 2,186 15,000 100%
4 Project monitoring data. 5 Estimation
13
Table 3. Estimated population of each village
Village Estimated individuals6 Estimated HH7
Galoole 1120 160
Raybadka 1960 280
Balimataan 3150 450
Ina Igarre 2100 300
Lebisagaala 5950 850
Total 14,280 2,040
Table 4. Number of direct project beneficiaries in planned and unplanned outputs and activities (Across
the two tables. Some individuals participated in more than one activity)8
Village CRMP
meeting attendees
Fodder training
participants
Xeer - env.
protection meeting
SFLG/ SHG
members
Trained FFS
facilitator
FFS students
Earthworks training
Galoole 455 25 511 20 2 23 51
Raybadka 224 22 238 20 2 17 28
Balimataan 189 31 210 19 2 30 61
Ina Igarre 119 0 119 10 2 21 38
Lebisagaala 189 48 266 20 2 22 87
Total 1176 126 1344 89 10 113 265
Village Sustained vuln. HH
pilot
Trees distributed
Trained CAHW
Forest Guards
Iron sheet dist. for roof
Galoole 53 2 0
Raybadka 28 21 2 0
Balimataan 43 2 0
Ina Igarre 62 2 1
Lebisagaala 28 98 2 2 12
Total 56 277 10 3 12
6 The total number of people is based on seven people per household (in line with the project planning) and is an estimation. 7 Estimate of households in based on surveys during the project and confirmed during the evaluation. Where the estimates differed, an average number of households is recorded in this report. It is based on seven people per household. 8 Source of information for the beneficiary numbers is the M&E Officer for WRG Somaliland
14
1.2. What has happened as a result of the project?
This section focuses on the degree to which women and men in the partner villages have been carrying out intensive planning, preparedness and cooperation (outcome 1), as a result of project. An overview of findings is provided, with a selection of the outputs where there is information and experiences of note.
The overarching approach of the project has been to develop groups and structures within the
communities that have formed the basis for the sharing of knowledge, skills and experiences, which
has proven to be very successful. The different elements of the project have worked well together to
support women and men to reduce the risks from shocks such as rain shortage and drought. These
structures have also become forums for support between the groups, such as farmers and groups of
women, as well as enabling the communities to collaborate to prepare for and manage the dry
seasons.
In addition to considering a sample of the output areas, the evaluation aimed to establish overall
changes in the way the communities in the partner villages are preparing and managing for the dry
seasons. This was facilitated by fopur elements of the methodology; the focus group discussions,
impact indicator ranking, development of seasonal timelines and scoring of the indicator by WRG
project staff.
Scoring of the indicator by WRG project staff
In order to support the findings of the evaluation and provide a different perspective, during the final
evaluation workshop the WRG project staff were asked to draw on their own experiences and insights
and score a selection of the OEDC DAC evaluation criteria. The scoring system was as follows:
0 = no evidence
1 = some evidence
2 = satisfactory evidence
3 = good evidence
4 = strong evidence
For impact, six out of 11 staff present gave a score of 3/4 (55%), with two of the 11 staff gave a score
of 4/4 (18%). The explanation the team members gave for a score of 3 was as follows:
We are seeing evidence of how the community is growing (CRMP), number of people
participating, with government support9.
Each community has a plan has a plan to prepare – some elements are challenged by lack of
rain, eg. fodder production. However, there are some cases of buying fodder, this shows how
an alternative income source is helping to improve resilience10.
We see communities caring for and protecting their environment. There is a relationship
between the district government office and the forest guards11.
9 Mohamed Abdirahman Hassen, WRG Programme Manager. 10 Hodo Abdilahi Hussein, WRG Community Mobiliser. 11 Liban Abdirahman Mohamed, Team Supervisor.
15
People are collaborating and working together, there are examples of small business such as
henna in Raybadka, which is of good quality12.
For the two out of 11 staff who gave a score of 4, for strong evidence of impact, the rational was as
follows:
Most of the communities understand the importance of caring for livestock. The community
is contacting the CAHW (who can contact the ministry) in cases of animal sickness13.
Preparation of communities for the dry season and how they are using knowledge and skills.
The women before they did not meet in groups, they lacked confidence to have their voices
in the community but now they are encouraged to speak out and share their knowledge – and
start small business, alternative income14.
Impact indicator ranking with communities
In addition to analysing the impact of each output area, in order to understand more about the impact
of the project, a series of six impact indicators were developed that were then ranked with the
participants of in the focus group discussions. The impact indicators were developed as part of pilot
focus group discussions in Galoole village.
Following this, focus group discussion groups in the five partner villages was asked to rank the impact
indicators according to the following system:
The impact indicator that represented the greatest impact of the project on their lives
The impact indicator that represented the least great impact on their lives.
The impact indicator ranking took place with 19 out of the 20 FGDs, with both male and female groups.
The results of this impact indicator ranking exercise are summarised below in table 5. The final results
show that, according to the communities, the three areas of greatest impact of the project on their
lives are:
1. Increased preparation activities for the dry season due to increased knowledge and skills
from the project activities.
2. Increased collaboration in the community and sharing ideas on how to prepare for the
drought.
3. Increased soil and water conservation due to increased knowledge, skills or farming tools.
12 Ashwaq Abdi Hashi, Finance Assistance (Community Mobiliser during the evaluation). 13 Yusuf Sh.Ahmed Ali, Community Mobiliser. 14 Abdikadir Mohamed Ahmed, Community Mobiliser.
16
Table 5: Summary of impact indicator ranking with evaluation focus group discussions
Based on 18 of the 20 FGDs
Ranked 1 by the communities
(greatest impact)
Ranked 2 by the communities
(second greatest impact)
Rank 3 by the communities
(third greatest impact)
1
Increased preparation activities for the dry season due to increased knowledge and skills from the project activities.
55% 33% 5%
2
Increased collaboration in the community and sharing ideas on how to prepare for the drought.
33% 22% 22%
3
Increased soil and water conservation due to increased knowledge, skills or farming tools.
5% 17% 38%
4
Increased health of herds/livestock due to the CAHW.
0 17% 22%
5
Households can more easily provide for their families due to increased business, SHG loans, farming or other activities.
0 5% 5%
6 Increased knowledge
about the environment and national resources.
5% 5% 5%
Seasonal timelines with communities
In addition to the impact indicator ranking, during a selection of the focus group discussion groups the
facilitators worked with the group participants to develop a seasonal timeline in a participatory
manner, covering the period October 2015 to August 2016 (during the second year of the project).
The timelines asked for information according to the following questions:
1. During which months were the dry/rainy seaons?
2. What did you do to prepare for the dry seasons?
17
3. Were there any factors that helped you to prepare and manage in the dry seasons?
4. Were there any factors that challenged your preparations and ability to manage?
In total, 14 seasonal timelines were completed out of 20 FGDs (as it was not planned for the CBOs to
complete a seasonal timeline in their FGD). The summary of data collected from the season timelines
is presented on the following page, with the full results in Annex E. Table 5 below list the FGD groups
that made a seasonal timeline.
Table 6. FGD groups that completed a seasonal timeline (14 in total; 9 female and 5 male)
Village FGD group Female/male group
1 Galoole Farmer Field School Male
2 Galoole Self Help Group Female
3 Galoole Women in the community Female
4 Raybadka Self Help Group Female
5 Raybadka Women in the community Female
6 Raybadka Vocational training Female
7 Raybadka Earthworks Male
8 Balimataan Self Help Group Female
9 Balimataan Farmer Field School Male
10 Lebisagaala Women in the community Female
11 Lebisagaala Self Help Group Female
12 Lebisagaala Earthworks Male
13 Ina Igarre Farmer Field School Male
14 Ina Igarre Vocational training Female
Summarised results of the seasonal timelines
The seasonal timelines showed that, on average, the rainy seasons were from October to December 2015 (Dehr rains) and then May to July/August 2016 (Gu rains). The dry seasons were recorded as being from January to April 2016 and September 2016 (in some cases the second dry season was considered to have begun earlier in August 2016).
The seasonal timelines are documented in full Annex E, with a summary of the main findings and trends in table 7 below.
Where possible, the differences in trends between the male and female groups have been recorded.
18
Table 7: Summary of the main findings and trends from the seasonal timelines (covering the period
October 2015 to August 2016)
Timeline of rainy and dry seasons
Type of information Main findings and trends
Rainy season: October to December 2015 (Gu rains)
Preparation activities during the rainy season
The male FGD groups recorded their main preparation activities in October 2015 as being; training (eg. FFS or Earthworks), farm preparation, fodder preparation, water harvesting (with plastic sheet below soil), continuing with CRMP plans.
The female FGD groups said their main preparation activities in October 2015 were the same as the males with some additional activities; each female group (mainly SHG or vocational training) also said savings, savings for food and generating an income.
Dry season: January to April 2016
Facilitating factors in the dry season starting in January 2016
Both male and female FGD groups recorded similar activities and factors present that helped them to manage in the dry season. The majority of the FGDs groups said that in January and February 2016, at the start of the longer dry season, they utilized fodder stores for their livestock, purchased additional fodder and food for the household.
Eight of the 14 groups said that they migrated from the villages to search for pasture at this time.
Eight of the 14 groups also said that they had support from relatives during this dry season – several of these groups mentioned that these were relatives overseas. A small number of the groups said that they sold livestock in January or February 2016.
Constraining factors in the dry season starting in January 2016
Male FGDs: shortage of rains, insect and animal pests damaging farms, water scarcity, shortage of income, lack of pasture. One group also mentioned low seed quality. Female FGDs: shortage of rains affected the ability to save, water shortage at the berkeds, shortage of fodder. In addition, seven of the overall 20 FGD groups said that malnutrition was also affecting the villages at this time (Galoole, Balimataan and Raybadka).
Rainy season: May to July/August 2016 (Dehr rains)
Preparation activities during the rainy season
Almost all of the 14 groups described fodder preparation, harvesting and storage as a key activity between April and July 2016. The importance of
19
harvesting fodder during the rainy season was also emphasized by WRG during the trainings15.
The agro-pastoralist FGDs (Galoole and Raybadka) also emphasized the farm preparation that takes place at this time. The selling of cash crops and milk was described by the vocational training group in Raybaka and by the SHG in Lebisagaala.
Challenges and hazards
According to the seasonal timelines, the main challenges faced by communities include;
Shortage of rains/period of drought
Insect and animal pests damaging farms
Water scarcity
Shortage of income
Lack of pasture
There was a trend of findings that the shortages of rain impacts on the ability of people to save and produce sufficient fodder. It was also reported that there have been reductions household food security to such an extent that malnutrition was present in the villages (the proportion of people that were affected isn’t confirmed in this evaluation). The issue of malnutrition has also been assessed and acted upon by WRG during the project period.
The household survey also confirmed that rain shortage and drought are the main hazards that people face, with 89% of households across the five partner villages considering this the greatest hazard that affects them. This was followed by deforestation with 37% of households considering this to be the second biggest hazard they faced.
Balance of decision making power between men and women
There is also strong evidence from the evaluation FGDs that the role of women in decision making is increasing and that the project has contributed to this. For example, of the 11 FGDs with male groups (across all partner villages), five groups talk about how in the past men made the household decisions, but since the droughts both men and women cooperate to reach decisions together.
One area of decision making was stated as being in relation to females managing their assets and livelihoods (Galoole FGD with men in the community). Two of the groups described how they thought that this balance of decision making was a positive thing (Raybadka Earthworks and Balimataan FFS).
15 Results of the evaluation Key Informant Interview, WRG staff. Hodo Abdilahi Hussein, Community Mobiliser
20
In addition, the household survey found that of the respondents (mainly the heads of household) 75%
said ‘yes’. there is equitable decision making power about property/assets between males and
females/husband and wife. Of the 75% who said ‘yes’, 61% were male and 39% were female.
In some villages, women were observed in leadership positions in the CBOs, for example, in Galoole.
1.3. What real difference has the activity made to the beneficiaries?
This section focus on evidence of impact including tangible, measurable information, as well as
subjective information (how the communities perceive their own resilience).
The evaluation analysed the evidence that there is intensive planning, preparing and cooperation among women and men for DRR (outcome 1), as well if the communities are growing in their ability to protect and restore their livelihoods (outcome 2). The evidence identified is summarised in the boxes below.
Tangible evidence of impact
There has been an increase in the activities and collaboration in the community and sharing ideas for how to prepare for the drought, eg. farm and irrigation preparation, increases soil conservation, communal fodder cultivation, communal farms and savings groups. The communities attribute this to the knowledge gained during the project. Almost all of the respondents said that they did not collaborate in this way before the project began or did not realise why it was important before the project, whereas now they do (evidence in seasonal timelines, FGDs, KIIs).
On average, 97% of households across the five villages said that they had experimented with new flood protection or erosion control techniques in the past year. The results were the highest in Galoole (90% of houeholds) and Raybadka (91% of households) – but the lowest score was still relatively high at 82% of households in Lebisagaala. This indicates the success of the elements of the project that focus on these activities - and that knowledge may be shared beyond the trainings and workshops (household survey16).
Increased health of herds/livestock due to the CAHW (weaker evidence as lower ranking in the impact indicator ranking exercise. Although the evaluation could not determine the degree of success, this was stated by the CAHWs in Galoole and Raybadka and verified by the CBOs.
78% of households in the five villages said that they had started farming activities in the last year. This may be due to direct participation from initiatives such as Earthworks and FFS, or the transfer of knowledge from these initiatives (household survey).
Other women have come to observe the SFL/SHG groups and in some cases started a second group (project progress reports, key informant interviews with WRG project staff).
16 The sample size and method of household selection allowed for a 90% confidence interval and 10% margin of error in each village and across the five partner villages as a whole.
21
Subjective evidence of impact
Increase in equitable decision making between males and females. The household survey found that 75% households across the five partner villages say ‘yes’, there is equitable decision making between husband and wife about property/assets (61% of the respondents who said ‘yes’ were male, 39% were female). The majority of the 20 FGDs also reported this and that this had changed since before the project began (stated by both male and female participants). Two of the male FGD groups said this was because of the increased knowledge of females and that they believe they can contribute better ideas as a result of participating in the SFL/SHG groups (household survey, FGDs).
The majority of the FGDs (which included female and male groups who had not participated directly in the activities such as FFS, SHG, Earthworks) said that before the project began two years ago, their community did not know how to prepare for the dry seasons and droughts, however, now they have learnt different ways to manage and share ideas (FGDs, impact indicator ranking).
84% of the populations across the five villages knew about the CRMP (household survey).
The household survey also showed that 76% of the households across the five villages believe that there is a system of social support in place if they lost their property/assets. Of these households, 69% believe that the CBO would give support, followed by relatives and friends (20% of households) and the village committee (11% of households). Given that the CBO was not in place in a small number of the villages before the project began, this may be an indication of the work the project has done to strengthen the role of the CBO in each village.
93% of the households across the five villages know that there are systems in place to manage natural resources (household survey). 75% of these can access the natural resources they need, with the main reasons for not being to access them being the need to pay and problems with access/too far. (Note, although these findings are likely to be impacts of the project, it cannot be known with certainty without a comparable survey baseline to compare to).
Increased collaboration in the community and sharing ideas for how to prepare for the drought (all 20 FGDs and impact indicator ranking exercise).
2. Effectiveness 2.1. To what extent were the objectives achieved? 2.2. What were the major factors influencing the achievement or non-achievement of the objectives?
2.1. To what extent were the objectives achieved?
In addition to considering what has happened as a result of the project and the impact, each output has been evaluated in Annex A (with achievements per impact, with enabling and constraining factors). Annex A shows that the majority of the planned outputs have been fulfilled, especially where activities can be implemented directly by the communities (facilitated by initiatives such as train-the-trainer, the CRMPs and strengthening the CBOs). These include the activities that fall under the CRMP such as the Farmer Field Schools, SFL/SHG groups, Earthworks and communal fodder production, which have shown particular success. In addition, Community Animal Health Workers and tree
22
planting. A selection of these outputs is provided below in the next section, also to highlight experiences of note.
The outputs that have not been met in full have faced several challenges. Although some of the challenges have been largely out of the control of project they should be possible to mitigate if future project work is designed taking them into account. These outputs include the development of an Early Warning System for climatic information, the household level holistic rangeland management pilot, the impact of the Xeer on illegal deforestation, and the impact of the CBOs awareness raising on stopping chewing qat (the last two being raised as concerns by the communities). For this group of outputs, a common factor with some of them is that they would benefit from support and collaboration with external stakeholders such as;
An agency with the necessary capacity to monitor climatic information from SWALIM.
Ministries and district offices to support Forest Guards (funding and status of the guards).
A key, challenging factor is the available capacity at such institutions.
Regarding the Xeer and chewing qat, the Xeer agreement is in place and the CBOs have proactively promoted information about qat in each village. However, according to the WRG project staff more time is needed for the communities to act on these initiatives (eg. a significant reduction in illegal deforestation and qat chewing – especially as both these challenges are interlinked). The evaluation FGDs also showed a trend that these two factors are the main concerns of the communities across the five villages, as well as rain shortages and drought.
The full summary of factors contributing to the achievement and non-achievement of these outputs can be seen in Annex A.
Community Resilience Management Plan (CRMP)
a. Output details: Community Risk Management Plans (CRMP) for 2015 have been developed and implemented (outcome 1) in each of the five partner villages. The villages are implementing the CRMPs over the course of the project.
The CRMPs were developed as planned with the initial four villages within the first six months of the project and this process was led by the CBOs with the support of WRG, utilising the PADR tool. The contents of the CRMPs were planned with the villages and the priority areas identified to include were; fodder production, Community Funds, Xeer agreement to prohibit illegal deforestation, Savings for Life groups/Self Help Groups and Farmer Field Schools (with Earthworks added later).
The period of drought between November 2015 and March 2016 has meant that the activities related to the CRMP had to be delayed. However, as of March 2016, Ina Igarre, the fifth village, finalized its
23
CRMP and started to implement the activities. Over the project period, 1176 individuals attended community meeting regarding the CRMPs.
Impact of the CRMPs on resilience
To understand how the communities are planning for the dry seasons and managing prolonged rain shortages, the evaluation collected information in all five partner villages from 20 focus group discussions (14 of which included seasonal timelines using a participatory approach), several community key informant interviews and a household survey. The seasonal timelines were developed within 14 of the FGDs (9 female and 5 male groups) and highlighted the activities that take place in different seasons and how the project has contributed to these.
How the communities prepared for the dry seasons
The seasonal timelines showed that during the rainy season Gu (reported by the groups as having happened from October to December 2015), the male FGD groups recorded their main preparation activities as being; training (eg. for FFS, fodder production or Earthworks), farm preparation, fodder preparation, water harvesting (with plastic sheet below soil), continuing with CRMP plans. The female FGD groups said their main preparation activities in
this period were the same as the males, with some additional activities. Each female group (who were mainly SHG or vocational training participants) also said they carried out activities relating to saving money (eg. for food) and generating an income.
In the second rainy season Dehr (reported as being May to July/August 2016), almost all of the 14 groups described fodder preparation, harvesting and storage as a key activity. The importance of harvesting fodder during the rainy season was also emphasized by WRG during the trainings17.
How the communities managed in the dry seasons (seasonal timelines):
In the first dry season (reported as being from January to April 2016), both the male and female FGD groups recorded similar activities and factors present that helped them to manage. The majority of the 14 groups said that at the start of this longer dry season, they utilised fodder stores for their livestock, purchased additional fodder and purchased food for the household.
The agro-pastoralist FGDs (Galoole and Raybadka) also emphasized the farm preparation that takes place at this time. The selling of cash crops and milk was described by the vocational training group in Raybaka and by the SHG in Lebisagaala.
Eight of the 14 groups said that they migrated from the villages to search for pasture for their livestock.
Eight of the 14 groups also said that they had support from relatives during this dry season – several of these groups mentioned that these were relatives overseas.
17 Evaluation Key Informant Interview, WRG staff. Hodo Abdilahi Hussein, Community Mobiliser
24
A small number of the groups said that they sold livestock in January or February – although it is not clear if this was planned destocking or an emergency coping strategy.
The preparation and management activities are indicated as utilizing knowledge gained in the training, information and inputs shared by the project with the partner villages. There was not a trend of any emergency coping strategies, expect for the possibility of selling livestock. It is recommended to review further such destocking activity and if this is an informed planned activity or an emergency coping strategy that could put households at risk.
Further ways in which the project has contributed to these preparation and management activities and how the communities are growing in their ability to become more resilience is discussed under outcome 2 below.
Community Funds
The Community Funds are also a key part of the CRMPs. These are livestock based funds, the purpose is for environmental/resilience initiatives, such as Earthworks in the agro-pastoralist villages. An example of how the funds have been used includes hiring machinery for collective fodder cultivation, which was indicated as being successful by the communities. In terms of the percentage of the people contributing to the Funds this was aimed at being 40% of the populations in Galoole and Raybadka and 20% of the populations of Balimataan, Lebisagaala and Ina Igarre. The results of this are provided in table 7 below.
Table 7: Percentage of village populations contributing to Community Funds
Village Number of HH Percentage18
Galoole 70 43%
Raybadka 30 11%
Lebisagaala 60 7%
Impact story relating to the CRMPs (Farmer Field Schools)19
Deeq Adan Husseina is a 56-year-old resident of Balimataan is a father of 11 children with one wife. Deeq described to the WRG Programme Manager how the project lifted him out of poverty thanks to training, especially the conservation farming. He also took the opportunity to become Farmer Field School facilitator. He led the classes in a demonstration plot within his farm and continued working hard every day. Deeq now grows
18 Based on project monitoring data, October 2016. 19 Collected by Mohamed Abdirahman Hassan, WRG Programme Manager, Somaliland.
25
tomatoes, water melon and also plans to start a seedling nursery. He sells to the produce within Balimataan and surrounding communities.
While many neighbouring farms are not harvested his farm is greener with minimum input. People within the community recognise his abilities and many of them rely him on. He realizes his farm is his lifeline and will continue working hard to improve it, with the funds generated, he supports back his family including the school fees of his children.
Deeq said during the evaluation focus group discussion: "Before I was not even able to talk to the community, I was very poor. Since the project began, the main difference is that we work together, rather than each individual working alone. And now I am a farmer, selling my crops".
In addition, each of the three evaluation FGD groups with FFS said that before the project began two years ago they worked individuals, whereas now the community works together to cultivate farms for cash crops and fodder. Other examples included lending seeds to each other. One member of the group in Balimataan said: "As a result of WRG, we have learnt skills and training such as fodder production, growing vegetables and the importance of collaboration that we did not have in the past". The group in Ina Igarre said that they had used their Community Fund to hire a tractor for fodder harvesting.
Community Animal Health Workers (CAHWs)
d. Output details: 12 CAHW's are training and providing fee based services to the 5 villages by December 2015.
During the project period, 12 Community Animal Health Workers (CAHWs) had begun the training
with support from the Ministry of Livestock and FAO. The CAHWs, under supervision of CBO, now
provide services for a fee - and report to WRG on a monthly basis.
In terms of the number of people affected, the
CAHW in Galoole reported during his interview
that he helps approximately 25 households each
month. Two CAHWs in Raybadka reported that
they help approximately 25 and 14 households
each month, respectively.
Three CAHWs were interviewed during the
evaluation in Galoole and Raybadka. All stated
that they believe the biggest impact of their work
is a reduction in the death of livestock, increased
health of livestock and increased awareness
within the community of animal health. The
increase in the health of livestock was also supported by the CBOs/leaders in each village where the
CAHWs operate. Representatives of the CBO’s said in follow up interviews that they have seen an
increase in the health of livestock due to the CAHWs. However, in terms of an increase in the number
of livestock, there was not sufficient evidence to state this. In addition, livestock numbers may have
been negatively impacted due to the recent periods of drought.
26
In term of the factors that have facilitated the CAHWs, the three interviewed CAHWs all said that the
training gave them the skills and knowledge to be able to carry out their role. "I like many things about
the training, such as to learn different kinds of animal diseases and also how to help the communities
to improve the health of livestock (CAHW, Galoole).
Additional factors influencing the achievements of the CAHWs include ministerial level support for
vaccinations when disease outbreaks have occurred and routine vaccination campaigns. The CAHWs
have played a role in this, for example, in Ina Igarre during a disease outbreak amongst camels, the
CAHWs alerted the authorities who then provided vaccinations for the remaining healthy animals.
The CAHWs also said that their services were interrupted by periods of drought, due to the migration
of many community members. In terms of sustainability, an issue was an issue raised by the CAHWs
in both villages was the ability of households to pay for the services. One of the CAHWs in Raybadka
said: "my challenge is when I carry out work in the community and cure livestock, the household
borrows from me and promises to pay for the services later. Then I have challenges buying medicine
and continue the service for them". Another CAHW in Galoole said: "the greatest challenge I face is
the service charge debt as the community does not have the money". The CAHWs also said that the
demand for their services is growing and that they need additional medicines, health kits, equipment
for weighing camels and continued training. These factors combined may challenge the sustainability
of the CAHWs, who are demonstrating that they provide a needed and valuable service.
In any future project work, it is recommended to review the extent of the challenge of household
payment, the request for additional supplies and training and also how the linkages to the ministries
could be further strengthened.
Saving for Life/Self Help Groups (SFL/SHG) and vocational training
b. Output details: A SFL group is established and meeting regularly in each of the five villages by Sept 2015 and has been transitioned to a SHG by Sept 2016.
In total, currently there are 89 recorded members of a SFL/SHG across the five villages, with around 20 members in Galoole, Raybadka, Balimataan and Lebisagaala and 10 members in Ina Igarre (the newest partner village).
Within the first six months of the project, the planned training days on savings and loans took place in the four initial villages. In terms of achievements, by October 2015, the four established SFL groups had taken loans from WRG to begin activities and some had started group businesses. By March 2016, the fifth group in Ina Igarre village (set up in the second year of the project) also finalised their business plan and was ready to receive their loan.
27
All of the groups have now lent funds to their members20 with terms of two months for between 15 and 40 USD21. The loans have not yet been paid back to the constraints of the dry seasons and droughts, which have affected the ability of the groups to save.
Further to this, WRG has provided the planned follow up training with the different WRG staff. SFL members have also received training in Islamic Finance methods but are yet to put it strongly into practice.
In addition to the SFL/SHG groups, 36 individuals from the groups in Galoole and Raybadka were selected to receive vocational training in the areas of cookery, tailoring and henna decorating.
Although it is too early to see the benefits of the training, there were some individual examples of groups or individuals had started a business and providing services. These included a henna business (Galoole) and selling clothes procured from Hargeisa (Galoole). Anecdotal evidence suggests that that 12 females are utilising the henna skills.
In terms of facilitating factors, the approach of empowering the members of the groups to save and begin their own business activities has been effective. The groups are indicated as being sustainable, describing their resources as being; their savings, knowledge from training, vocational skills, fodder harvested in the rainy seasons, land, livestock, unity and human resources.
Their resources are sustainable in terms of not relying on outside support (aside from any loans needed to startup
businesses) but some are sensitive to the shock of drought.
In the evaluation, the three groups also said they were motivated to continue saving and would continue to collaborate to generate income through small businesses, Community Funds and agricultural activities.
For factors that contributed to the non-achievement of objectives, during February and March 2016, the groups reported
20 Tearfund Monitoring trip report, Somaliland, World Relief Germany, November 2015 21 WRG Community and Household Resilience, project progress report. October 2015
Impact story
Sabah, the leader of the Savings for Life Group in Ina Igarre. She talked about the business the group has set up selling fuel imported by truck from Hargeisa based in a local shop. Customers include people from the village and other trucks and vehicles passing through.
Sabah also said that the business made about $15 USD per month in profit for each member of the group – and that the initiative had given the women the confidence to try new initiatives.
28
that it has been difficult to meet and save due to the stresses of the drought impacting on their activities.
It is also indicated from both SFL/SHG members both during the evaluation FGDs and from WRG staff that skills training in literacy and numeracy (as well as other possible business related skills) may facilitate the general capacity of the groups to start up and manage small business ventures more effectively.
2.2. What were the major factors influencing the achievement or non-
achievement of the objectives?
Based on the findings of this evaluation, there are several key facilitating factors behind the
achievements of outcomes one and two across the five partner villages. These are summarized in the
box below.
1. The 14 FGD groups that completed the seasonal timelines all described fodder preparation, harvesting and storage as a key activity in the rainy season between April and July 2016. The importance of harvesting fodder during the rainy season (and not waiting for the dry season as some of the communities were doing before) was also emphasized by WRG during the trainings22.
2. The sharing of knowledge, skills and experience by WRG with the different groups – and the low cost inputs such as tools. This has also increased sustainability with community-based facilitators who have been trained in different areas such as FFS, CAHWs, CBO members and the heads of the SFL/SHG groups. It is recommended to continue the focus on train-the-trainer activities.
3. Regarding the CRMPs, the household survey shows that across the five villages, 84% of households (with mainly the head of household being the survey respondent) are aware of the CRMP in the community. The framework of the CRMP seems to have given the communities a sense of ownership and a structure to the activities.
4. The CBOs were more easily able to mobilise the communities in Galoole and Raybadka for the CRMP activities – WRG had had a long relationship with both of these villages and this good partnership undoubtedly facilitated the process.
5. There is evidence that the different groups such as FFS, SFL/SHG groups place high value on the knowledge and skills they have learnt during the project, which will increase project impact and overall sustainability.
6. Regarding the CAHWs, a factor influencing the achievement of this objective includes ministerial level support for vaccinations when disease outbreaks have occurred and routine vaccination campaigns. The CAHWs have played a role in this, for example, in Ina Igarre during a disease outbreak amongst camels, the CAHWs alerted the authorities who then provided vaccinations for the remaining healthy animals.
7. The Forest Guards seem to have a positive impact on natural resources, not only because they deter people from illegal deforestation (through reporting individuals who may then be fined) but also through the awareness raising they do
8. The high standard of the WRG team to listen to the communities from the initial design and planning of the project, to responding to ongoing feedback and ensuring that the villages have
22 Evaluation Key Informant Interview, WRG staff. Hodo Abdilahi Hussein, Community Mobiliser
29
the information they need to conduct resilience building activities. For example, the training for Earthworks and soil/water conservation was adapted according to the communities.
9. The emphasis WRG has placed on developing community structures, strengthening the role of the CBOs and encourage collaboration between households and groups. When asked, each evaluation focus group discussion group and key informant interview across the five villages described how they are working together as a community, compared to before the project began two years ago when they worked more individually.
10. Focusing on women and increasing their potential to increase their household security through cooperative groups. This has also contributed towards increasing the role of women as decision makers.
Following this, there were several key facilitating factors found behind the non-achievements of
elements of the project, when considering the project in frame of the outputs. These are summarized
in the box below.
There was some resistance to destocking and banking, particularly among purely pastoralist communities. This made adoption of a Community Fund challenging and sometimes the Funds were inaccessible when needed (for example, because funds were tied up in devalued livestock). The project has looked into ways to mitigate this.
It is indicated from both SFL/SHG members during the evaluation FGDs and WRG staff that skills training in literacy and numeracy (as well as other possible business related skills) may facilitate the general capacity of the groups to start up and manage small business ventures more effectively using basic business skills.
MoERD have not been able to fund Forest Guards in all of the partner villages (forest guards are in place in Lebisagaala and Ina Igarre). MoERD do support the initiative of forest guards - also citing the success of this preventative measure in other parts of Somaliland23. However, the limited resources of the ministries do pose an ongoing limitation on this strategy.
The Forest Guards face challenges explaining who they are to individuals or groups cutting down trees illegally and have even been attacked in some cases. Formal ID cards may help them to explain their role.
The majority of the 20 FGDs and the community KIIs were concerned about illegal
deforestation. This is likely to be influenced by charcoal production, for there is a demand
especially from urban areas – which also in turn may fund qat chewing, also highlighted as a
concern by several of the groups and very challenging to overcome. The Forest Guard in
Lebisagaala said that large, illegal enclosures are increasing illegal deforestation.
There was a trend in the male FGD groups (eleven in total across the five villages including the
CBOs) to cite insect and animal pests damaging farms as an issue.
The WRG staff also described in the final evaluation workshop that farming activities cannot
take place in the dry season. It is recommended to consider ways to strengthen rangeland
management to further support to both the agro-pastoralist and pastoralist communities
during the dry seasons, as well as to consider appropriate alternative livelihoods (considering
the role of the traditional, productive livelihoods in any livelihoods strategy).
All of the seasonal timelines documented shortage of rains and water scarcity (including at
the berkeds). Several groups reported shortage of pasture and shortage of fodder.
23 Evaluation Key Informant Interview. Abdikarim Omar, Director of Planning, MoERD. October 2016.
30
The five male FGDs that completed a seasonal timeline reported that the main constraining
factor to their preparation and management activities were shortage of rains, insect and
animal pests damaging farms, water scarcity, shortage of income, lack of pasture. One group
also mentioned low seed quality.
The 9 female FGDs which completed a seasonal timeline reported that shortages of rains
affected their ability to save. Other challenges they described were water shortage at the
berkeds and a shortage of fodder. In addition, seven of the overall 20 FGD groups said that
malnutrition was also affecting the villages during the longer dry season, which WRG has
assessed and responded to earlier in 2016 (Galoole, Balimataan and Raybadka).
3. Cohesion 3.1. Why was coherence present or lacking? Was it necessary? 3.2. What were the particular factors that led to coherence or its lack?
One of the evaluation questions related to coherence and the evaluation process also collected data
to inform these questions from a range of sources, such as FGDs, KI interviews with WRG staff and
ministries and also general observations. Coherence will be considered in three ways; cohesion within
the communities, internal coordination within WRG and communication between WRG and the
partner villages.
3.1. Why was coherence present or lacking?
Cohesion within the communities
The structures that the project has developed through the FFS, SHG/SFL groups, Earthworks groups
and strengthening the role of the CBOs have been the foundation for improved communication
cohesion. The evaluation found in the focus group discussions in the five villages that the respondents
placed high value of collaboration, as well as several examples of households and groups working
together. Some of these accounts are listed below:
“Before the project we tried to overcome the drought individually, now we work as a
community. For example, we cultivate farms as a group and use fodder during the dry season.
We are seeing a decrease in household migration because we are working together on things
like vegetable production, water conservation and fodder production (evaluation FGD,
Galoole FFS, males).
“We didn’t work together before but now we understand and collaborate together”
(evaluation FGD Balimataan, SHG, females).
“As a group, we prepared our farms for cash crops and producing fodder. We have used the
Community Fund to hire tractor for fodder harvesting and we also lend seeds to each other”
(evaluation FGD Ina Igarre FFS, male).
31
“We are working together and helping each other - we have had discussions about how to
best cope in the dry season. Also, we have collected fodder as a group and taken some of this
to the vulnerable households, as well as sometimes taking them food” (evaluation FGD,
Galoole vocational training, females).
An important indicator of cohesion across the district was a major clan meeting that took place
in Salahley between March and October 2015. The focus of the meeting was the importance
of environmental protection and resulted in a declaration to stop charcoal production24.
Internal coordination within WRG and with stakeholders
Key individuals from WRG’s Somaliland programme and from a key government ministry gave positive
feedback about the project’s coordination:
“We have a weekly coordination meeting as a whole staff. The staff give feedback from beneficiaries and plan how it will be responded to. We also have a participatory decision making process within the team” (Seth ‘Yuusuf’ Clarkson, Country Representative, WRG Somaliland).
“I was involved in the proposal design so by the time the project starts I knew what is happening. I was able to give input into the activities along with the Community Mobilisers. We also have a Gantt Chart to help with planning. We have weekly coordination meeting so everyone knows the priorities and what each person need to do that week” (Mowlid Abdi Hassan, Support Office Manager, WRG Somaliland).
“We coordinate well with WRG and they also participate in the coordination forums regarding environmental conservation and protection” (Abdikarim Omar, Ministry of Environment and Rural Development).
Communication between WRG and the partner villages:
As well a good communication and coordination being demonstrated within both the communities
and the WRG project team, the project has performed particularly well in terms of listening to
feedback from communities and incorporating this into the design of the project. This seems to have
particular important for the approach of the project; based in partnership, collaboration and
communities taking the lead on the different initiative supported by WRG with inputs of training, skills,
energy and relatively low-cost inputs. The respondents in each of 20 evaluation focus group
discussions and seven key informant interviews across the five partner villages reported on this
element of the project positively, especially in terms information they have received about the
activities and how they have been listened to by the WRG team. Some accounts are highlighted in the
box below:
“We can communicate problems and suggestions to the CBO and to WRG. "We believe that
this is the only organization that bases all its intervention based on our feedback” (evaluation
FGD Lebisagaala, Earthworks participants, male).
“During the last year there was shortage of food and we were facing starvation. We contacted
WRG and they responded with food for us. Also we often make suggestions which WRG
24 WRG Community and Household Resilience, project progress report, October 2015.
32
accepts, for example, extra trainings and farming equipment” (evaluation FGD Galoole,
vocational training participants, female).
“We requested a fund from WRG which we received. We requested shoat distributions to
vulnerable households and we requested and received additional support with farm
preparation” (evaluation FGD Raybadka, Earthworks participants, male).
One of the factors that has contributed to this positive feedback is that during the first six months of
the project, 28 focus group discussions took place across the five villages with women and men on
resilience, women's involvement and environmental protection25. The feedback from these
discussions was key in the development of the project.
In terms of ongoing stakeholder feedback mechanisms, before the two-year Household and
Community Resilience project began, most feedback from individuals and communities happened
informally during or after project meetings. Now, in addition to speaking to WRG staff when they visit
communities, community members telephone individual staff members should they need to and there
are also regular visits from community members to the WRG office in Hargeisa. In addition, this
feedback, along with staff feedback, is recorded in the minutes of the weekly planning meeting and
helps shape and influence upcoming activities as well as long-term planning26.
Recommendations for beneficiary accountability systems
Although the project has provided the communities with information about the activities and been
responsive to any feedback and requests, there is not a specific, confidential feedback mechanism for
individuals in the communities to use should they wish to. This was planned initially; in 2014 a
confidential feedback telephone number was planned and established to allow project participants to
express feedback anonymously that they might otherwise be uncomfortable sharing in person.
However, this telephone number has not been utilized - although efforts were made by the project
team to promote it. Although members of each CBO and village leadership in each village report that
they feel they can give feedback to WRG and that this is responded to (as well as other various
members of the community), there is a risk that this may not apply or be accessible to other members
of the villages - and doesn't fall within the standard of a more formal and confidential feedback
mechanism that is accessible to all27.
Although the project staff are readily sharing feedback given to them directly or through their direct
mobile/cell phones, it is important from an accountability perspective to have a centralized way of
monitoring, managing and coordinating incoming communications and the formal responses from the
project.
Implementing a confidential, anonymous and fully accessible means of communication from the
partner villages to WRG is challenging due to several factors including; not everyone will have access
to a mobile phone, illiteracy may be a barrier to written communication, there may be gender barriers
and people may not wish to communicate outside of the village leadership/CBO. However, it is
recommended to consider confidential ways that populations can give confidential feedback to WRG,
25 WRG Community and Household Resilience, project progress report. March 2015 26 WRG Community and Household Resilience, project progress report. 27 http://www.chsalliance.org/
33
should they need to. This is important from an accountability point of view, as well as safeguarding
vulnerable individuals.
Experience of INGOs across a range of contexts (eg. Oxfam, Save the Children UK) has shown that
regularly and systematically promoting a telephone number and its purpose to communities can help
to increase usage. It is also important to stress that the telephone number is there for people to use
for several reasons, eg, suggestions, thanks, concerns or complaints. It is also key to highlight that any
feedback is confidential and will not have negative impact on them.
However, even though promoting a telephone number may increase awareness and uptake, it is often
the case that such telephone lines do not seem to a be popular mode of communication, it is
recommended to try other methods in addition to the telephone line. As some members of the
community are illiterate, suggestion boxes may not be workable.
Other methods that have seen success with other INGOs and national NGOs are to set up an area in
communities during distributions for one-to-one meetings. This does not have to be in a separate
room but just allowing for a private conversation. Again, the key is promoting the purpose and location
of this each time. It is important to keep maintaining and promoting the available, confidential
communications channels even if they do not seem to be utilized frequently because it may take time
for people to become used to them - and it is important that the lines of such communication are
open in the event that they are needed.
A further recommendation is to have a nominated project staff member to manage a notice board28
with the CBO in each village giving information about the project (in picture/symbol form as possible);
The dates and objectives/mission and activities of the project.
Who the participants are and how they are selected/who is welcome.
How individuals and groups can contact WRG and who is welcome to do that (eg. all).
A space for regular updates about what is coming up.
What will happen at the end of the project – future plans, date of next update.
A space on the notice board for the village to manage with their own updates.
28 http://www.chsalliance.org/
34
4. Efficiency 4.1. Were the objectives achieved on time?
4.1. Were the objectives achieved on time?
Scoring of the indicator by WRG project staff
Similarly to the criteria of ‘impact’, the WRG project staff scored efficiency during a final workshop. In
this case, seven out of seven staff gave a score of 3/4 for ‘good evidence’ of efficiency. The rational
for this was as follows:
FFs facilitators, forest guards, CAHWs, Earthworks, SHG and SFLG and vocational training were
cost efficient and according to the terms of reference for these activities.
Based on CRMP Implementation at village level in Raybadka, a grant was provided to
accelerate CRMP activities, such as preparing for fodder production.
There was a successful distribution of seeds for FFS and trees during World Environment Day.
Key output activities such as developing the CRMPs with the partner villages, fodder training,
Earthworks, conservation farming, loans for SFLG, seedling distribution on World Environment
Day were all on time.
When considering the results of the staff scoring of the OECD DAC criteria and the project progress
reports, many of the activities were achieved on time and efficiently. When looking at the overall
project, a six-month project extension was planned and implement covering the period April to
October 2016, continuing from the existing 18-month project.
The extension had the aim of strengthening the ongoing project components and addressing
challenges resulting from the periods of drought, in order to increase the project’s impact and
sustainability. The project identified that pastoralists and ago-pastoralists in Somaliland are
increasingly at risk of losing their livelihoods (primarily in their herds) and WRG and Tearfund extended
the project with the aim of continuing work with pastoralists in Somaliland to continue support to the
long-term objective to develop an environment, communities and households that are resilient to
seasonal changes, drought and climate change by 202029.
Some of the main for the extension were identified as being as follows:
The project faced due to frequent and longer droughts, as well as encroachment and
degradation of their grazing lands, which affected the implementation of activities, the
strategies of the villages and the overall timelines of the project.
There have been frequent times with key individuals at MoERD and other ministries have not been available to support the project due to their other commitments30.
29 WRG Community and Household Resilience in Rural Somaliland, Project Extension Report, February 2016. 30 WRG Community and Household Resilience in Rural Somaliland, various project progress reports.
35
Given one of the main approaches of the project is the building of community structures and
developing cohesion, it takes times to see the impact of the project and for people to
collaborate together.
5. Sustainability
5.1. What were the major factors which influenced the achievement or non-achievement of sustainability of the project? 5.2. To what extent did the benefits of a programme or project continue after donor funding ceased or are likely to continue without external funding?
5.1. What were the major factors which influenced the achievement or
non-achievement of sustainability of the project?
Scoring of the indicator by WRG project staff
The project staff also scored the criteria of sustainability in a final workshop. One out of seven staff
gave a score of 1/4 for some evidence (14%), and six out of seven staff gave a score of 3/4 for good
evidence (86%).
The justification for a score of one was a follows:
Components like CRMP, Early Warning System (EWS) and World Environment Day (WED) need
more time for the communities to take full advantage of, as these were new concepts to the
community.
The EWS component is ineffective due to lack of government capacity to facilitate and enforce
through NERAD.
For WED, there was only a small amount of evidence of the communities commemorating this
on WED.
The rational for a score of three was a follows:
Components such as Xeer could continue based on local context, as most communities respect
their local bylaws and it is something can keep in their mind.
Community preparation for dry season; there is growing evidence of activities for drought
preparation like fodder preparation/storage and community collaboration.
SHG, FFS, Earthworks and vocational training participants acquired enough skills and
knowledge so they can continue without external input, it depends on the communities to
continue these activities.
36
In addition to this staff scoring, the evaluation found that the communities place high value on
relatively sustainable elements of training, skills, knowledge and collaboration – and consider these
amongst their most important resources for increasing their resilience.
This, combined with the increasing challenge of increasing periods of drought, demonstrate that it is important to continue engaging community members through training to prepare and manage in the dry seasons, to continue to encourage collaboration, explore livelihood diversification opportunities and continue to build on their current, traditional livelihoods.
For example, in the three FGDs with Farmer Field Schools, when asked what resources the FFS had to continue with their strategies to prepare and manage in the dry seasons the main items listed were; land for farm preparation, knowledge and skills, tools, CAWS and the FFS facilitators. All of these areas do seem to be sustainable as long as the tools can be repaired or replaced if needed by the farmers, as well as their continued ability to pay for the CAHWs in the long term.
The FFS groups also recognised both the importance of community collaboration for crop production
and problem solving - and that they have an important role sharing information and ideas with other
community members. The role of the CBOs and the other groups (such as FFS, SHG) to interact with
other people in the community and share their knowledge is also indicated as being be crucial. The
importance of the role of these groups and their value should continue to be highlighted to these
groups to encourage their leadership and ability to give a good example to others (for example, sharing
this finding from this evaluation). Also to enhance sustainability, the communities will need to find the
means in the future to fund the inputs they need for their strategies.
Also, the focus group discussions and the household survey results showed that the partner villages
believe that the natural resources available to them can benefit all - and also that their loss can affect
all. The findings also indicate that there is a trend in the villages that the populations believe that only
together can they protect their resources and increase livelihood opportunities and ability to manage
in dry seasons – especially when these periods are becoming harder to manage due to shortage of
rains and increasing period of drought.
Overall sustainability may have been further enhanced by the changes in the balance of decision
making power between men and women, this is clearly reported to have become more balanced
between as a result of the project activities (nine of the 20 FGD groups reported and supported this
change – both males and females).
A specific activity area posing a risk to sustainability is regarding the CAHWs. They said during the
evaluation KI interviews that there sometimes households are unable to pay for the services they
provide. One of the CAHWs in Raybadka said: "my challenge is when I carry out work in the community
and cure livestock, the household borrows from me and promises to pay for the services later. Then I
have challenges buying medicine and continue the service for them". Another CAHW in Galoole said:
"the greatest challenge I face is the service charge debt as the community does not have the money".
This factor, combined with the potentially growing need for the services of the CAHWs (and their
indication that additional medicine and health kits are needed to meet this demand) may challenge
the sustainability of the CAHW services. In addition, the value of a simple EWS is recognised by the
WRG team to support the village populations to better plan and prepare for the dry seasons and
potential droughts. It is also worth noting that in 2011, the year of the last major drought in the region,
that 2010 gave no local clues to the upcoming drought - access to more detailed climatic information
37
was neededxi. Some recommendations regarding EWS based on input and ideas from the WRG project
team are provided in the recommendations section below.
5.2. To what extent did the benefits of a programme or project continue
after donor funding ceased or are likely to continue without external
funding?
Looking to the future, there are good indications that the benefits of project could continue in the
event that donor funding ceased, largely because of the strong emphasis of the project on developing
partnerships between the communities and other stakeholders (eg. MoRED) and strengthening the
CBOs so they can take the lead on activities. It is a case of continuing to invest in the structures that
have been put in place and building on skills and knowledge.
Currently the communities have asked for WRG to continue project activities, which is reflection of
the good relationships and coordination with the project villages. Introducing outcome and impact
indicators to understand more about where the project is achieving impact and what the enabling and
constraining factors are, as well as routine monitoring and evaluation of the OECD DAC evaluation
criteria during the project period. This will help to know where resilience is being achieved, what
factors are contributing to impact and sustainability - and where to invest with a view to CBOs and
other community structures taking on full implementation and accountability for the activities.
Some additional activities that would also increase sustainability and long-term impact have also been
highlighted from the different sources of data collected during the evaluation. This include;
Literacy and other skills for the savings groups.
Capacity building with young men and women for traditional and non-traditional livelihood
activities. Helping the communities to plan how they will fund farming inputs, developing the
Community Funds and the Emergency funds.
Sustained ways to fund the forest guards and household strategies to pay for the CAHWs.
Strengthening links with district authorities, ministries and development actors to support
initiatives such as EWS, Forest Guards, livelihood diversification and rangeland management.
Establish ways for the communities to continue to fund the inputs they need for farming
activities, Earthworks, sustainable homesteads, EWS, savings groups, business startups and
rangeland management and training.
The focus on train-the-trainer has been well received and respected by the communities;
future work could continue to build on this in all aspects of the project – with additional focus
on females and young men and women (for agro-pastoralist and pastoralist activities (such as
rangeland management) and vocational skills training).
In addition, for pastoralist systems in dryland areas in general, other studiesxii have consistently
confirmed that putting in place preventative measures and the ability to rapidly adjust is key to
reducing sensitivity to shocks such as drought. According to these studies, the focus should be less on
increasing productivity and more on management of elements including (not an exhaustive list);
Improving animal health services to maintain the mobility of animals between drier areas (for
breeding) and higher rainfall areas (for fattening).
38
Facilitating early destocking when drought is imminent and restocking when rains resume.
Additional feedback from the villages to WRG
In addition to the questions on the focus group discussion tools in Annex J, each of the 20 FGD groups
were asked: “do you have anything else you would like to add?” as the final FGD question. All of the
feedback to this question from the groups is provided in table 8 below. It is recommended to consider
this feedback in any future work, in terms of enhancing project impact, relevance and sustainability.
Table 8. Final feedback from the partner villages to the project
Response/feedback
Number of FGDs that gave the response
(some may be counted more than once)
“We are working together and are making an effort to improve our lives - but we continue to need your partnership”. “We welcome any new partnership/project“ (activities specifically mentioned were: WRG’s general coordination of activities, FFS, Berkeds, Earthworks (half-moons), irrigation improvements and provision of seeds).
10
More skills and training, especially vocational training for woman, literacy skills for SFL/SHG groups, training in income generation and support for agricultural inputs.
7
We appreciate the help of WRG and the new skills we have learnt – thank you.
3
We would like more information about how we, as a community, can support the more vulnerable households.
2 (Galoole, men in the
community FGD. Lebisagaala, CBO)
We recommend communal grazing land improvement is a better way to get enough pasture during the dry seasons.
1 (Lebisagaala, SFL group)
We request from WRG plastic sheets for farming and water preservation.
1 (Ina Igarre, FFS)
We recommend to strengthen the relationship between our community and WRG.
1
To expand the number of women in the SFL group. 1 (SFL group, Raybadka)
39
Livelihood diversification
The project also had a focus on livelihood diversification; for example, agricultural intensification in pastoralist villages, small business startup and vocational training in areas including tailoring, henna decorating and cooking. To support sustainability (and impact), this report also recommends other potential ways of diversifying livelihoods to try to mitigate the current coping mechanisms. Throughout the evaluation process, several recommendations were made for ideas for livelihood diversification in the partner villages, these include:
Skills in electricity and carpentry (recommended by the WRG team).
Rangeland protection work which members of the community could be engaged in - with support from MoRED to say which land can be used (recommended by MoRED and a previous assessment of the projectxiii).
Bee keeping (recommended by the Ministry of Agriculture (MoA) as a low input initiative – they said that they also currently planning to pilot bee keeping in rural areas – it is recommended to review if the project could be part of this pilot31).
Growing different types of vegetables, fruits and staple products. This may be a natural starting point for poor, rural producers with low levels of capital to change their production mix rather than invest in a non-agricultural livelihoodxiv. It is recommended to continue the exposure visits to farmers that are successfully growing other types of produce for their own consumption or sale – this may help other farmers to adopt the same practices.
Crop-livestock integration to help maintain fertility through the incorporation of animal litter into the soil.
Destocking healthy livestock before the dry season begins (for a good price) and investing in other types of animals that need less water and feed to maintain (eg. chickens). Given the value traditionally placed on shoats, communities may need to be convinced of this, perhaps seeing for themselves an example of success32.
Considering other types of animal product.
Following this, it is recommended to assess how the different livelihood strategies could complement
one another, as rural producers make their way in what is often a risky and resource-poor
environment. There are two reasons for this:
An increase in the number of diversification activities undertaken does not necessarily mean
that livelihoods are becoming more sustainable. Poor farmers who increasingly engage in off-
farm activities may rather be doing so as a long-term adaptation to stress, shocks and poverty
– these farmers are trying simply to survive in a poorer, riskier world, rather than to improve
livelihoods and invest in productionxv.
This view of adaptation can be disputed, however. Practices and adaptive strategies may be
positive, functional, sustainable, and thus lead to sustainable livelihoods. Or they may be
dysfunctional (eg. if local markets do not support the livelihood), leading to non-sustainable
livelihoods and depletion of the environment or external supportxvi.
31 Key Informant Interview, Ministry of Agriculture, October 2016 32 Recommended by WRG Somaliland Country Representative.
40
In terms of a solution, overall this is to tailor livelihood diversification strategies to the particular
context. This evaluation also learnt the following:
One of the issues raised in the final evaluation workshop with the WRG team. They thought that
agro-pastoralists farmers are often left without a livelihood activity during the dry seasons.
Livelihood diversification may help to address this issue - but consider if the livelihood activity is
replacing or reducing traditional and potentially productive activities.
Evidence from this evaluation indicates that pastoralism is currently facing challenges for the
partner villages for a variety of reasons (including overgrazing of some areas and shortage of
rains). Initiatives to address these issues, such as holistic rangeland management, may be a better
source of livelihoods for households. This could also form the foundation of inter-village
cooperation and sustainably improved livelihoods. This has the potential to address other
challenges to resilience including illegal deforestation, charcoal production and qat – by giving
people an additional collective purpose and also a further way of managing natural resources to
support livelihoods.
In addition, the issue of migration out of the villages (eg. by pastoralists or those seeking work in
Hargeisa) is mentioned by the project and by the communities during the evaluation focus group
discussions. However, the project team has also realised that that this is very much part of these
communities, especially for pastoralists, and not easy to control. The main impact of the project
from the perspective of the team is more about household not operating on their own any more
and an increase in the collective knowledge - this will decrease vulnerability33.
This does not mean to say that alternative livelihoods and vocational skills should not be invested in by WRG in partnerships with the villages – but as the project has been doing, it is recommended to continue strengthening traditional livelihoods, alongside other initiatives such as carpentry, electricians34 and training specific for young people to provide additional livelihood options. As noted above, this is especially important for the months when community members are in the villages unable to farm (due to the season) and need to boost their household and community livelihood security. It was also highlighted in the focus group discussions that there is a trend of young people chewing qat who could benefit from investments in training in vocational skills and other trainings. Overall, research into the area livelihood diversification for dryland regions is not conclusive. However, there are some key questions that are recommended to consider when promoting and facilitating alternative livelihoods - and may help decision making in this area. There are summarised in the box below.
1. Does the new livelihood activity risk the depletion of natural or human resources? 2. Does the new livelihood activity reduce or partially replace other long term and important
productive livelihoods such as farming, pastoralism or other traditional sources of income? If yes, are there other strategies that could be considered to improve these traditional farming or pastoralist etc activities? This is not to say that the new livelihood activity cannot be included but in a way that compliments the traditional activities as far as possible.
3. How sensitive is the new livelihood activity to drought? Households or groups adopting the new activity may be able to prepare for the dry season and manage today – but if the activity is going to be negatively impact by shortage of rains/droughts, they may move into poverty later on.
33 Evaluation Key Informant Interview with WRG staff. Programme Manager. 34 Carpentry, electrics and specifically considering young people were suggested as potential vocational skills areas by the WRG team in the final evaluation workshop.
41
4. Is the new livelihood activity likely to receive internal support from the household, village community, CBO and village leadership?
5. Are all relevant groups able to access the training and livelihood – eg. are young people included?
6. Could the livelihood activity compliment the overall strategy of the project? For example, this evaluation found that young people suffer from high unemployment rates in the communities and are also more likely to find ways to cope such as chewing qat and charcoal production. A livelihood strategy could address these issues, eg. rangeland protection initiatives, including young people being part of the related decision making structure on environmental protection and the community Xeer on deforestation (perhaps a committee for young people led by youth ‘champions’ on these issues, with the approval of village elders). Other, newer vocational skills could also be appropriate following consultation with young people and a review of the local markets.
7. Is the new livelihood activity likely to receive external support from local, district or national (ministerial) authorities or, if relevant, from local or regional markets? Can active support be guaranteed?
Complementary activities
In addition to enhancing sustainability and impact, it is recommended to consider some complementary activities to be included in this project (or delivered by another agency in coordination or partnership with WRG). For the foreseeable future, most of people living in drylands in east Africa will continue to make their living from herding and farming. The emphasis of the project in improving the ways the communities and households manage these livelihoods, combined with the complementary activities of promoting livelihood diversification is improving the resilience of the populations in the five partner villages. It is also widely recognized that, as well as livelihood diversification, complementary activities such as education, health/hygiene knowledge and family planning within projects can have a major influence on improving resilience and sustained livelihoodsxvii. In the case of family planning and birth spacing, these are considered, generally, part of the development agenda in terms of women’s ability to engage in productive activities. Other factors to consider are the effect on population growth, household wellbeing and access to and availability to natural resources. These factors have the potential to represent a slow onset shock that should not be ignored in resilience programmingxviii. These complementary programming activities would ideally be in combination with governments and nations strengthening financial services, focusing on job creation and developing social protection programmes – however these areas are likely to meet many constraints in this contextxix.
42
6. Relevance 6.1. To what extent are the objectives of the programme still valid? 6.2. Are the activities and outputs of the programme consistent with the intended impacts and effects?
6.1. To what extent are the objectives of the programme still valid?
Based on the feedback from the communities listed, of the 20 evaluation focus group discussions, the following information was volunteered by the groups:
Feedback Number of groups
“We are working together and are making an effort to improve our lives - but we continue to need your partnership”. “We welcome any new partnership/project“
(activities specifically mentioned were: WRG’s general coordination of activities, FFS, Berkeds, Earthworks (half-moons), irrigation improvements and provision of seeds).
10
More skills and training, especially vocational training for woman, literacy skills for SFL/SHG groups, training in income generation and support for agricultural inputs.
7
This indicates that the populations of the villages believe that the activities of the project are relevant to help them to improve their lives. The specific skills that have been shared during the project are still in demand to be built upon including; savings groups, income generation, sustainable farming practices, Earthworks and irrigation. The physical inputs that have been distributed were mentioned by the majority of the FGD groups such as seeds and agricultural inputs. This feedback could be included in directing future work and how to continue these activities and the availability of the inputs in the long-term, in a way that does not reply on donor input.
6.2. Are the activities and outputs of the programme consistent with the
intended impacts and effects?
There are several examples that demonstrate how the activities and outputs of the programme are consistent with the intended impact and effects. Regarding the contribution of the project to the communities growing in their ability to protect and restore sustainability improved livelihoods (outcome 2), almost all of the 20 overall FGD groups and the KII interviews in the communities attributed the preparation and management activities to the project, both in terms of knowledge, skills and training and also the emphasis on collaboration and working together. Also, four focus group discussions were conducted with groups men and women in the communities who are not part of a formal activity (in Galoole, Raybadka, Balimataan and Lebisagaala). This was to gain information about the wider impact of the project outside of the planned activities. Each of these four groups (two female and two male groups) stated that before the project began two years ago,
43
their community did not know how to prepare for the dry seasons and droughts, however, now they have learnt different ways to manage. With a clear link to the CRMP, they also said that new skills and training in areas such as fodder preparation and storage was a main factor for this, as well as collaborating on activities. Similar responses were given in the FGDs with those participating directly in the project’s activities, such as FFS, SFL/SHG groups and Earthworks. Further to this, according to the household survey, 78% of the households across the five villages had started a new agro-pastoral or pastoral techniques in the last year. This may be due to direct participation from initiatives such as Earthworks and FFS, or the transfer of knowledge. Also, all of the twenty focus group discussion groups described how they are engaged in fodder production, which may account for these results.
External factors influencing results
In addition, there are other, external factors that are indicated as positively influencing resilience or coping strategies during the dry seasons, although the extent of these strategies are not reported on in this evaluation it is useful to note their possible influence on the resilience of the communities;
Support from the relatives – this is mainly indicated as being from relatives based overseas35.
Anecdotal/unconfirmed accounts of distributions of food aid from other NGOs.
35 Evaluation Focus Group Discussions. Seasonal timelines.
44
Conclusion
Programmes aiming to improve rural livelihood activities have the potential to reduce vulnerability and enhance the resilience of populations living in the dryland, rural areas of Somaliland. During the two-year project, WRG has been working in partnership with the five villages to protect, improve and sustain existing livelihood activities, as well as facilitating new livelihood activities. In addition, WRG is engaging policy makers at the Ministerial level to support facilitate and coordinate elements of its resilience project. This approach will enhance the sustainability of the project and the success of the different outcomes. The partner villages have faced several significant challenges that are out of their control – and out of the control of the project. For example, the insufficient Dehyr rains (May 2015 onwards) and overgrazing in October 2015 in the relevant districts not only left an already depleted rangeland barren, but meant that tomatoes, fodder and other crops did not grow well and left communities vulnerable for the approaching dry season. In addition, the rains did not come as expected in October 2015. Following this, the period between November 2015 and March 2016 was affected by drought. These factors contributed to food insecurity and desertification. They also affected the work of the project in terms of village and groups meetings, as well as general coordination. In addition, a poor livestock market combined with the rain shortages has also affected the partner villages. Individuals have described how they and others are moving further away from the village to find pastures, with several describing themselves as 'displaced' at these times.
What has happened as a result of the project?
In the face of the climatic stresses and shocks that the communities continue to face, as a result of this project there is strong evidence that the following has taken place in all of the five partner villages (with evidence of positive change since before the two-year project began):
Increased preparation activities for the dry season due to increased knowledge and skills from the project activities.
Increased collaboration in the community and sharing ideas on how to prepare for the drought.
Increased soil and water conservation due to increased knowledge, skills or farming tools36.
That the project purpose has been fulfilled; “that the five project villages are better equipped to cope with and adapt to drought and seasonal changes”.
In addition, the activities and structures initiated by the project have become the main preparation activities that the communities undertake for the dry season. During the October 2015 rainy season, the main preparation activities for males were recorded as being; training through FFS or Earthworks, farm preparation, fodder preparation, water harvesting (with plastic sheet below soil) and generally continuing with CRMP strategies.
36 Results of evaluation impact indicator ranking.
45
The main preparation activities reported for females in October 2015 were the same as the men, with some additional activities. These included each female group (mainly SHG or vocational training participants) also said savings, savings for food and generating an income37. There is also a wealth of evidence from the focus group discussions and household survey that the role of women in decision making is increasing and that the project has contributed to this. In some villages, women are now being seen in leadership positions in the CBOs, for example, in Galoole. In terms of the overall approach of the project, there is a trend across the communicates that there is a positive change in attitudes to the importance of preparing for the dry seasons together38 – these strong social structures that can continue to be strengthened and built upon. Following this, when considering the individual project outputs, most of the outputs have been met and have demonstrated that they can be sustained according to the current project strategies; especially regarding elements of the CRMPs in each village such as FFS, SFL/SHG groups and Earthworks. There now there is the potential to strengthen sustainability of some the other output areas and activities, according to the context in each village. For example; the CAHWs, the impact of the Xeers, the impact of qat awareness campaigns the Forest Guards. There is also now the opportunity to strengthen the links between MoRED and other actors with the villages, especially in support of advocacy opportunities (eg. for Forest Guards and other support) and in the areas where the ministries have expressed interest. MoRED has expressed interested in supporting a pilot for holistic rangeland management that could address several needs including environmental protection and livelihood opportunities. The Ministry of Agriculture has also said they are able to provide information regarding bee keeping, which is relatively low input39. However, the limited resources of the ministries are likely to continue to pose a challenge to be factored into project design.
What real difference has the project made to the project beneficiaries?
In terms of the impact of the activities on the five partner villages, there is both tangible and subjective evidence of impact. When looking at outcome 1, that the communities are intensively planning, preparing and cooperating among women and men for DRR, there are several examples of this. For example, there is clear evidence that the main preparation activities for the dry season are linked to the project and the CRMPs (sustained farming and irrigation preparation due to the FFS and Earthworks, communal fodder cultivation, SFL/SHG groups and indications of communal farms). The communities attribute this to the knowledge gained during the project. Almost all of the respondents said that they did not collaborate in this way before the project began or did not realise why it was important before the project, whereas now they do40. Also, on average, 97% of households across the five villages said that they had experimented with new flood protection or erosion control techniques in the past year. Although the impact of this on the households and the communities has not been determined, this indicates the success of the elements of the project that focus on these activities - and that knowledge may be shared beyond the trainings
37 Results of evaluation seasonal timelines. 38 Results of evaluation focus group discussions. 39 Results of evaluation KIIs with ministries. 40 Results of seasonal timelines, FGDs, KIIs.
46
and workshops41. In terms of the SFL/SHG groups, other women have come to observe the SFL/SHG groups and in some cases started a second group42. As described in the methodology section, subjective evidence of impact is a relatively new but effective way to assess impact – especially for resilience programming. It relates to the way that communities perceive their own resilience. The evaluation found several examples including; a notable increase in equitable decision making between males and females with 75% households across the five partner villages say ‘yes’, there is equitable decision making between husband and wife about property/assets (61% of the respondents who said ‘yes’ were male, 39% were female). The majority of the 20 FGDs also reported this and that this had changed since before the project began. The household survey also showed that 76% of the households across the five villages believe that there is a system of social support in place if they lost their property/assets. Of these households, 69% believe that the CBO would give support, followed by relatives and friends (20% of households) and the village committee (11% of households). Given that the CBO was not in place in a small number of the villages before the project began, this may be an indication of the work the project has done to strengthen the role of the CBO in each village. There is also significantly increased collaboration in the community and sharing ideas for how to prepare for the drought – also attributed by the communities to the project43.
The extent to which objectives have been achieved
In addition to considering what has happened as a result of the project and the impact, each output has been evaluated in Annex A (with achievements per impact, with enabling and constraining factors). This shows that the majority of the planned outputs have been fulfilled, especially where there has been a strong sense of ownership by the communities (facilitated by strategies such as train-the-trainer, the CRMPs and strengthening the CBOs). These include the activities that fall under the CRMP such as the Farmer Field Schools, SFL/SHG groups, Earthworks and communal fodder production. In addition, Community Animal Health Workers, Xeer agreed and in place, World Environment Day and tree planting. The outputs that have not been met in full have faced several challenges. Although some of these challenges have been largely out of the control of project they should be possible to mitigate if future project work is designed taking them into account.). Where outputs have not been met in full there is now the opportunity to invest further in these areas, some of which may depend on support and collaboration with external stakeholders. Also, building on the strong community structures and knowledge base that have been developed during the project period. The summary of factors contributing to the achievement and non-achievement of these outputs can also be seen in Annex A.
Contribution of the project to impact
There are several examples that demonstrate connections between the project activities and the impacts being seen. Regarding the contribution of the project to the communities growing in their ability to protect and restore sustainability improved livelihoods (outcome 2), almost all of the 20 overall FGD groups and the KII interviews in the communities attributed their preparation and
41 Results of the household survey. 42 Information in project progress reports. Results of evaluation KIIs with WRG project staff.
43 Results of evaluation FGDs and impact indicator ranking exercise.
47
management activities to the project, specifically because of the knowledge, skills and training they had acquired and the emphasis on collaboration and working together. Also, four focus group discussions were conducted with groups men and women in the communities were are not part of a formal activity (in Galoole, Raybadka, Balimataan and Lebisagaala). This was to gain further information about the impact of the project with indirect beneficiaries. Each of these four groups (two female and two male groups) stated that before the project began two years ago, their community did not know how to prepare for the dry seasons and droughts, however, now they have learnt different ways to manage. With a clear link to the CRMPs, they also said that new skills and training in areas such as fodder preparation and storage was a main factor for this, as well as collaborating on activities. Similar responses were given in the FGDs with those participating directly in the project’s activities, such as FFS, SFL/SHG groups and Earthworks. Further to this, according to the household survey, 78% of the households across the five villages had started a new agro-pastoral or pastoral techniques in the last year. This may be due to direct participation from initiatives such as Earthworks and FFS, or the transfer of knowledge. Also, all of the twenty focus group discussion groups described how they are engaged in fodder production, which may account for these results. It would be useful to follow up on what the specific new agro-pastoral or pastoral techniques are, in order to share experiences with other members of the communities. In addition, there are other, external factors that are indicated as positively influencing resilience or coping strategies during the dry seasons, although the extent of these strategies are not reported on in this evaluation it is useful to note their possible influence on the resilience of the communities. This includes support from the relatives – this is mainly indicated as being from relatives based overseas and anecdotal/unconfirmed accounts of distributions of food aid from other NGOs44.
Factors contributing to the achievement and non-achievement of objectives
A key factor contributed to the achievement of the project’s objectives is the knowledge that WRG has imparted through train-the-trainer initiatives, saving groups and CBOs. Specific areas of knowledge that have been particularly impactful include fodder production; the importance of harvesting fodder during the rainy season (and not waiting for the dry season as some of the communities were doing before) was emphasized by WRG during the trainings45. This overall approach of training community based facilitators and increasing the accountability of the CBOs for the CRMPs has also contributed to impact and sustainability. The low-cost inputs provided have also been important for some activities such as sustainable farming and Earthworks46. The structure approach of the CRMPs has also been an important facilitating factor. The household survey shows that across the five villages, 84% of households (with mainly the head of household being the survey respondent) are aware of the CRMP in the community. The framework of the CRMP seems to have given the communities a sense of ownership and a structure to the activities. Long-standing relationships and partnerships with the communities has also shown to be a facilitating factor. The CBOs were more easily able to mobilise the communities in Galoole and Raybadka for the CRMP activities – WRG had had a long relationship with both of these villages and this good partnership undoubtedly facilitated the process.
44 Results of evaluation FGDs and KIIs with the communities and the household survey. 45 Results of evaluation KIIs with WRG staff. Hodo Abdilahi Hussein, Community Mobiliser 46 Results of evaluation FGDs and KIIs with the communities and the household survey.
48
In addition, the high value that the communities place on knowledge and skills has contributed to the impact that the project wanted to achieved, as well as increased sustainability. Focusing on women and increasing their potential to increase their household security through cooperative groups has also contributed towards increasing the role of women as decision makers. In terms of the overall approach of the project, an enabling factor to impact is the high standard of the WRG team to listen to the communities. This has included from the initial design and planning of the project, responding to feedback on an on-going bases and ensuring that the villages have the information they need to conduct resilience building activities. For example, the training for Earthworks and soil/water conservation was adapted according to the communities. Lastly, the emphasis WRG has placed on developing community structures has been key, such as strengthening the role of the CBOs and encourage collaboration between households and groups. When asked, each evaluation focus group discussion group and key informant interview across the five villages described how they are working together as a community, compared to before the project began two years ago when they worked more individually. For the factors contributing to the non-achievement of objectives, the evaluation found these to be mainly external to the project. According to all 20 focus group discussions and the community KIIs with vulnerable households (two interviews) the main challenges faced by communities are:
Shortage of rains/recurrent period of drought affecting strategies and resulting in people migrating out of the villages.
Insect and animal pests damaging farms.
Illegal deforestation for charcoal that affects soil conservation.
Water scarcity.
Shortage of income.
Lack of pasture.
The shortages of rain impacts on the ability of people to save, produce sufficient fodder and was reported to reduce household food security to such an extent that malnutrition was present in the villages (the proportion of people that were affected isn’t confirmed in this evaluation). The household survey also confirmed these findings. 89% of the households believe that drought/short rains is the main issue that affects them. This was followed by deforestation with 37% of the households saying that this was the second biggest hazard they faced after drought.
Sustainability
The project has had a strong emphasis on developing partnerships between the communities and
other stakeholders (eg. MoRED) and strengthening the CBOs so they can take the lead on activities. It
is recommended to continue to build on these activities and, as far as possible, develop regular and
systematic communication between stakeholders and the communities, strengthen links to other pilot
projects in Somaliland (eg. ministerial pilot projects and programmes), the Savings for Life Group
movement in Somaliland, EWS and other initiatives that have the potential for collaboration across
and between communities (eg. rangeland management). Looking to the future, this could then shift
the balance of accountability further towards the community structures such as the CBOs, FFS, SFL
groups and others, as well as towards the partnerships developed with external stakeholders.
49
In addition, some of the project activities have relied on low-cost inputs for their implementation, for
example, farming tools, seeds, plastic sheets, CAHW health kits and medicines, micro-credit for SHG
and teaching materials. It is recommended to include objectives relating to this in any future work,
such as WRG supporting the communities to develop strategies to continue to prioritise and plan to
fund these inputs (eg. through collective funds).
The evaluator also concluded that the project is likely to benefit from further assessment to support
the project’s strategy for livelihood diversification (please see page X) and the addition of
complementary activities (please see page X).
Measuring impact and theory of change
In terms of measuring impact and M&E systems, several recommendations are made in the recommendations section below to build on the existing systems. For example, in addition to the project output indicators, it is recommended to develop outcome indicators to support future M&E work. For example, if the following outcome were to be included “improved resilience of beneficiaries using project outputs”, then an outcome indicator could be ‘the number of households with improved resilience according to a pre-defined resilience index”. An example of such an index is in the table below47 (which could be adapted, built upon or combined with the CLEARINGS tool indicators):
Attribute Yes No
Household adopted one drought resistant crop on >1/4 hectare
1 0
Access to micro-irrigation on >1/10 hectare 1 0
A family member in a savings group 1 0
Current savings >$20 1 0
Have used a weather forecast in the last two years to decide when to plant
1 0
Such indicators could then feed into an updated, overall theory of change as follows (that could then be disaggregated by village, numbers, age groups (including young people) and female/male).
47 Mills and Garama for the UK DFID (2014) Assessing the impact of ICF programmes on households and community resilience to climate variability and climate change <https://assets.publishing.service.gov.uk/media/57a089b840f0b64974000222/EoD_Consultancy_June2014_Measuring_Resilience.pdf>
50
Figure 2. Suggested overall theory of change flow chart
*Well-being refers to a range of criteria including human wellbeing
(sufficient food (eg. HH food security status), sufficient safe water, safe
sanitation, education, health life, gender equality, income, population
growth, good governance); environmental well-being (biodiversity,
renewable water sources, sustainable consumption of natural resources,
energy use, energy savings, renewable energy); and economic well-being
(organic farming, genuine savings, gross domestic product, employment,
debt levels)48.
48 Sustainable Society Foundation <http://www.ssfindex.com/>
Project inputs:
Training in sustained farming
practices via the FFS.
Training in Earthworks
(irrigation systems).
Training of CAHWs
Vocational training depending
on assessment of
complementary skills that are
likely to complement
traditional activities and be
sustained.
Training young people to be
‘champions’ for not chewing
qat and embarking on
initiatives.
Loans from WRG for savings
groups, farming practices
irrigation, rangeland
management.
Capacity building for planning
and purposes of community
funds (eg. for farming inputs,
livestock medical bills,
emergency food, Earthworks,
rangeland management,
forest guards).
Savings training (for SFL/SHG
and more?)
Numeracy and literacy skills
for savings groups.
Capacty building in EWS,
reading weather forecast
symbols/information.
Project outputs:
New crop options
Micro-irrigation
Savings groups
FFS and Earthworks
groups
Vocational training
Weather forecasts
accessible and
available
Improved livestock
health
Improved
household wealth
Improved
household food
security
Outcome:
Improved
resilience of
beneficiaries
adopting/
using project
outputs
(using
resilience
criteria, eg. in
the table
above in
conjunction
with
CLEARINGS
indicators).
Impacts =
improved
beneficiary
well-being* by
2020, despite
experiencing
shocks and
stresses.
51
Monitoring and Evaluation
The project has a strong M&E framework which is well managed. The project team are also included
in the regular review of activities and impact. However, the framework currently includes mainly
measurable indicators relating to the project outputs, reflecting the project logframe. It is
recommended to link M&E more closely to a theory of change model and impact and outcome
indicators.
In addition, it is recommended to include a plan to measure the progress and impact of impact,
outcome and output indicators (eg. indicator target, source of monitoring data, frequency of data
collection and who is responsible). This will assist understanding of how the project is increasing the
resilience of the communities despite the ongoing hazards they face – and enable greater potential
for the impact and sustainability of the activities - and community ownership in the long-term.
To increase the impact of M&E on the project, it is also recommended to inclue in the framework the
OECD DAC evaluation question and to routinely monitor these with the project team (eg. every three
months), including the enabling and constraining factors to each criteria. A plan for one-off internal
and external evaluation events could also be included and an outline of the beneficiary feedback
system to receive and respond to feedback – including a record of the specific pieces of feedback and
the responses provided by WRG. Overall, a system including these elements will contribute to the
achievement of project impact, quality and organizational learning (Nb. a system for Monitoring,
Evaluation, Accountability and Learning - MEAL).
To increase sustainability, training could be provided to the CBOs on M&E and beneficiary feedback
systems in order to plan for long-term when there may not be an external organization partnering
with the villages.
Recommendations
- What lessons learned were expressed or inferred from surveys or discussions with stakeholders? - Assuming that this project will be modified, expanded and renewed in the period from 2017 through 2019, what recommendations for improvement can be made based on comments from stakeholders, analysis of data, and observations from field visits or personal experience, to make the programme more relevant, effective, efficient, coherent, impactful, and sustainable going forward?
Community Risk Management Plan:
Continue plans to link Earthworks to the Community Fund in the three pastoralist communities (and plans to establish a Community Emergency Fund in the two agro-pastoralist villages).
In any future work with the villages, include an objective relating to WRG supporting to communities to develop plans and strategies to continue to sustainably fund the inputs needed to continue the activities. Inputs have included farming tools, seeds, plastic sheets for water conservation in the soil, CAWHs health kits and medicine, micro-credit to begin SHG saving and initiatives, training materials and transport costs. There is also the need for households to pay for
52
the services of the CAHWs. A Community Fund may be suitable for this and has already been tried and tested in the villages.
The Community Funds have had particular success in Galoole village, in terms of the numbers contributing. It is recommended to apply specific experiences from Galoole to other communities, tailored as needed.
Review a possible strategy for holistic rangeland protection management in conjunction with other stakeholders and possibly other communities (please see page 19 for more information).
Some of the FGDs groups mentioned in the development of the seasonal timelines that they sold livestock at the start of the dry season. It is recommended to review further such destocking activity and if this is an informed planned activity or an emergency coping strategy that could put households at risk.
According to the household survey, 78% of the households across the five villages had started a new agro-pastoral or pastoral techniques in the last year (increased fodder production may account for these results). However, it would be useful to follow up on what the specific new agro-pastoral or pastoral techniques are, to share experiences with other members of the communities.
Self Help Groups:
Based on the success of farmers’ exposure visits to successful farms, it is recommended to facilitate SFL/SHG exposure visits to other villages to share ideas and experiences.
Consider including training in literacy, numeracy, accountancy and project management skills – some of the groups said they needed these skills.
Review other potential areas for vocational training with females, men and young people, as well as train-the-trainer initiatives.
In the household survey found that 21% of the respondents said they had started a new business in the last year, which did not exist in the community before. Although a relatively small percentage, the main types of businesses described was tailoring (in Galoole and Raybadka only). As WRG carries out vocational training in tailoring in Galoole and Raybadka, this may indicate that there has been update of this skill, resulting business activities. It is recommended to continue to follow up and evaluate the results from the vocational training to see if any experiences can be shared with other members of the community (and to inform programming).
For the same reason, it is also recommended to follow up with small businesses such as tea shops, as well as the other businesses that have been set up by farmers and the SFL/SHG groups.
It is recommended that indicators be developed to measure and evaluate the impact of the SFL/SHG groups on household food and economic security.
Earthworks
The household survey found that the majority of households in the five partner villages have
experimented in new flood protection or erosion control techniques in the past year. This indicates
the success of the elements of the project that focus on these activities - and that knowledge is
shared beyond the trainings and workshops.
Some of the focus group discussion groups described that they are now working together in
communal farms. It is recommended to review these communal farms to see if they could be
further expanded/supported and become a specific project objective. Also to review if any
experiences could be shared with vulnerable households and the households in the more remote
areas of the villages.
Farmer Field Schools:
53
Review the inputs needed for sustainable farming activities and consider supporting the farmers to plan for how they will continue fund those items in the future.
There was a trend in the male FGD groups (eleven in total across the five villages including the CBOs) to cite insect and animal pests damaging farms as an issue – recommended to review this problem with the farmers in each village and ways current practices could be built upon.
According to the household survey findings, only a small proportion of the village communities knew that the Farmer Field Schools were part of the CRMPs. It may be worth considering some further ways to extend the reach of the FFS beyond those directly attending the trainings, for example, the knowledge transfer to more remote households, building on the communal farms or more exposure visits by the general community to successful farms/homesteads.
Xeer and Forest Guards:
It is recommended to discuss with CBOs and village leadership their experiences of supporting members of the community to stop chewing qat. These experiences could be built on and there may be the possibility to share experiences across the villages or establish support groups.
It is recommended to continue to connect the communities with MoERD regarding some of the key areas including the funding of Forest Guards, the formal registration of Forest Guards (eg. with the district authorities), rangeland protection and the use of resources by different communities.
As the ability of the ministry to fund forest guards may be a challenge, it is suggested that WRG assess other options to fund them with the partner villages.
The Forest Guards who were interviewed said that continued awareness raising is needed about the value and management of the forested areas, as well as the Xeer.
Community Animal Health Workers and animal health:
Continue to improve animal health services to maintain the mobility of animals between drier areas
(for breeding) and higher rainfall areas (for fattening).
Follow up trainings to update the knowledge of the CAHWs – and consider expanding to the other
partner villages49.
Continue to engage with the Ministry of Livestock so the ministry could work with them and ensure
they have access to the drugs they need50.
Assess the potential of training and facilitating early destocking when drought is imminent and
restock when rains resume.
Early Warning System:
It is recommended to plan two or three options for a simple EWS - and who the different agencies and individuals would be in each potential option (eg. MoRED, Salale district office, CBOs, FAO, coordinating information with the coordination forums).
The household survey element of the evaluation found trends of the most used information sources for early warning to drought, with the CBO being the most used followed by the village committee and then the village leader. Based on these findings, it is recommended to focus on these structures in the development of an early warning system with the partner villages.
Consider the role that national NGOs or the CBOs could play in collating information about weather forecasts and climate – and sharing this with the communities (eg. using community meetings and a community notice board using pictures and symbols to represent the information where possible).
49 Evaluation Key Informant Interview with WRG staff. Mohamed Abdirahman Hassan, Programme Manager. 50 Ibid
54
A key challenge to establishing a simple EWS with the partner villages is the need for an agency to monitor climatic and other information from SWALIM and MoERD and communicate this to the CBOs or leaders in the five villages. It was identified that an established national NGO could be ideal to fulfil this role – but an issue is that they do not have sufficient capacity in terms of human and IT resources51.
Following from the point above, it is recommended to assess options for increasing the capacity of a national NGO to fulfil a monitoring and communications role in an EWS – or consider additional resources in the WRG team for this role.
Contact SWALIM to see if they might be able to provide some training to WRG/other stakeholders on interpreting the drought monitoring and forecast information they provide.
Another challenge in the project was the budget – this planned approximately $300 USD for an EWS52.
Pilot with vulnerable households:
Review the situation for the households that have had some success to see what the facilitating
factors have been, as well as the challenges they still face.
Consider things that do not go well, as well as things that go well, as part of the pilot learning.
Consider how the knowledge gained could be expanded to other households and communities – and
shared with other actors and coordination forums.
Review the budget and general strategy for the pilot with vulnerable households.
Examples of learning that could be built upon from the pilot with vulnerable homesteads, this includes:
Crop rotation (by changing paddock areas around) and shoot fattening was considered to be
successful by WRG.
The distribution of livestock to the households, plus an allocation of land and water and the
commitment of the households to care for the livestock seems to be successful - and was certainly
appreciated by the households. One household in particular has been really successful in managing
the shoats53.
The living fence was not possible implement in the conditions. It is recommended to consider if
there is an alternative way to implement this (as the project has identified they have several
benefits including acting as an animal enclosure, providing fodder, preventing soil erosion and they
can also have a role in pest reduction).
It was not clear if there would be enough fodder for each household in the pilot to sustain the
livestock through the dry seasons.
The households interviewed also reported the following challenges; recurrent drought and water
scarcity, low market price of shoats, lack of pasture, livestock diseases.
Coherence and accountability to the partner villages:
Consider a community notice board to share key information from WRG and the CBO, as well as
sharing the findings of this evaluation with the communities.
Review ways to re-establish a confidential, anonymous means of communication from the villages
to WRG accessible by all.
51 Evaluation Key Informant Interview. Abdirizak Osman Askar, M&E Officer 52 Evaluation Key Informant Interview with WRG staff. Mohamed Abdirahman Hassan, Programme Manager. 53 Evaluation Key Informant Interview with WRG staff. Mohamed Abdirahman Hassan, Programme Manager
55
Livelihood diversification:
Consideration of other potential ways of diversifying livelihoods to try to mitigate the current
coping mechanisms (such as migration out of the village to rangelands or to Hargeisa). Assess which
area are most likely to gain support from other stakeholders such as ministries and districts, as well
as the market for such livelihoods.
If livelihood diversification will continue to be a strategy for the project, it is recommended to
follow up with Igarre, Balimataan and Lebisagaala (where the household survey showed that higher
proportions of the communities are engaged in business activities) to find out more about the
factors that enable business activities in these three villages. Also, to see if any experiences can be
shared with Galoole, Raybadka and other villages.
Complementary activities:
Consider if any additional complementary activities could be integrated in to the project to enhance
the positive impact on resilience or if collaboration with other actors in Somaliland is an option (eg.
education, health knowledge and services, family planning information). Please see page 43 for more
details.
Several of the FGD groups from different villages said that wished to learn about ways to filter water
from the berkeds for drinking.
Monitoring and Evaluation:
It is recommended that the key findings be shared with the communities in the five partner villages, as well as other stakeholders such as ministries and other officials.
The project has had a strong system for monitoring progress and evaluating this in the weekly
coordination meetings. In addition to this, it is recommended to hold regular in-depth project review
meetings with the team (eg. every three months) to discuss progress against objectives, any
enabling/constraining factors.
It is also recommended to link M&E more closely to the theory of change model and develop and
measure outcome and impact indicators to know if the activities are increasing resilience and can be
sustained in the long term by the CBOs and the villages. Also to systematically monitor and evaluate
in the M&E framework progress against the DAC evaluation criteria questions, with any enabling,
constraining and sustainability factors noted.
Include beneficiary feedback systems into the framework, to develop a MEAL framework
(Monitoring, Evaluation, Accountability and Learning).
It is also recommended to clarify the lists of beneficiaries in the beneficiary tables; for example, how
the impact on boys and girls is assessed and what is the age range used for boys and girls (eg. age 17
and under).
Continue M&E and follow up with vocational training participants that have already received training
to assess the impact of the training.
The household survey found that there are, on average, eight people per household. It is
recommended to adjust beneficiary numbers accordingly.
Monitoring and evaluation of the impact of the ‘Zai Holes’ initiatives (introduced on World
Environment Day in June 2016). The concept was that every household who plants a tree should dig
a hole and also plant vegetables and that whenever you water the vegetables the tree also benefits.
This was promoted this in all five villages and it was received well by the communities. In total, 450
Zai Holes were planted and the impact should be seen in the longer term.
56
Measuring impact:
To support assessment of the impact of the project on households, it is recommended to carry out
a project baseline, mid-term and end-line survey that includes a Household Economic Analysis
surveys and Comprehensive Food Security Assessmentsxx. For the sampling, if a lower confidence
level ad a wider margin of error is satisfactory, sample sizes (household included) and costs can be
kept down. It is key to ensure that the sampling of the households is random across the villages.
Due to the livelihoods approach of the project and the problem of malnutrition raised in some
communities during the dry seasons (that WRG was aware of, and responded to), it is also
recommended to conduct a Household Economic Assessment and an in-depth food
security/nutrition assessment in the initial months of any future project to address this potential
issue before it becomes a larger problem. This could be done early or one-third of the way through
the dry season to see the situation of the households and communities at this time. It may also be
appropriate to consider times when pastoralists are in the villages (eg, in April or May). This could
support the analysis of impact (in the point above), allowing any adjustments to be made to the
project before the next dry season and also helping to foresee likely food insecure periods.
The baseline household survey includes the question “does your household have enough assets that
could be sold off if drought happens?” (question 20). This showed that 15% of households have many
assets, 41% have a small amount of assets and 44% of no assets. The villages with the highest number
of households that said they have no assets was Galoole (69% of HH said no) and Raybadka (44% of
HH). Given that these are the agro-pastoralist villages, this indicates the other three villages are
considering their livestock to be the asset that they would sell if drought happens. It is recommended
to review this further in future work, for example, if households sell their livestock in the event of a
drought, what risk might this then pose to those households? What proportion of households have
the knowledge and training to carry out planned and strategic destocking? Or would households be
carrying out an emergency coping strategy? Do the households have other assets as well as
livestock? A Household Economic Survey (with a sufficient sample size and method) which includes
a Household Wealth Index at the beginning, mid-term and end of the project would also help to build
on this knowledge base.
Sustainability:
Establish ways for the communities to continue to fund the inputs they need for Earthworks,
sustainable homesteads, EWS, savings groups, business startups, farming activities and rangeland
management.
The focus on train-the-trainer has been well received and respected by the communities; future work
could continue to build on this in all aspects of the project – with additional focus on females and
young men and women (for agro-pastoralist and pastoralist activities and vocational skills training).
The importance of the role of groups such as FFS, Earthworks and SFL/SHG groups and their value
within the communities for leadership and knowledge sharing should continue to be highlighted to
these groups to encourage them further. Development of ‘youth champions’ on certain issues (such
as qat chewing) will also increase sustainability and impact.
The communities are quite reliant on the role of WRG within their communities to enable the
activities. Although the project has a strong emphasis on developing partnerships between the
communities and other stakeholders (eg MoRED) and strengthening the CBOs so they can take the
lead on activities, it is recommended to continue to build on the partnerships between the villages
and these other stakeholders that can offer support. As far as possible, develop regular and
systematic communication between such stakeholders and the communities, as well as
strengthening links to other pilot projects in Somaliland (eg. ministerial pilot projects and
57
programmes), the Savings for Life Group movement, EWS and other initiatives that have the
potential for collaboration across and between communities (eg. rangeland management).
Coordination:
Consider encouraging or establishing a rangeland management coordination forum to enable the
engagement of the different entities involved in supporting sustainable range management and the
development needs of the pastoralist communities in the area – and to coordinate with them and
share experiences and learningxxi.
Theory of Change:
Review and update the Theory of Change model with the additional information resulting from the
evaluation. For example, the inclusion of young people in decision making structures, vocational
training, as well as awareness raising on qat.
It is recommended to consider a youth focused component on young males and females. For
example, the development of ‘youth champions’ or ‘peer educators’ to facilitate:
o Knowledge transfer on issues such as chewing qat, decision making quality between males
and females and other life skills (team-work, leadership).
o Youth train-the-trainer in vocational skills.
Distribution of trees:
Continue with tree distributions to communities and households, which has been a successful
elemenr of the project. Tree-based systems have enormous potential to reduce vulnerability
and increase resilience of households in dryland areas – and are potentially more resilient to
drought than other types of foliage. Two strategies include:
Natural regeneration including; planting of local species in crop areas, pastures, follows or in
enclosures (to prevent overgrazing of seedlings). This can provide a tree-based system that can
provide multiple products and services.
Purposeful tree planting to provide specific products and services including; foliage for animal
feed, wood supply for building, sources of organic matter needed to improve the structure and
fertility of soil, fruits and vegetable food sourcesxxii. The MoERD stated during the evaluation
KII that it is not in the policy to plant trees for charcoal production.
58
References
i Cervigni R, Morrie M (editors) (2016) Confronting Drought in Africa’s Drylands: Opportunities for Enhancing Resilience. Africa Forum Development Series. Washington, DC: World Bank. Doi 10.1596/978-1-4648-0817-3. ii Food and Agricultural Organization of the United Nations (FAO). Drought: <http://www.fao.org/emergencies/emergency-types/drought/en/> consulted 12 October 2016. iii Ibid iv Al Jazeera News <http://www.aljazeera.com/indepth/features/2016/04/somali-camel-traders-pay-price-war-syria-160411092956190.html> consulted 1 October 2016. v Cervigni R, Morrie M (editors) (2016) Confronting Drought in Africa’s Drylands: Opportunities for Enhancing Resilience. Africa Forum Development Series. Washington, DC: World Bank. Doi 10.1596/978-1-4648-0817-3. vi The project compliments the strategy of the Somaliland National Development Plan (2012-2016) in the Rural Development section, p. 302 f. “Empowerment of the pastoral community to control resource utilization; Establishment of alternative economic activities in rural areas; Adoption of strategies for disaster preparedness and management; Introduction of microfinance schemes; Improving livestock production and sustainability of livestock economy; Improvement of rangelands; Improvement of pastoral community livelihoods; Regulation of the establishment of rural settlements and water points; Establishment of facilities to address livestock disease; Development of rural markets” vii Clayton S et al (2015) Psychological research and global climate change. Nat Clim Change 5:640–646. doi: 10.1038/nclimate2622 viii Ibid ix Ibid x Ibid xi FAO (2011) Drought Monitoring with the Combined Drought Index (CDI). SWALIM. <http://www.faoswalim.org/resources/site_files/Combined_drought_index.pdf> consulted 1 October 2016. xii Cervigni R, Morrie M (editors) (2016) Confronting Drought in Africa’s Drylands: Opportunities for Enhancing Resilience. Africa Forum Development Series. Washington, DC: World Bank. Doi 10.1596/978-1-4648-0817-3. xiii Mckemey K, McKemey J, Otsimi O (2016) Holistic Rangeland Management Consultancy Report for WRG. February. xiv Hussin K, Nelson J (1998) Sustainable Livelihoods and Diversification. IDS Working Paper 69. Institute of Development Studies and the Poverty Research Unit at the University of Sussex xv Berry S (1989) Property rights and rural resource management; the case of tree crops in West Africa. African Studies Centre, Boston University, USA. xvi International Institute for Sustainable Development (IISD) IISD Annual Report 1996. Winnipeg, MB. Canada: International Institute for Sustainable Development (IISD). 1996. xvii Cervigni R, Morrie M (editors) (2016) Confronting Drought in Africa’s Drylands: Opportunities for Enhancing Resilience. Africa Forum Development Series. Washington, DC: World Bank. Doi 10.1596/978-1-4648-0817-3. xviii USAID, World Bank, DFID, African Union, University of Nairobi, WFP (2012) Enhancing Resilience to Food Security Shocks in Africa. Discussion Paper November. <http://www.fsnnetwork.org/sites/default/files/discussion_paper_usaid_dfid_wb_nov._8_2012.pdf> consulted 10 October 2016. xix Ibid xx World Food Program methodology recommended. xxi Mckemey K, McKemey J, Otsimi O (2016) Holistic Rangeland Management Consultancy Report for WRG. February. xxii Cervigni R, Morrie M (editors) (2016) Confronting Drought in Africa’s Drylands: Opportunities for Enhancing Resilience. Africa Forum Development Series. Washington, DC: World Bank. Doi 10.1596/978-1-4648-0817-3.