final draft monitoring and evaluation framework and … · final draft monitoring and evaluation...
TRANSCRIPT
1
FINAL DRAFT
MONITORING AND EVALUATION FRAMEWORK AND PLAN
FOR THE SOMALI HEALTH SECTOR STRATEGIC PLANS
Developed by
Khadar Mahmoud Ahmed
MPH, M.E Expert in Health, Nutrition & Population Programs
With the support of
World Health Organization - Somalia Office
With the funding by
Somali Joint Health and Nutrition Program
DfID, AusAID, SIDA, USAID
2
TABLE OF CONTENTS
Table of Figures .............................................................................................................................................
Acronyms ......................................................................................................................................................
Operational Definitions ................................................................................................................................
Executive Summary .....................................................................................................................................
PART I. INTRODUCTION ................................................................................................................................
Context...........................................................................................................................................................
Overview of the health system of the country (NHP, HSSP, EPHS)........................................................
Challenges of the health systems monitoring and evaluation...............................................................
PART II. MONITORING AND EVALUATION FRAMEWORK
Introduction to the M.E framework...............................................................................................................
Goal, Objectives, Key Results and Principles of the HSSP M.E framework ...........................................
Goal ..................................................................................................................................................
Specific Objectives .........................................................................................................................
Key Results.. ......................................................................................................................................
Key Principles....................................................................................................................................
Core Indicators for HSSP M.E. Framework .................................................................................................
Program/Project Specific Indicators ....................................................................................................
Data Management .....................................................................................................................................
Sources of Data for Health Sector Monitoring..............................................................................
Data Collection Methods and Tools .............................................................................................
Data Analysis and Synthesis ...........................................................................................................
Data Accuracy and Reliability ......................................................................................................
Data Dissemination .........................................................................................................................
Data Communication ....................................................................................................................
Responsibilities for Data Management by Level .........................................................................
The HSSP Monitoring and Review Processes..............................................................................................
Performance Monitoring and Review at Central level...............................................................
Performance Monitoring and Review at Regional level............................................................
Performance Monitoring and Review at District level................................................................
Performance Monitoring and Review at Facility levels...............................................................
Joint Annual Review .......................................................................................................................
The National Health Accounts ......................................................................................................
Performance monitoring & review of implementing partners....................................................
Program/project specific reviews .................................................................................................
Routine Feedback to sub-national and key stakeholders .........................................................
Evaluation .....................................................................................................................................................
Program/Project Evaluation ..........................................................................................................
Mid Term Review .............................................................................................................................
HSSP End Term Evaluation ..............................................................................................................
3
Surveys ...........................................................................................................................................................
PART III. HSSP M.E PLAN................................................................................................................................
Introduction to the HSSP M.E plan.............................................................................................................
The results framework.....................................................................................................................................
The detailed implementation plan.............................................................................................................
The Key HSSP M.E plan Implementation Tasks and Assumptions...........................................................
HSSP M.E plan Performance Indicators ...................................................................................................
Roles and Responsibilities of key stakeholders............................................................................................
Monitoring the Implementation of the HSSP M.E plan ...........................................................................
Annexes .........................................................................................................................................................
HSSP indicators with targets ..........................................................................................................
Reporting timelines..........................................................................................................................
4
FORWORD
The Ministries of Health of the three zones launched the Health Sector Strategic Plans
2013 – 2016 which define the medium term health agenda and plans of the health
sector. The development of M.E framework and plan for HSSPs were developed after
the HSSPs have been in operation for a year. The process for the development of the
HSSPs took into consideration a wide range of policies, the new emerging diseases,
issues of regional and international health, etc.
The process of development of the HSSPs M.E framework and plan was highly
consultative, participatory and transparent. A consultant was recruited by WHO to
guide and support the development of the framework and the plan. Stakeholders from
national and local institutions, Development Partners (DPs), Civil Society, private sector
and academia were consulted during the development process.
The M.E framework and plan for HSSPs aims at establishing a system that is robust,
comprehensive, fully integrated, harmonized and well coordinated to guide monitoring
of the implementation of the HSSPs and evaluate its impact.
The M.E framework and plan is based on principles intended to institutionalize the use of
M.E as a tool for better public sector management, transparency and accountability,
so as to support the overall direction of the HSSPs and achievement of the results. It is
envisaged that this comprehensive M.E framework and plan to which all health partners
subscribe shall be the basis for improving the quality of routine information systems and
be used to institutionalize mechanisms and tools for measuring quality of both facility
and community based services.
The underpinning principles include simplicity, flexibility, progressiveness, harmonization,
alignment, and enhancing ownership. It describes the processes, methods and tools
that the sector will use to collect, compile, report and use data, and provide feed-
back. It translates these processes into operational activities, and assigns responsibilities
for implementation.
I wish to express my appreciation to the consultant Mr. Khadar Mahmoud Ahmed and
all of you who worked tirelessly to develop the M.E framework and plan for HSSPs on
behalf of the Somali people. I look forward to the acceleration of the implementation
of the M.E framework and plan for the HSSPs towards attainment of our national and
international health goals.
5
ACKNOWLEDGMENT
The Monitoring and Evaluation Framework and Plan, a companion document to the
Health sector Strategic Plans (HSSPs), is developed through a joint effort of the Ministries
of Health, international partners and other stakeholders.
We would like to express our deep appreciation to all the participants of the technical
consultative workshops and to the members of the zonal taskforces leading the field
consultations.
Special thanks to Mr. Khadar Mahmoud Ahmed, WHO consultant for his tangible
contributions, leadership, technical guidance and coordination for the entire piece of
work. We are so much grateful to his support and enthusiasm for the formulation of such
comprehensive M.E framework and plan within a limited period of time.
We would also like to express our sincere thanks to the donors, particularly DfID, SIDA,
AusAID and USAID for their financial support to the development of the M.E framework
and plan. We also highly appreciate the valuable technical input of WHO and other
stakeholders, who actively supported the development of this document.
6
Operational Definitions
Benchmarking: Benchmarking refers to comparisons between and within entities to assess performance.
There are different types of benchmarking which may vary according to level of comparison (national or
sub-national comparisons), level of assessment (individual service provider – facility – district – region –
national), measurement focus (process, outcomes, quality, performance), and use of data (public
reporting, accountability, internal reporting only, self-learning and improvement).
Civil Society Organization: any organization except the government and the UN system.
Data Management: comprises all processes related to data collection, analysis, synthesis and
dissemination.
Data Quality Assurance: The process of profiling data to discover inconsistencies, and other anomalies in
the data cleansing activities (e.g. removing outliers, missing data interpolation) to improve the data quality
Efficiency: This relates the level of attainment of goals to the inputs used to achieve them. Efficiency
measures the extent to which the resources used by the health system achieve the goal that people value.
Efficiency analyses will be part of the end term evaluation of the HSSP and health systems strengthening
projects.
Equity: The progress in terms of distribution of health system interventions will involve analyses of differences
within and between population groups, among regions, etc. using a series of stratifiers and summary
measures.
Evaluation: The rigorous, science-based collection of information about program activities, characteristics,
outcomes and impact that determines the merit or worth of a specific program or intervention.
Impact: Fundamental intended or unintended changes in the conditions of the target group, population,
system or organization.
Indicator:
Inputs: are resources that are put into a program in order to achieve the delivery of services;
Knowledge Management: Is a set of principles, tools and practices that enable people to create
knowledge, and to share, translate and apply what they know to create value and improve effectiveness.
Monitoring: The routine tracking and reporting of priority information about a program and its intended
outputs and outcomes.
Monitoring & Evaluation Plan: Is an integral part of the component of the national health strategy that
addresses all the monitoring and evaluation activities of the strategy.
Monitoring & Evaluation Framework: Refers to the performance based framework for monitoring and
evaluation of health systems strengthening.
Outcome: Actual or intended changes in use, satisfaction levels or behaviour that a planned intervention
seeks to support.
Outputs: are tangible products that are necessary to achieve the objectives;
Performance: The extent to which relevance, effectiveness, efficiency, economy, sustainability and impact
(expected and unexpected) are achieved by an initiative, program or policy.
Performance management: Reflects the extent to which the implementing institution has control, or
manageable interest, over a particular initiative, program or policy.
7
Performance measurement: The ongoing monitoring and evaluation of the results of an initiative, program
or policy, and in particular, progress towards pre-established goals.
Processes: are activities carried out for the achievement of one’s goals;
Progress towards HSSP goals: Refers the process of monitoring and reviews aiming to measure the extent to
which the objectives and goals of the HSSP (core indicators and their targets) have been attained. This will
be complemented by a stepwise analysis to assess which policies and programs were successful; from
inputs such as finances and policies to service access and quality, utilization, coverage of interventions,
and health outcomes, financial risk protection and responsiveness.
Qualitative assessment and analysis of contextual change: This takes into account non-health system
changes, such as socio-economic development that affect both implementation and the outcomes and
impact observed. Qualitative information on the leadership, policy environment and context is crucial to
understand how well and by whom government policies are translated into practice.
Review: Is an assessment of performance or progress of a policy, sector, institution, program or project,
periodically or on an ad hoc basis. Reviews tend to emphasize operational aspects, and are therefore
closely linked to the monitoring function.
Target:
8
EXECUTIVE SUMMMARY
The development of the Health Sector Strategic Plans (HSSPs) of the Somali health sector is an important step
in building the government’s capacity to improve access to health services for the Somali people. The HSSPs
sets realistic, measurable and understandable priorities appropriate to Somali context, rather than setting
unreachable global targets. They recognize the role of the private sector and the preference for the private
purchase of health services in Somali health sector. The plan provides a guide for external investments in the
health sector by traditional and non-traditional donors, the Somali diaspora, charities and NGOs.
The HSSPs M.E framework and plan has been developed to operationalise the strategic orientation
provided for comprehensive monitoring and evaluation in HSSP. M.E aims at informing policy makers about
progress towards achieving targets as set in the HSSP. In combination with other initiatives, the M.E
framework and plan will focus attention of stakeholders and direct efforts towards the ultimate goal of the
sector.
The HSSP M.E framework and plan has been developed in a participatory manner and shall guide all HSSPs
M.E activities. It specifies the type of monitoring, monitoring reports, timing of evaluations, roles and
responsibilities for the overall process and how they interact with the reporting each implementer is
required to perform (clear roles and responsibilities with respect to data gathering and reporting;). It also
focuses on the main M.E activities and aligns them to the existing national and international structures and
frameworks. It is intended to document what needs to be monitored, with whom, by whom, when, how,
and how the M.E data will be used. It also outlines how and when the different types of studies and
evaluations will be conducted by the sector.
In addition to the above considerations, the M.E framework and plan has been developed to address
some of the M.E challenges identified during HSSP. There have been no systems for performance
monitoring and evaluation in place during the development of the HSSP and there are enormous
challenges. Most of the challenges are due to lack of an M.E framework and plan for the HSSP. The
national M.E arrangements have been weak or nonexistent and comprised only a few semi-functioning
systems at program/project level. However, even those functioning systems, they are characterized by
fragmentation; duplication; weak co-ordination; lack of a clear results chain; poor definitions, tracking and
reporting of outcomes and results; use of different formats and approaches with no common guidelines
and standards; lack of national ownership; inadequate feedback and poor sharing of results across the
sector and other stakeholders.
The M.E framework and plan is based on principles intended to institutionalize the use of M.E as a tool for
better public sector management, transparency and accountability, so as to support the overall direction
of the HSSP and achievement of the results. The underpinning principles include simplicity, flexibility,
progressiveness, harmonization, alignment, and enhancing ownership. It also describes the processes,
methods and tools that the sector will use to collect, compile, report and use data, and provide feed-
back. It translates these processes into operational activities, and assigns responsibilities for implementation.
The goal of the HSSP M.E framework is to establish a system that is robust, comprehensive, fully integrated,
harmonized and well coordinated to guide monitoring of the implementation of the HSSP and evaluate
impact. The key objectives focus on establishing M.E system to enable the sector to track HSSPs
implementation and its impact; improve the completeness, accuracy and timely reporting of the HMIS at
all levels; strengthen early warning and surveillance systems; establishing research and survey plans;
developing vital registration system; enhancing governance and the institutional capacity; improving data
demand and information use as well as the use of information technology including GIS;
The key principles that guided the development of the national M.E Framework include: building strategic
partnership for M.E; mainstreaming the M.E system into the HSSPs which will be used to monitor the
indicators and strengthen health sector monitoring and evaluation; mobilizing and securing adequate
financial resources for strengthening the M.E system; setting standardized core set of indicators; simplifying
the data collection, analysis and the dissemination of information to the stakeholders; ensuring data quality
using protocols to verify the completeness and accuracy of the data collected; using data for decision
making; ensuring the timeliness and reliability of data and finally ensuring all programs and partners to be
transparent and accountable to the M.E system;
9
It is envisioned that the M.E framework and plan will result in timely reporting on progress of implementation
of the HSSPs; timely meeting of reporting obligations to Government, DPs and International Partners;
objective decision making for performance improvement; planning and resource allocation; better
accountability to government, DPs and citizens; appropriate policy dialogue with stakeholders; evidence-
based policy development and advocacy; as well as creating institutional memory on HSSPs
implementation
This comprehensive M.E framework and plan to which all health partners subscribe shall be the basis for
improving the quality of routine information systems and be used to institutionalize mechanisms and tools
for measuring quality of both facility and community based services. It should also strengthen dissemination
and use of information at both national and sub national levels.
In order to achieve the above a lot will have to be done to improve recording and reporting, and use of
data at all levels and all stakeholders, public, private and community to effectively monitor and later
evaluate the HSSPs implementation, including the M.E framework and plan itself.
Finally, there is need for sufficient funding and human resources with adequate technical capacity to
manage the various components of the M.E system.
10
PART I. INTRODUCTION OF THE HSSP M.E FRAMEWORK AND PLAN
1.1. THE CONTEXT
The civil war seriously affected the whole country politically, socially and economically. Health care
delivery suffered: hospitals, health posts, Maternal and Child Health (MCH) centres and other health
institutions were either seriously damaged or completely looted. Trained health personnel migrated
overseas in large numbers.
In 1991, the Republic of Somaliland was formed and now functions with a stable government and
considerable level of peace. In 1997, Puntland state of Somalia was formed by a grand elders and enjoys
peace and stability.
The country is divided into eighteen regions. The population is estimated around 10,000,000, with an annual
population growth rate of 3.14%. Life expectancy at birth is 49-60 years. The population consists of nomads
(55%) and urban and rural dwellers (45%), Population density is estimated at 28 persons per sq km.
With constant drought and famine affecting humans and livestock, poverty and unemployment are
widespread. Over 60% of the population is considered to live below the poverty line. The backbone and
the source of wealth of Somali economy is livestock. About 65% of the population depends either directly
or indirectly on livestock and livestock products for their livelihood. Crop husbandry provides subsistence for
about 20% of the country’s population. Foreign aid and remittances from Somali diaspora also play a major
role in the economy of the country. (additional information is necessary particularly for South Central Zone)
1.2. OVERVIEW OF THE HEALTH SYSTEM OF THE COUNTRY
1.2.1 Background
The development of the Health Sector Strategic Plans (HSSPs) of the Somali health sector was an important
step in building the government’s capacity to improve access to health services for the Somali people. The
HSSPs sets realistic, measurable and understandable priorities appropriate to Somali context, rather than
setting unreachable global targets. They recognize the role of the private sector and the preference for the
private purchase of health services in Somali health sector. The plan provides a guide for external
investments in the health sector by traditional and non-traditional donors, the Somali diaspora, charities and
NGOs.
The HSSPs provide a starting point for the government to develop annual work plans and budgets that will
detail the specific activities and funds that are needed to accomplish the strategies and objectives of the
plan. These plans provide statement to funding and implementation partners of government priorities for
investment so that their support can be better harmonized, effective and efficient.
The emphasis of the HSSP is the rolling out of the EPHS which was designed in 2009 and widely endorsed. The
package includes six core programs and four supplementary programs. It is implemented in four levels of
service provision. The MOHs will implement this strategy in phases; the initial pilot program is currently
underway in Sahil, Karkaar and Gedo regions with the support of the DfID through the NGO consortium.
Following a review late in 2013, the EPHS will be rolled out in nine more regions across the country with the
support of the Joint Health and Nutrition Program (JHNP).
In this regard, there is an imperative need of improving the capacity of the M.E function of the government
to monitor and evaluate the implementation of the HSSP and the rollout of the EPHS across the country.
1.2.2 Health Status
According to the preliminary results of the 2011 UNICEF Multi-Indicator Cluster Survey, or MICS, under-5
mortality is 90 per 1,000 live births. 42 per 1,000 newborn infants die within the first month of life. Maternal
mortality is also among the highest in the world, estimated at between 1,044 and 1,400 per 100,000 live births.
These figures mask considerable inter-regional and rural-urban variations.
11
1.2.3 Challenges of the health systems monitoring and evaluation
There have been no proper systems for performance monitoring and evaluation in place during the
development of the HSSP and there are enormous challenges. Most of the challenges are due to lack of
an M.E framework and plan for the HSSP. The national M.E arrangements are weak or nonexistent and
comprised only a few semi-functioning systems at program/project level. However, even those functioning
systems, they are characterized by fragmentation; duplication; weak co-ordination; lack of a clear results
chain; poor definitions, tracking and reporting of outcomes and results; use of different formats and
approaches with no common guidelines and standards; lack of national ownership; inadequate feedback
and poor sharing of results across the sector and other stakeholders.
The HMIS was also noted during the HSSPs to have various weaknesses. The low and declining trends for
timelines of monthly reporting by regions was worrying. There are mismatches of data between the key
points of data collection through the various management levels and programs. Insufficient funds featured
as the major constraint to implementation. The markedly no government budget funds for HMIS lead to
over reliance on donor project resources often associated with piece-meal initiatives. Human resource for
implementation remains inadequate at all levels of the structure. At national and regional levels, the HMIS
units are functioning in Somaliland and Puntland. At district level, staff to handle data remain non-existent.
The continuous inadequacy (numbers) of HMIS tools has also greatly affected expectations of having HMIS
as a major source of information for monitoring the sector plans. The HMIS remains manual, which affects
quality, timeliness and completeness of reports. The low level of prioritization of the HMIS at all levels, and
the inadequate utilization of data is cause for concern.
In addition to the above challenges, there are weak linkages between the various data producers leading
to inadequate sharing of information. Analysis, synthesis, effective dissemination and use of information to
guide policy dialogue and implementation of health programs remain a challenge. Following a
comprehensive analysis of the HSSPs, a number of recommendations were made for improving data
management at all levels:
Improve the level of prioritization of information management in the sector. Appropriate and strategic
advocacy should be carried out for various aspects of sector managers and decision-makers;
Particular efforts to be made for appropriate funding (level, mechanisms) for information
management;
The human resources issues should be addressed at all levels. There should be regular training and
updating of skills for health workers on data management;
Availability of HMIS tools must be improved including establishment of software packages and
provision of IT equipments at all levels (national, regional, district and facility);
Efforts must be made to establish mechanisms of data sharing by all producers;
Use of data should be enhanced through provision of timely analysis and effective dissemination;
There is urgent need to improve the timeliness, completeness and quality of facility generated data
with the help of information technology and supported by an up-to-date national health facility
database that covers all public and private health facilities with data on infrastructure, equipment
and commodities, service delivery, and health workforce integrated;
12
II. THE HSSP MONITORING & EVALUATION FRAMEWORK
2.1 OVERVIEW OF THE M.E FRAMEWORK
The HSSP M.E framework elaborates a monitoring framework for the sector which includes a range of
indicators at various levels, sources of information, regularity of various reports, and monitoring structures.
The M.E framework for tracking progress is informed by the need to comprehensively monitor and review
sector progress. The framework for the analysis is based on the common steps of the M.E logical framework,
which shows the way in which inputs may lead to desirable health impact.
Figure 1: Common steps of the M.E. logical framework
Input Output Outcome Impact
Monitoring
(Daily, Monthly, Quarterly)
Review
Quarterly, Annually)
Evaluation
(Mid-term, Final)
The M.E. framework is an adaptation of the M.E framework for health systems strengthening (HSS) which
was developed by Global Partners and countries. The framework builds upon principles derived from the
Paris Declaration on aid harmonization and effectiveness and the International Health Partnership (IHP+). It
is intended to ensure that all indicator areas - from inputs to impact - are considered in the analysis, and
pathways of influence are clarified
Figure 2: Monitoring and Evaluation of the Health Systems Strengthening
Table 1: Main features of Monitoring, Reviews and Evaluations
Description Monitoring
(Daily, monthly, quarterly)
Review
(Quarterly, Annual)
Evaluation
(mid-term, final)
Objective To track changes from
baseline conditions to
desired output
To track and validate
mainly outputs and
outcome to some
extent
To validate what results were
achieved and why they were
not achieved
Focus Focuses on the inputs and
outputs of annual plans
Focuses on the annual
plan targets mainly on
output and outcome
Compares planned with
intended outcome
achievement. Responds
questions on relevance,
effectiveness, sustainability
and change
Methodology Tracks and assesses
performance (progress
towards outcomes)
through analysis and
comparison of indicators
over time
Evaluates annual
performance by
comparing indicators
before and after. Relies
on monitoring data from
routine HMIS
Evaluates achievement of
outcome by comparing
indicators before and after
HSSP. Relies on monitoring
data on information from
external sources
Information
sources
HMIS, Supervision report,
Activity reports
HMIS (monitoring report)
Annual facility surveys,
JAR,
Surveys (harmonized to meet
the HSSP M.E framework and
plan, research reports, JAR
reports, observations
Conduct Continuous and Annual by key partners 5 yearly, external evaluators
13
systematic by directors,
program managers and
partners,
with or without help of
external facilitators
and partners
Use Alerts managers to
problems in performance
and provides options for
corrective actions
Provides input to the
planning of the next
annual work plan
Provides policy makers and
managers with strategic and
policy options
Main users Service providers,
Program managers
Program managers,
development partners
Policy and strategic planners,
development partners,
Table 2: Frequency and participation in M.E activities
HSSPME Series Level Responsibility Facilitation Timing / Deadline
Monthly Facility HFMT Health Facility
In-charge
5th of the following
month
Monthly Health Facilities DHMT/RHMT DHO/RHO 10th of the following
month
Quarterly Programs/Divisions Program/Division
Managers
M.E & Research
Office
1st week of the
months Apr, July,
Oct, Jan
Annual review Facility Facility In-charge DHMTs/RHMTs July each year
Annual review District/Region DHMT/RHMT M.E & Research
Office August each year
Annual review National Dept of Planning M.E & Research
Office
November each
year
Mid-term
review National Dept of Planning
M.E & Research
Office Jan-March 2015
Evaluation National Dept of Planning M.E & Research
Office Oct-Dec 2016
2.3 GOAL, OBJECTIVES, KEY RESULTS AND GUIDING PRINCIPLES OF THE M.E FRAMEWORK
2.3.1 Goal
The goal of the HSSP M.E framework is to establish a system that is robust, comprehensive, fully integrated,
harmonized and well coordinated to guide monitoring of the implementation of the HSSP and evaluate
impact.
2.3.2 Objectives: The specific objectives of the HSSP M.E framework are:
1. Establish an M.E system to track HSSP implementation and impact by 2016,
2. Timely, complete and accurate HMIS at all levels by 2016.
3. Effective early warning and surveillance system at all levels by 2016.
4. Establish a “survey and research agenda” by 2016.
5. Establish vital registration (birth and death) system in the country by 2016.
6. Enhance governance, institutional capacity, partnership and coordination by 2016
7. Improve data demand, dissemination, communication and use at all levels by 2016
8. Objective 8: Use geographic information system for the health sector planning,
14
2.3.3 Key Results:
The M.E framework should result in:
a) Timely reporting on progress of implementation of the HSSPs;
b) Timely meeting of reporting obligations to Government, DPs and International Partners;
c) Objective decision making for performance improvement; planning and resource allocation;
d) Accountability to government, DPs and citizens;
e) Policy dialogue with stakeholders;
f) Evidence-based policy development and advocacy;
g) Institutional memory on HSSPs implementation;
2.3.4 Guiding Principles
The following principles guided the development of the national M.E Framework:
I. Building strategic partnership for M.E: The HSSPs builds on partnership among various stakeholders.
This principle also applies to the development of the national M.E system,
II. Mainstreaming the M.E system will be incorporated into the HSSPs and will be used to monitor the
indicators and strengthen health sector monitoring and evaluation,
III. Enough financial resources will be mobilized and allocated for the strengthening of the M.E system,
IV. Standardized core set of indicators. The national M.E system will have a core set of national
indicators,
V. Simplicity: Data collection, analysis and the dissemination for information to the stakeholders will be
simplified and made user friendly,
VI. Data Quality Assessment (DQA): The Ministries will put in place DQA protocols to verify the
completeness and accuracy of the data collected. These will ensure both internal self-assessment
and external verification of data/information,
VII. Data collected at the sub-national or national levels will be used for decision-making,
VIII. Timeliness and Reliability of Data: data collected, disseminated and used through a good M.E
system will be timely and reliable. All programs and partners will be required to be transparent and
accountable to the M.E system they have and the data they collect.
Fig 3: Diagram of Health Information System
2.4 INDICATORS
2.4.1 Core Indicators for HSSP M.E. Framework
HSSP has defined core indicators of about 30 over the four year period informing the progress in critical
elements of the health sector strategic plans. The table below provides more details of the core indicators
including the units for the measure, the baseline value, the target. Most of the indicators are measured in
numbers, percents or rates for a specified time period.
15
Table 3: Core Indicators with Annual Milestones and Targets
S.N INDICATOR BASELINE (2012) Milestones Target
2016
Data Source
Source 1 Source 2 2013 2014 2015
IMPACT INDICATORS
1 Neonatal mortality rate 45 35 MICS
2 Infant mortality rate 108
(UN IAE)
73 (MICS,
11)
75 MICS
3 Under-five mortality rate 180
(UNIAE)
90 (MICS,
11)
110 MICS
4 Maternal mortality ratio 1000
(UNIAE)
700 MMR Study
5 Proportion of underweight in under five year
children
MICS, Nutrition
Survey
6 Total fertility rate 6.3
(MICS, 11)
5.7 MICS
HEALTH FINANCING INDICATORS
1 % increase of government allocation to
health;
2 6 NHA
2 Number of annual audited reports published 0 0 1 2 3 4 NHA
3 Per capita out of pocket expenditure on
health;
80 (WHO
OBS, 2011)
NHA
HUMAN RESOURCE FOR HEALTH INDICATORS
4 Health professionals (doctor, nurse, midwife)
to population ratio per 1000 population
HRIS
5 Percentage of health professionals
registered by category
0 0 25 50 75 100 NHPC
6 Proportion of health workers and managers
with signed Performance Agreements
0 0 10 25 40 60 HRIS
HEALTH SERVICE ACCESS AND COVERAGE INDICATORS
7 Contraceptive prevalence rate (Percentage
of women 15-49 using modern
contraceptive methods)
2.6
(MICS 2011)
9 MICS
8 Proportion of people who are on ARVs; HMIS
9 Proportion of health facilities with PMTCT
services;
AHFS/HMIS
10 TB case detection rate 41
(Gl TB
Report)
70
11 Treatment success rate 86
(Gl TB
Report)
96 HMIS
12 DPT 3 coverage rate for 1 yr 7.2
(MICS 2011)
50 MICS
13 Percentage of deliveries conducted at
health facilities
32.7
(MICS 2011)
50 MICS
14 Skilled birth attendant 38.4
(MICS 2011)
57 MICS
16
15 Percentage of health facilities providing full 6
core programs of the EPHS (EPHS target
regions)
0 0 25 40 50 70 AHFS
16 The % of health facilities with Basic and those
with Comprehensive emergency obstetric
care;
70 AHFS
Super. repots
17 % of RH and hospitals readily available with
Blood Transfusion Servicess;
100 AHFS
Super. repots
18 Proportion of under-fives with fever who
receive malaria treatment within 24 hours;
MICS, MIS
19 Proportion of mothers of newborns 1-2 weeks
practicing clean cord and skin care,
keeping babies warm, exclusively breast
feeding and recognizing danger signs;
MICS, KABP
ESSENTIAL MEDICINE AND MEDICAL TECHNOLOGY
20 Percentage of health facilities reporting no
stock outs of essential drugs (six tracer
medicine)
80 AHFS
21 Percentage of health facility rehabilitated 60
22 % of referral health centers and hospitals
with emergency transport system (one
functional ambulance)
90 AHFS
Super. repots
23 % of health facility with WASH available for
the providers/clients/patients
80 AHFS
Super. repots
GOVERNANCE
24 Number of regulatory bodies functioning 1 4
25 Number of regions with functioning regional
health management systems
80
26 Number of senior and midlevel managers
trained in leadership and management;
HEALTH INFORMATION
27 HMIS reporting rate 95 HMIS
28 % of IDSR weekly reports submitted and
disseminated on time;
100 IDSR reports
29 The proportion of planned periodic reviews
that are carried out;
30 Proportion of planned surveys and research
studies carried out;
17
2.5 DATA MANAGEMENT
2.5.1 Brief Introduction
This section on data management concentrates on the sources of data; data collection methods and
tools; responsibility for data collection and processing; data analysis and synthesis; data quality assessment;
data dissemination; communication and use as well as responsibilities for data management at all levels.
2.5.2 Sources of Data for Health Sector Monitoring
The data needs of the HSSP are based on agreed performance indicators (core and program specific) to
facilitate monitoring, evaluation, reporting and decision-making. The main data sources include:
2.5.2.1 Service delivery data will be collected by all public and private health service delivery facilities,
outreach teams, mobile clinics and community. In addition different programs and projects managed at
the various levels of the Ministry shall provide reports to the HMIS on program-specific activities. Health
projects managed by implementing partners (NGOs and CSOs) at regional/district or community level shall
provide reports through the district/regional health management information system. This data will be
collected routinely using established data collection methods and tools and aggregated at health facility,
district, regional, program and national level.
2.5.2.2 Administrative data sources will provide information on health inventories, supervision, management
meetings, logistics management, human resource, financial resource flows and expenditures at national
and sub-national levels.
2.5.2.3 Population based health surveys mainly carried out by the MOHs and other institutions that generate
data relative to populations (population studies) as a whole. Research Institutions and academia that carry
out health systems research, clinical trials and longitudinal community studies will also provide data for
interpretation and possible use by the sector.
2.5.2.4 Civil registration and vital statistics system is essential for providing quality data on births, death and
causes of death. Efforts will be made to establish and link this system to the Health Management
Information System. Currently this system is not functioning.
2.5.2.5 Population and Household Census is carried out every ten years; however, the last census was
carried out in 1986 in Somalia. Census is the primary source of data on size of the population, its geographic
distribution, and provides the social, demographic and economic characteristics. It also produces annual
projections at national and sub-national level.
2.5.5 Data Collection Methods and Tools
The methods of data collection will be a combination of quantitative and qualitative methods. As far as
possible, standardized data collection tools and techniques will be used. Most data in respect of indicators
will be collected monthly, quarterly and annually, whereas any survey-based indicators will be collected at
baseline, mid-term where possible and in the last year of HSSPs implementation.
The specific tools and techniques will among others, include; the HMIS; Human Resource Information
System (HRIS); Logistics Management Information System (LMIS), National Health Accounts (NHA). Specific
questionnaires will be designed for surveys (baseline, mid and end term). Standardized checklist will be
used to collect data during ongoing monitoring field visits. Formats shall be applied for case studies,
stakeholder meetings, performance review forums and management meetings. Geographical Information
System (GIS) shall be used to enhance documentation and accountability where applicable. The main
tools and techniques for collection of HSSPs M.E information are explained below:
2.5.5.1 Health Management Information System: data collected during health service delivery is critical for
tracking performance and trend analysis. It cannot be substituted by any other form of data. It will
therefore form an important source of data for measuring progress of the HSSPs implementation. The HMIS
has the following categories of information: data on individual clients, information on curative services,
information on preventive services, resource management e.g. inventories (staff list, health facility,
equipment), logistics and commodities, finance/user fees.
18
2.5.5.2 Integrated Disease Surveillance and Response (IDSR): This system is part of Health Management
Information System (HMIS). It is a systematic data management system that deals with epidemic-prone
diseases, diseases targeted for eradication/ elimination and selected diseases of public health importance.
This system is just scaled up across the country with the support of WHO using IDSR strategy. It relies mainly
on weekly and immediate reporting for early detection of respective diseases.
2.5.5.3 Human Resource Information System: Is a system for collecting, processing, managing and
disseminating data and information on human resource for health (HRH). HRIS in Somaliland is implemented
with the support of Tropical Health and Education Trust (THET). The HRIS implemented in Somaliland is a
valuable source of information about health workers deployed throughout Somaliland. The system captures
health personnel's data by organization unit, cadre, etc., and generate various general and aggregated
reports in different formats. It enables the MOH and other institutions to quickly and easily obtain up-to-date
information specific to the current health workforce. The HRIS is the main source of data on staffing levels in
public facilities. Data and reports from these systems can be very useful for health planning at different
levels. The system requires further development in Somaliland and a rollout plan to Puntland and
South/Central zones.
2.5.5.4 Supply Chain Management System (SCMS): A SCMS will be established to strengthen the information
systems for medicines and health supplies. This system should allow facilities to conduct proper ordering as
well as the MOHs and agencies should find it easy to disseminate information about ordering, prices and
available quantities.
2.5.5.5 The National Health Accounts: The National Health Accounts will be established and institutionalized
as a process of generating routine and standardized health expenditure data to inform HSSPs
implementation and policy decisions. Resources flowing in the health system from all sources managed by
all agencies and used to provide services whose primary intention is health will be tracked. Among the
objectives to be achieved is, benchmarking performance against established targets; allocating scarce
resources according to needs; improving accountability and efficiency; planning for future and raising
additional funds (based on gaps and needs) and; ensuring sustainability. A comprehensive NHA with sub
accounts (TB, Malaria, HIV, RH, CH, etc), as deemed necessary by stakeholders will be produced every two
years. All actors in the sector (service delivery levels, central level institutions, regions, private providers,
CSOs and Development partners) will be obliged to compile, analyze, utilize health expenditure data and
report to appropriate levels. However, it is envisioned that the development of a comprehensive NHA will
take a substantial time; therefore, a phasing plan must be rationally implemented with initial focus in
Somaliland and gradual expansion to other zones
2.5.5.6 Field visits checklists will be used from time to time to obtain information that may be required to
improve performance or even for obtaining insights for example the Pre-Joint Annual Review Visit,
Quarterly Joint Field Monitoring Visits and more in-depth investigations. Current supervision tools and visits
will be harmonized for all programs with view to improve efficiency, reduce redundancy and ensure
synergy of all field monitoring activities at all levels.
2.5.5.7 Geographical Information System: The GIS system will be established with advancement of
technology. GIS enabled photographic and video recordings and will be used to track changes of
implementation of particular programs of the HSSPs by geographical location. GIS provides a means of
analyzing coverage of general or specific health services in relation to need (e.g. disease prevalence
rates) and how these services are related to communities (e.g. income level), one another and the larger
health infrastructure. M.E data on key health targets at different levels of health service delivery (e.g.
district, region, national) will be used to generate maps and other graphics (like bar and line graphs) that
show which areas are meeting the targets or are lagging behind. Overlays of different indicators and
further spatial analysis can also identify the hotspots which would be the basis for prioritization of resource
allocation. Maps showing health indicators at district, region and even health facility level can help health
planners and managers to identify disadvantaged areas; examine equity issues; and improve decision
making on where to invest.
2.5.5.8 The Vital Registration (VR) system: Vital registration system will be established to enable the
recording of births and deaths in health facilities and in the communities. Currently, the system is not in
place across the country; however, plans are underway to pilot it in Somaliland.
19
2.5.5.9 The population-based surveys: Population-based surveys will be carried out at community level, to
obtain information from households. Sources of data will be determined using random-selection methods
and the data collected is used to represent regions or the country. Population-based surveys will be
conducted periodically.
2.5.5.10 Demographic Surveillance Sites (DSS): A DSS needs to be set up to monitor and reduce incidence
of communicable diseases. Such surveillance sites would provide a good opportunity in the country to
monitor trend of morbidity and mortality related to communicable diseases,
2.5.5.11: Behavioural Surveillance:
2.5.6 Data Analysis and Synthesis
Data analysis and synthesis will be done at various levels of HSSPs M.E (national, region, district to health
facility) to enhance evidence based decision making. The results obtained will be summarized into a
consistent assessment of the health situation and trends, using core indicators and targets to assess
progress and performance. The focus of analysis will be on comparing planned results with actual ones,
understand the reasons for divergences and compare the performance at different levels (Quarterly and
annual progress reports, mid and end term evaluations, thematic studies and surveys). In addition, health
systems research as well as qualitative data gathered through systematic processes of analyzing health
systems characteristics and changes will be carried out.
2.5.7 Data Accuracy and Reliability
All reports submitted to the HMIS and M.E. & Research Units will be reviewed for accuracy and clarification
sought where necessary. Even where there is no need for clarification acknowledgement of receipt of
reports will be provided before the due date for the subsequent report. Data quality assurance processes
will include periodic Data Quality Audits (DQA) of recorded data by supervisors; regular training of staff,
and provision of routine feedback to staff at all levels on completeness, reliability and validity of data; and
data quality assessment and adjustment which will be carried out periodically.
The objective of data validation is to ensure that the data used by the health sector to make decisions is
sound and accurate. Specific efforts will be made to undertake data validity including: application of the
computed validation/data accuracy index into regional and national annual reports; specific support for
outliers; routine (quarterly) data checks on a sample of regions or districts. Regular data quality assurance
for facility based data including regular review and verification for accuracy and completeness will be
carried out monthly by the health facility in-charges at all levels. All periodic reports should be checked
and endorsed before submission to the relevant stakeholders. DQA will be carried out at points of data
collection, collation and analysis by the technical staff of the HMIS for districts or regions.
Standardized DQA tools will be developed for application at all levels. DQA for sector evaluation studies
shall be carried out using agreed formats by the M.E and Research unit which is the coordinating entity for
the sector evaluation studies as well as with responsibility of data validation for health systems researches
carried out in the respective institutions.
2.5.9 Data Dissemination
Data need to be translated into information that is relevant for decision-making. Data will be packaged
and disseminated in formats that are determined by management at the various levels. Service delivery
data shall be packaged and displayed at the various health facilities using the HMIS formats already
provided. The timing of information dissemination should fit in the planning cycles and needs of the users.
2.5.9.1 Monthly, Quarterly and Annual Health Statistical Reports: These reports will be compiled from the
periodic statistical reports submitted through the Health Management Information System (HMIS). The
quarterly and annual health statistical reports provide ample attention to data quality issues, including
timeliness, completeness and accuracy of reporting, as well as adjustments and their rationale. The HMIS
officers will be responsible for compiling and disseminating these reports. Detailed data should also be
available on the MOH websites.
2.5.9.2 Quarterly Performance Review Reports
20
Quarterly sector performance review reports will be presented by the various sector technical working
groups during the sector quarterly review meetings. Quarterly regional and central performance reports will
also be disseminated and discussed at this forum. At regional level, quarterly performance reports will be
presented and discussed at the quarterly review meetings attended by the key implementers.
2.5.9.3 Annual Health Sector Performance Report (AHSPR) The AHSPR is useful in highlighting areas of
progress and challenges in the health sector. The reports will assess progress on the annual work plans and
an overall assessment of sector performance against the targets set in the HSSPs. It will also review progress
against the sector priorities set during the preceding Joint Annual Mission with stakeholders. The different
levels of health care delivery are expected to compile their reports according to the HSSPs reporting
timeline, and use them for performance review. The annual regional performance reports are then
forwarded to the national level for compilation of the AHSPR.
The AHSPR brings together all data from different sources, including the facility reporting system, household
surveys, administrative data (minutes, supervision reports, financial reports, SCM reports, HRIS reports, etc)
and research studies, to answer the key questions on progress and performance using the HSSP core
indicators and health goals. The AHSPR will present a detailed account of annual performance against the
core and programmatic indicators of the sector strategic plan, comparing current results with results of
previous years, and formulate challenges and recommendations by sector and program. The AHSPR will
provide the background and in-depth information for annual reviews and disease specific reports. It will be
presented by MOH to health stakeholders and discussed at the JAR.
The format of the AHSPR shall be developed in order to present the sector performance issues in a format
that can easily facilitate the knowledge management process. The compilation of the AHSPR shall be
coordinated by the Director of Planning in the MoH. The budget of collating annual sector performance
data and report writing will be provided for under the monitoring and evaluation budget. Annual regional
performance reports shall be presented and discussed at the relevant annual stakeholders‟ forum.
2.5.9.4 Dissemination of Survey Findings Feedback on survey findings will be in the form of workshops and
dissemination of reports which will be circulated to relevant stakeholders in hard copy and soft copies as
well as through the MOH websites.
2.5.10 Data Communication
Data communication shall follow the existing MoHs coordination structures. In order to ensure routine
feedback on performance to sub-national and service providers. The MoHs will use various communication
channels in order to ensure public access to data and reports. Quantitative and qualitative data will be
made publicly accessible through the MoH database under the Department of Planning. Email accounts
will be created for all regions and hospitals and should be used for communication with the Departments
as well as for Continuous Professional Development (CPD) activities. The public will also be able to access
health information on the MoH websites. In addition to the Information and Communication Technology
(ICT) facilities at the MoHs and institutions, the M.E unit shall collaborate with the Health Education and
Promotion Unit in the MoHs to translate data and information according to the target audience and utilize
various communication channels e.g. radio, T.V, video conferencing, tele-conferencing, newsletters,
booklets, etc.
2.5.3 Use of IT technology in Data Management
Electronic Disease Surveillance System: To minimize morbidity and mortality due to communicable diseases
by detecting epidemics at their earliest possible stages, it is necessary to introduce the national weekly
reporting based surveillance system called Disease Early Warning System (DEWS). The DEWS is effective in its
core functions of alert detection and early outbreak containment. Its approaches for outbreak detection
include immediate alert reporting, investigation, timely response and weekly data collection. It is an
initiative that promotes the use of data and information system standards to advance the development of
efficient, integrated, and interoperable surveillance systems at all levels. A primary goal of eDEWS is to
capture data using mobile software and secure automatic electronic transmission and analysis of data,
alert generation and dissemination of information to main stakeholders. It will also facilitate identification of
national public health threats more promptly, more timely and accurate disease reporting and may
provide platform to facilitate integration of disparate reporting system. It will be a secure online framework
21
that will allow healthcare professionals and government agencies to communicate about disease patterns
and coordinate national response to outbreaks.
The idea of eDEWS will help in streamlining weekly data collection and better integration with health
management information system and parallel surveillance programs such as routine, polio and EPI
surveillance. The data transmission will be simplified by GPRS/SMS reporting where possible; importantly,
data will be managed more effectively so as to maximize their usefulness, through a new software
application. Alert detection will be enhanced further by including automatic alert detection module in the
software that will generate alerts based on thresholds and will send messages to DEWS response team for
immediate response within 24 hours of alert notification.
The development of a data management system for electronic DEWS will consist of three phases of
application development. Phase I will involve developing a system to conduct weekly disease surveillance
using mobile phones in pilot regions. Phase II will involve providing additional system enhancements that will
be weekly based and will be implemented throughout country. Phase III system enhancement will cover
more advanced features, such as sophisticated data analysis, graphing and mapping, and integration of
weekly DEWS data with the monthly HMIS data.
2.5.4 Responsibilities for Data Management by Level
Brief Introduction:
Overall, the sources of M.E information will be guided by different information needs, particularly the
government, development partners, private sector and the community. The MoH will house the central
database for reporting on progress of the HSSP. The MoH M.E section will serve as a repository for all service
delivery data and information at national level. This implies that all health service delivery data and
information should be routed through the MoH for validation, analysis & synthesis, and dissemination.
2.5.4.1 Administrative Data Management
At National Level: The M.E. Office is responsible for:
Ensuring compilation and processing of administrative data into departmental/Institutional
records(minutes, inventory) and reports (supervision, activity);
Analyzing the quality of all reports received and ensuring follow-up in case of incompleteness,
problems with validity, as well as delays;
Compiling all reports from the program M.E. Officers into a single departmental/institutional report;
Preparing an analysis of the data for discussion during the
departmental/institutional/program/project implementation review meetings and sector
performance review meetings for decision-making;
Forwarding the departmental/institutional/program/project reports to the top management and to
the development partners and stakeholders;
Providing quarterly feed-back to the departments/institutions/programs/projects;
Disseminating quarterly and annual reports to the top management and to the development
partners and stakeholders,
At Regional Level: The RHO is responsible for:
Ensuring compilation and processing of administrative data (minutes, inventory, supervision and
other activity reports);
Analyzing the quality of all reports received and ensuring follow-up in case of incompleteness,
problems with validity, as well as delays;
Compiling all reports from the Regional Health Management Team (RHMT) members into a single
Regional Health Office report;
Preparing an analysis of the data for discussion during the RHMT meetings and regional forum for
decision-making;
Forwarding the Regional Health Office administrative report to the Department of Planning
(National M.E & Research Office);
Providing quarterly feed-back to the RHO/DHO and health facilities management teams;
Disseminating quarterly and annual administrative reports to RHMT, Regional Health Board (RHB)
and Regional Forum;
22
At Health Facility Level: The Facility In-Charge is responsible for:
Ensuring compilation and processing of administrative data (minutes, inventory, supervision and other
activity reports);
Analyzing the quality of all reports received and ensuring follow-up in case of incompleteness,
problems with validity, as well as delays;
Compiling all reports from the Technical Officers (Section heads) into a single health facility report;
Preparing an analysis of the data for discussion during the health facility staff meetings and CHC
meetings for decision-making;
Forwarding the health facility administrative report to the DHO or RHO where there is no district health
management system;
Providing quarterly feed-back to the health providers;
Disseminating quarterly and annual administrative reports to the DHO and health facility forum;
2.5.4.2 Service Delivery Data Management
At National Level: The HMIS Office is responsible for:
Receiving all regional data (including those from the national referral hospitals);
Ensuring entry of all regional data (including those from the national referral hospitals) into the
National Health Management Information System (NHMIS) software package;
Analyzing the quality of all reports received and ensuring follow-up in case of incompleteness,
problems with validity, as well as delays;
Preparing an analysis of the data for discussion during the Technical Working Group meetings and
sector performance review meetings for decision-making;
Providing quarterly feed-back to the RHOs/ NRHs;
Disseminating weekly IDSR reports to all stakeholders and community;
Disseminating quarterly and annual national performance reports at sector review meetings;
At regional level: The regional HMIS officer is responsible:
Receiving all districts and all health facilities data where there is no district health system (including
those from the general and referral hospitals);
Entering all districts and heath unit data (including those from the general and referral hospitals) into
the Regional Health Management Information System (RHIS) software package;
Analyzing the quality of all HMIS reports received and ensuring follow-up in case of incompleteness,
problems with validity, as well as delays;
Compiling all reports from the districts and health units into a single regional report using the RHMIS
software;
Preparing an analysis of the data for discussion by the RHMT for decision-making and participating in
the RHMT discussion;
Forwarding the RHMIS report electronically to the National HMIS;
Providing quarterly feed-back on data management to the health units;
Disseminating quarterly and annual regional performance report to Regional stakeholders' forum.
At District Level: The District Health Management Information Office is responsible for:
Receiving all health unit data (including those from the private providers) in the District;
Entering all health unit data (including those from the private providers) into the DHMIS software
package;
Analyzing the quality of all HMIS reports received and ensuring follow-up in case of incompleteness,
problems with validity, as well as delays;
Preparing an analysis of the data for discussion by the District Health Management Team (DHMT) for
decision-making and participating in the discussion;
Forwarding the district report to the Regional Health Office by the 7th day of the following month;
Providing quarterly feed-back on data management to the health units;
Disseminating quarterly and annual district performance report to district stakeholders' forum;
At health facility level: the heath information assistant or medical records officer or where there is no such
dedicated person, the health facility in-charge is responsible for:
Regularly compiling relevant patient data from patient registers including those from private
providers, community and outreach programs into health facility registers,
23
Analyzing the quality of all patient registers, community and outreach reports received and
ensuring follow-up in case of incompleteness, problems with validity, as well as delays;
Compiling all reports from the sections/units/departments into a single health facility report using the
health facility HMIS database?
Plotting monthly performance on the displayed monitoring graphs;
Preparing an analysis of the data for discussion within the health facility for decision-making and
participating in the discussions;
Forwarding or delivering the health facility report to the DHO/RHO by the 7th day of the following
month. In case of IDSR, weekly reports should be forwarded every Saturday;
Providing quarterly feed-back on data management to the sections/units/departments and
community;
Disseminating monthly performance reports during monthly facility meetings;
Disseminating quarterly and annual facility performance reports to the health facility forum;
2.6 THE HSSPs MONITORING AND REVIEW PROCESSES:
2.6.1 Brief Introduction
The framework for reviewing health progress and performance covers the M.E process from routine
performance monitoring, quarterly reviews, annual review and evaluation of all the HSSP indicator
domains. Specific questions will have to be answered during the different review processes, especially the
annual reviews, but also the performance monitoring.
Health progress and performance assessment will bring together the different dimensions of quantitative
and qualitative analyses and will include analyses on: (i) progress towards the HSSP goals; (ii) equity (iii)
efficiency; (iv) qualitative analyses of contextual changes; and (v) benchmarking.
Table 4: Monitoring, Review and Evaluation Processes
Methodology Frequency Output Focus Level of monitoring
and review
Performance
review
meeting
Quarterly Quarterly progress
reports;
Done by Joint (Government/
Partners). A review of progress
against targets and planned
activities.
Inputs, process,
and output
Joint annual
review and
planning
Annually Annual progress
reports,
Done Jointly with development
Partner, key stakeholders, and
planning entities as from district
level onwards. A review of progress
against set target outcomes
Input, process,
output, and
outcome levels
Mid Term
Review
After 2
years
Midterm review
report
Done by sector review progress
against planned impact
Input, process,
output, outcome
and impact levels
End Term
Evaluation
At end of
HSSP
End term
evaluation report
Independent review of progress,
against planned impact
Input, output,
outcome and
impact levels
2.6.2 Performance Monitoring and Review at Central Level:
The performance monitoring and review at central level will review the monitoring reports and recommend
action to the Top Management, Technical Departments and Working Groups as appropriate. The
monitoring and review process will use indicators for the HSSPs for monitoring overall performance of the
health sector. Central level institutions and departments will submit their periodic performance reports to
the Department of Planning. This will be reviewed by the Department of Planning and request for
clarifications as necessary. The Quarterly Sector Performance Review will assess progress on the quarterly
work plans of MoH departments and programs. The aim of these reviews are to:
Assess progress made on action points/recommendations of previous quarterly reviews;
Assess implementation of planned activities against set targets.
Highlight budget performance during the quarter (utilization against allocation).
24
Propose strategies to address challenges in subsequent quarters.
This shall be carried out using a standardized reporting format during the quarterly performance review
meetings, where reports produced by each department/institution will be presented and discussed in
plenary sessions. The meetings will be attended by representatives of all reporting units, programs and
development partners. The Department of Planning will be responsible for organizing the quarterly review
meetings, compiling and disseminating the quarterly reports. The compiled quarterly reports will feed into
the AHSPR.
2.6.3 Performance Monitoring and Review at Regional Level:
The Regional Health Departments shall be responsible for conducting quarterly performance review
meetings in their respective regions, where reports produced by each district in the catchment area will be
presented and discussed in plenary sessions. The meetings will be attended by representatives of all
reporting districts, representatives from central Ministry of Health, development partners, regional
stakeholders’ forum. The fora will use district and regional data to discuss performance within the region,
and agree on priorities to guide districts and to regions in their respective planning and implementation
processes. Department of planning shall provide standardized planning and reporting formats to all the
regional health officers, to guide them in their stakeholders meetings.
2.6.4 Performance Monitoring and Review at Health Facility Level:
Performance monitoring and review in health facilities (including private health facilities will be carried out
using standard planning and reporting formats to be developed by department of planning. The Health
Information Officer in the health facility or where not available a designated person shall be responsible for
compilation of all relevant data from patient registers and compile a health facility report. Health facility In
Charges are responsible for verification and analysis of administrative and service delivery reports. The
generated reports shall be used for health facility performance review and improvement, planning, and
resource mobilization. The Health Information Officer or designated person is responsible for submission of
the health facility reports to the DHO or where not available to the RHO. Each report should be received at
the DHO or RHO office by the date due. The monthly health facility meetings performance review should
focus on timeliness, completeness and accuracy of the reports. The health facility quarterly assessment
reports will be used for performance review during the quarterly district and regional meetings.
Health facility stakeholders’ fora Each health facility in the country needs a defined catchment area, for
which it is responsible for coordinating delivery of services to implement the HSSPs. All stakeholders in the
catchment area of the facility shall come together to discuss health and health related issues affecting
them quarterly each year. The fora shall use health unit data to discuss performance of health within the
catchment area, and agree on priorities to guide the facility and other service providers in their respective
planning and implementation processes. The stakeholders' meetings shall be coordinated by the head of
the Health Facility. The RHOs shall provide standardized planning and reporting formats to all the health
facilities, to guide them in their stakeholders meeting.
2.6.5 Joint Annual Review
The JAR is a national mission for reviewing sector performance annually. The annual reviews will focus on
assessing performance during the previous fiscal year, and determining actions and spending plans for the
year ahead (current year+1). These actions and spending should be addressed in amendments to the
HSSPs. Annual Sector Reviews should be completed by the 30th September each year, to ensure that the
findings feed into the planning and budget process of the coming year. The annual review shall be
organized by the MoH (Department of Planning) in collaboration with Health Development Partners. The
proceedings of the JAR will be documented and signed by the MoH and DPs.
2.6.6 Programs/Projects Reviews
Detailed program/project specific reviews shall be linked to the overall health sector review processes and
contribute to it. Program/project specific reviews should be conducted prior to the overall health sector
review, and help inform the content of the health sector review in relation to that specific program/project
area. It is important that the specific program/project reviews involve staff and researchers not involved in
the program/project itself to obtain an objective view of progress. Progress review reports shall be
submitted to the MoH M.E & Research unit of the Department of Planning in order to inform quarterly and
annual sector reviews as well as evaluation exercises.
25
2.6.8 Performance Monitoring and Review of Implementing Partners
Implementing partners contribute significantly to health service delivery in the country. Most times their
input and attribution to health outcomes is not captured in the sector performance reports. In order to
measure their contribution to the overall sector performance they will be required to report to the relevant
sector programs and departments. The MoH M.E & Research Unit will play a coordination role in monitoring
all national level Implementing Partners to ensure alignment with national priorities. RHO will coordinate
monitoring and reporting of region-based Implementing Partners. Statistical outputs from these partners
should be routed through the Regional HMIS. Department of Planning will support this process through
providing the necessary M.E tools and reporting format.
2.6.9 Performance monitoring and review for global health grants
The HSSP has both core and program specific indicators. Whereas the main purpose of this M.E plan is to
provide a framework for monitoring the HSSP core indicators, program managers and other stakeholders
are encouraged to develop and implement program/project specific M.E plans for monitoring their
indicators and performance reviews with linkage to the general health sector review outlined in this plan.
Under the Global Health Initiatives, the health sector is supported through initiatives like the Global Fund for
Tuberculosis, HIV/AIDS and Malaria (GFTAM) and Global Alliance for Vaccines and Immunization (GAVI)
which provide funds based on performance. There are other sector support programs/projects which also
disburse funds such as JHNP, Health Consortium Somalia and NGOs which contribute to the overall sector
performance. All these require M.E Plans. These M.E plans shall be carried out within the National M.E
framework and plan using tools that consider outputs and indicators to be drawn from approved work-
plans and budgets for the HSSP. Program specific indicators from program strategic plans and or M.E plans
will be used to supplement the national level indicators in monitoring specific program performance.
Program managers will provide oversight for monitoring implementation of work plans and preparation of
quarterly and annual performance reports. The M.E focal persons (specialists) of those programs will work
under the overall stewardship of the M.E. & Research Unit. They will be responsible for analyzing data and
assembling reports that will be reviewed and verified by program managers before submission to the M.E &
Research Unit and relevant working groups. Specifically these programs (GFTAM, JHNP, GAVI, etc); will use
and adopt the existing framework for M.E performance by using agreed indicators. Implementers at
national and local levels will prepare technical reports on a quarterly basis. Further, annual performance
reports shall be prepared and presented at the JAR and other sector review meetings that capture the
overall performance of the HSSP.
2.6.10 Performance Monitoring and Review for Civil Society Organizations and Private Sector
CSOs and the private sector contribute significantly to health service delivery in the country. Most times
their input is not captured in the sector performance reports. In order to measure their contribution to the
overall sector performance they will be required to report to the relevant sector entities using the existing
monitoring and review structures. The CSO and Private Providers. Department of Planning will support this
process through providing the necessary M.E tools.
2.6.11 Routine Feedback to Sub-National Levels and Stakeholders
The advantages of routine performance feedback include; helping local managers, supervisors and
implementers to consider what their own strengths and weaknesses are, and where they need to be
making more of an effort. Secondly, for those collecting the information, seeing how that data is used, and
how it can assist their own work and that of their colleagues, helps to motivate them to improve the quality
of information they provide. Feedback on performance shall be provided for national, regional and
institutional level performance on quarterly basis through performance review meetings and reports.
2.7 HSSP EVALUATION
2.7.1 Program/Project Evaluation
A number of health sector investment and intervention projects will be undertaken during the period of the
HSSP 2013-2016. All projects will be subjected to rigorous evaluation. The type of evaluation to be planned
26
for and conducted should reflect the nature and scope of the investment. For example, pilot projects that
are being conducted amongst a random group of participants shall be selected for impact evaluation to
determine whether or not the investment should be scaled up. As a minimum requirement, each project in
this category will be required to conduct the following:
A baseline study during the preparatory design phase of the project or the program;
A mid-term review at the mid-point in the project to assess progress against objectives and provide
recommendations for corrective measures;
A final evaluation or value-for-money (VFM) audit at the end of the project. A VFM audit will be
carried out for key front-line service delivery projects where value for money is identified as a primary
criterion. All other projects will be subjected to standard rigorous final evaluation.
The MoHs through the specific program/project managers will be responsible for the design, management
and follow-up of the program and project evaluations (including baseline and mid-term reviews). All
projects are required to budget for periodic project evaluations. All project evaluations will be conducted
by external evaluators to ensure independence. Program/Project evaluation reports shall be disseminated
during the sector quarterly and annual review meetings.
2.7.3 Mid - Term Review
A Mid-Term Review of the HSSP 2013-2016 will be done after two year. The purpose of the MTR is to review
the progress of implementation; identify and propose adjustments to the HSSP and other government
policies as required. The specific objectives of the MTR are to:
Assess progress in meeting HSSPs targets and to make recommendations for their adjustment if found
necessary;
Review the appropriateness of outputs in terms of inputs, processes and desired outcomes;
Review the costing and financing mechanisms of the HSSPs; and
Coordinate the MTR process with the NDP review.
The MTR shall entail extensive review of documents including routine reports and recent studies in the
sector; special in-depth studies may also be commissioned as part of the MTR; and interviews with selected
key stakeholders. The MTR is undertaken in a participatory manner involving government line ministries,
national level institutions, service delivery levels, DPs, civil society, private sector and academia. The
analysis will focus on progress of the entire sector against planned impact, but will also include an
assessment of inputs, processes, outputs and outcomes, using the HSSP indicators (core and others). The
main result will be a list of recommendations for the remaining HSSP years.
The MTR will be an internal, joint exercise involving all stakeholders. The overall responsibility of the process
will be with the Director of Planning Department. M.E. Advisory Committee shall be constituted to support
and co-ordinate the MTR process. The secretariat is to be located in the Health Planning Department of the
MoH. The review shall be carried out by the Technical Working Groups. Each TWG will be responsible to
undertake a review according their specific Terms of Reference. Issues not covered by any specific working
group will be a responsibility of the MTR Team. External facilitation may be required to address critical issues
identified by the Working Groups.
2.7.4 HSSP End Term Evaluation
The End Term Evaluation will be conducted in the second half of 2016 (Three and half years of the HSSP
implementation) in order to enable the sector to make use of its findings and recommendations for the
formulation of the next strategic plan. Like the mid-term review, the analysis will focus on progress of the
entire sector against planned impact, but will also include an assessment of inputs, processes, outputs and
outcomes, using the HSSP indicators (core and others). It will focus on expected and achieved
accomplishments, examining the results chain, processes, contextual factors and causality, in order to
understand achievements or the lack thereof. The evaluation will have to answer questions of attribution
(what made the difference?) and counterfactual (what would have happened if we had not done A or
B?) and take into account contextual changes (economic growth, social changes, environmental factors
etc.), as well as policies and resource flows:
a. Relevance: Did the HSSP address priority problems faced by the target areas and communities?,
was the HSSP consistent with policies of both the Government and Health Development Partners?
b. Economy: Have the HSSP inputs (financial, human, Assets etc) been applied optimally in the
implementation process?
27
c. Efficiency: Were inputs (staff, time, money, equipment) used in the best possible way to maximize
the ratio of input/outputs in HSSP implementation and achieve enhanced outputs; or could
implementation have been improved/was there a better way of doing things?
d. Effectiveness: Have planned HSSP outputs and outcomes been achieved?
e. Efficacy: To what extent have been the achievements of the HSSP objectives and goal?
f. Impact: What has been the contribution of the HSSP to the higher level development goals, in
respect of national development goals; did the HSSP have any negative or unforeseen
consequences?
The evaluation will be conducted by a team of independent in-country institutions in close collaboration
with international consultants. The purpose of conducting the evaluation prior to the conclusion of the HSSP
is to generate lessons and recommendations to inform the next HSSP
2.8 SURVEYS
2.8.1 Brief Introduction
Surveys will be conducted where there are information gaps or outdated information throughout the HSSP
period as a basis to confirm the occurrence of change. Surveys shall be carried out at all levels to provide
basis for a 'before and after' assessment of the HSSP progress. The results of these studies are supposed to
inform decision making hence contribute to improving delivery of and access to health care and nutrition
services.
2.8.2 Health Research and Evidence Generation
The HSSPs acknowledges that a lot of research is conducted in the country. The results of these studies are
rarely shared to inform decision making hence contribute to improving delivery of and access to health
care and nutrition services. Operational researches shall be carried as planned. Operational research shall
encompass a wide range of problem-solving techniques and methods applied in the pursuit of improved
decision-making and efficiency. The M.E. and Research Unit of MOHs shall be responsible for coordinating
all the health related research in the country. Health Research Committee will be set up to oversee the
health research related activities and mobilize resources, setting, updating and disseminating health
services research agenda, commissioning and organizing health research in collaboration with other
research and academic institutions and NGOs. The M.E. and Research Unit shall coordinate the sharing of
research findings in the MoH by liaising with research institutions, universities and Department of Statistics of
the Ministry of Planning.
2.8.3 Surveys to be commissioned by the MoH
These may be carried out directly by programs under the MoH or contracted out. These will include; the
Antenatal HIV sentinel surveillance, malaria indicator survey, availability of the six tracer medicines study,
client satisfaction surveys, facility assessment for service delivery, health, human rights and gender survey as
well as other operational surveys and researches:
1. HIV/AIDS Epidemiological Surveillance
Active surveillance of HIV infection based on biannual ANC-based HIV sero-prevalence surveys will be
conducted in sentinel clinics distributed in all geographic areas of the country. The methodology of ANC
HIV surveillance involves anonymous and unlinked HIV serological testing of residual blood samples after
performing routine ANC serological testing for syphilis at the clinics. Blood specimens are collected from
mothers attending their first visit for ANC during a defined survey period. The sampling frame is consecutive
and therefore all eligible women who present at the sentinel sites in the sampling period are sampled. A
minimum of 300 samples are collected from each sites. However, in high volume clinics in major urban
areas, deliberate over sampling is conducted to obtain at least 500 samples from each clinic to permit
stratified analyses. A sampling period of ten weeks is observed simultaneously in all clinics. The
epidemiological data will be collected bi-annually and presented in an annual report based on calendar
year.
2. Six tracer medicines study (Measles, Paracetamol, Amoxicillin, COC, Multi-Micronutrient, ORS)
28
Availability of the six tracer medicines is one of the core indicators of HSSP to be monitored at facility level
monthly through the HMIS. At national level availability of the six tracer medicines will also be monitored
annually through the six tracer medicine availability study conducted in selected districts by the MoH
Pharmaceutical Department. The findings of the six tracer medicines availability study will be analyzed and
reported in the AHSPRs. The information generated shall be used for improving the supply chain
management and feedback will be provided to the regions, districts and health facilities.
3. Annual Facility Survey
This is an annual system of health-facility assessments, including an assessment of service readiness
combined with a record review. This will serve to fill critical data gaps on service delivery as well as serve to
verify the utilization and the quality of the services provided to the public (clients). Every year, prior to the
JAR, a facility assessment should be conducted in a sample of facilities to independently review the quality
of health data and the status of service delivery. Such assessment will provide essential information on
service delivery (such as the availability of key human and infrastructure resources, essential medicines,
and on the readiness of health facilities to provide basic health-care interventions relating to maternal and
neonatal health, child health, communicable and non-communicable diseases, etc
4. The Somali Malaria Indicator Survey
The Somali Malaria Indicator Survey is designed to determine the progress made in malaria control and
prevention in the country. It provides data on key malaria indicators. The MIS will be carried out every two
to three years by the Malaria Control Program, using a nationally representative sample of households. The
Survey will provide some of the core HSSP indicators as well as national and regional estimates of a range
of malaria indicators and thus provides a robust and comprehensive picture of malaria control in the
country. It captures both biological and behavioral information relevant to malaria and will provide a
useful reference tool and evidence base for national policy decision making.
5. National Tuberculosis prevalence survey
This population-based TB disease prevalence survey is aimed at establishing an accurate estimate of the
burden of TB disease; estimating the age-sex distribution of prevalent TB cases; establishing the proportion
of prevalent cases found by the health system; and comparing the health seeking behavior of prevalent
cases that were not being treated to those being treated by the health system. This will be a cross sectional
descriptive study targeting adults aged 15 years and above from all over the whole country. Capture-re-
capture
6. Client Satisfaction Survey
Client satisfaction surveys will be carried out at all levels of service delivery to determine the quality of
services offered in the client perspective. A client satisfaction survey tool shall be developed and
incorporated into the HMIS. Facility client satisfaction surveys will be carried out biannually (December and
June every year) and findings utilized for quality improvement. Population-based national client satisfaction
survey will be needed to provide the baseline, mid-term and end of HSSP indicators.
7. Non-Communicable Diseases (NCD) Survey
During HSSP, the MoH will initiate a plan of conducting a national NCD baseline survey on risk factors and
magnitude of non-communicable diseases in the country. The NCD baseline survey will provide baseline
data on the prevalence of NCDs and their risk factors. This should result in the formulation of evidence
based national NCD policies and strategies as well as the development of a comprehensive and
integrated action plan against NCDs in our population. Other relevant surveys may be initiated by the MoH
during the course of implementation of the HSSP. Research activities by academic institutions that are
related to the survey agenda of the health sector and HSSP M.E Plan will also contribute in this regard.
8. Demographic and Health Surveys (DHS)
DHS surveys are nationally-representative household surveys that provide data for a wide range of
monitoring and impact evaluation indicators in the areas of population, health, and nutrition. There are two
main types of DHS Surveys:
Standard DHS Surveys have large sample sizes (usually between 5,000 and 30,000 households) and
typically are conducted about every 5 years, to allow comparisons over time.
29
Interim DHS Surveys focus on the collection of information on key performance monitoring
indicators but may not include data for all impact evaluation measures (such as mortality rates).
These surveys are conducted between rounds of DHS surveys and have shorter questionnaires than
DHS surveys. Although nationally representative, these surveys generally have smaller samples than
DHS surveys.
DHS Survey Topics includes modules on Anemia, Child and Newborn Health, Domestic Violence ,
Education, Environmental Health - water, sanitation, cooking fuel, Family Planning, Female Genital Cutting,
Fertility and Fertility Preferences, Gender/Domestic Violence, HIV/AIDS Knowledge, Attitudes, and Behavior,
HIV Prevalence, Household and Respondent Characteristics, Infant and Child Mortality, Malaria, Maternal
Health, Maternal Mortality, Nutrition, Tobacco Use, Unmet Need, Wealth, Women's Empowerment, fistula,
health expenditures among others.
9. Operational Researches
9.1 Health seeking behaviour
9.2. Access of basic health care services by nomadic people
2.8.9 KNOWLEDGE MANAGEMENT
During the course of implementing HSSP, MoHs will introduce a comprehensive knowledge management
approach in the sector. This should guide a comprehensive look at information needs, analysis and use to
better guide decision making for health. The definition of a comprehensive performance monitoring
approach for the sector, using input, output, outcome and impact indicators as outlined in the HSSP M.E
Plan should be able to generate adequate information for analysis and use. Data and information
generated at all levels of the sector and from different sources will be shared, translated and applied for
decision making during routine monitoring, periodic sector performance review, planning, resource
mobilization and allocation, accountability, designing disease specific interventions, policy dialogue,
review and development. Effective knowledge management will be based on the following assumptions:
First all relevant data will be aggregated, synthesized and analyzed for use at various levels of the
sector;
Second is that all reports produced through M.E activities, once approved, will be made easily
accessible and in a timely manner to all stakeholders, including citizens;
Thirdly all M.E results users should be able to translate and use the data/information for decision
making, policy dialogue, review and development;
III. MONITORING AND EVALUATION PLAN
3.1 Brief Introduction
The M.E Plan describes the strategic objectives, key milestones for the next three years as well as priority
activities and the costs required to implement those priority activities. It also describes the roles and
responsibilities for key stakeholders at all levels. It articulates results framework and detailed implementation
plan with estimated budget for the next three years. It also considers key tasks and assumptions necessary
for the effective implementation of the plan. It defines the key indicators of monitoring the performance
and the implementation of the M.E plan.
3.2 Strategic Objectives:
Objective 1: To establish an M.E system to track HSSP implementation and impact by 2016,
Key milestones under this objective include regular performance monitoring and reviews at all levels, joint
annual review by the MOHs and all DPs, specific programs or projects reviews including global health
grants and their implementing partners as well as mid-term and final evaluation of the HSSP
30
Objective 2: Timely, complete and accurate HMIS at all levels by 2016.
Important milestones will begin with strengthening data collection, analysis and reporting. Information from
vertical programs' will be integrated into the HMIS. New data-base structures will be designed and the
International Classification of Diseases (ICD) 10 introduced. HMIS reporting will be expanded to Primary
Health Units and to the private sector. Appropriate reporting system will developed for key health systems
such as HR, finance, supplies which will be integrated into the HMIS. Training in data management
including analysis, interpretation and use for planning will be undertaken. Appropriate ICT technology will
be made available for proper data management at all levels.
Objective 3: Effective early warning and surveillance system at all levels by 2016.
The nutrition surveillance system will be integrated into the HMIS, the sentinel sites for IDSR and nutrition
surveillance will be reviewed and updated and the capacity of MOH for the collection, analysis and
reporting surveillance data will be strengthened,
Objective 4: Establish a “survey and research agenda” by 2016.
Important milestones under this objective include an inventory of all health and nutrition studies,
establishing a Surveys and Research Committee to oversee the health related research activities in the
country, development of standard tools, protocols and methodologies, implementation of population-
based, facility-based and school-based surveys as well as other priority operational research activities
Objective 5: Vital registration (birth and death) system in Somalia by 2016.
A vital registration will be established, piloted and gradually rolled out to all regions
Objective 6: Enhance governance, institutional capacity, partnership and coordination by 2016
Key milestones under this objective include the development of an M.E. or HMIS policy and legal framework,
establishment of M.E. advisory committee and competent M.E. and Research Units at national and regional
levels, training of directors and program managers in M.E. disciplines, etc
Objective 7: Improve data demand, dissemination, communication and use at all levels by 2016
Important milestones under this objective include use of various communication channels to ensure public
access to data and reports, development of email accounts for all regions and health facilities,
development of MOH websites, regular feedback workshops for survey findings as well as review and
evaluation results, printing of reports, etc
Objective 8: Use geographic information system for the health sector planning,
key milestones under this objective will include the mapping of all health facilities using GIS technology,
defining the demarcations of the catchment areas and target populations of all health facilities, analyzing
coverage of general or specific health services in relation to need and how these services are related to
communities. The system will be used to generate maps and other graphics (like bar and line graphs) that
show which areas are meeting the targets or are lagging behind.
3.3 RESULTS FRAMEWORK
Baseline
2013
Milestones
2014 2015 2016
Objective 1: To establish an M.E system to track the progress of the HSSP implementation and its impact by 2016,
There is an M.E units in each of the
three MOHs but with limited
capacities. However, plans are
underway in recruiting M.E. advisors to
build the capacity of the three MOHs
There is regular coordination meetings
organized by Ministries of Health and
participated by IPs and DPs, however
these meetings do not focus on the
performance review and evaluation of
the sector plans,
The preparation of the first ever JAR is
Conduct 2013 JAR,
Establish strong M.E. Units staffed
with competent and skilled staff in
the three MOHs
Undertake quarterly performance
review meetings
Develop the plan and conduct
2014 JAR,
Strengthen the
capacity of the M.E.
Units,
Continue quarterly
performance review
meetings
Develop the plan and
conduct mid-term
review of the HSSP
Continue
quarterly
performance
review meetings
Develop the plan
and conduct
independent final
evaluation of the
HSSP
31
underway,
Programs and projects have their own
arrangements of performance
appraisals and evaluations which are
not linked to the HSSP,
Indicator Percentage of HSSP results-framework indicators reported in the AHSPR
Source AHSPR
Responsibility M.E & Research Unit
Indicator The proportion of implementing partners (NGOs, CSOs, Private sector) contributing
to periodic reports;
Source AHSPR
Responsibility M.E & Research Unit
Indicator The proportion of planned periodic reviews that are carried out;
Source Review Reports
Responsibility M.E & Research Unit
Objective 2: Timely, complete and accurate HMIS at all levels by 2016.
The HMIS is functioning in Somaliland
and Puntland, It needs to be rolled out
to South-Central zone.
The system currently uses excel-based
tool at regional and central levels
which is not able to generate
comprehensive reports. At facility
level, the system is managed
manually,
Use of standard case definitions is
minimal or non-existent at all levels,
Data quality audit and feedback
mechanism is very weak,
Data from private health sector is
completely no captured,
Data for some programs are still
vertical (TB, HIV/AIDS, Nutrition, etc),
There is no data collection system at
the community level,
Information on infrastructure, supplies
and logistics, human resources,
finance, etc are not linked with the
HMIS,
Design HMIS database using
Microsoft Access which is more
stable, flexible, and user-friendly
system,
Pilot the database into selected
areas,
Rollout the HMIS into South/Central
Zone,
Introduce ICD version 10 and train
health workers in public and
private sectors,
Rollout the HMIS into PHU level in
line with the rollout plan of the
EPHS,
Develop strategy of introducing
HMIS into private sector,
Develop a strategy of linking HRIS,
SLMIS, infrastructure and finance
information into the routine HMIS,
Develop strategy of collecting
community-based HMIS,
Introduce computerized data
management at facility level in
selected regions,
Review, update and
finalize the data-base
structures and rollout to
all regions,
Expand the
implementation of the
HMIS into PHU,
Implement HMIS in the
private sector,
Define the catchment
areas and target
population of all health
facilities across the
country,
Implement the strategy
of linking key health
systems information
into the HMIS,
Pilot community-based
HMIS in selected areas,
Expand the distribution
of computers to
support data entry and
analysis at facility level,
Expand the
implementation
of the HMIS in the
private sector,
Rollout the
implementation
of the
community-based
HMIS,
Implement the
strategy of linking
key health
systems
information into
the HMIS,
Ensure availability
of computers in
all health facilities,
Indicator HMIS completeness rate
Source Monthly, quarterly and annual health statistics report (HMIS)
Responsibility HMIS Office
Indicator HMIS timeliness rate
Source Monthly, quarterly and annual health statistics report (HMIS)
Responsibility HMIS Office
Indicator HMIS reporting rate
Source Monthly, quarterly and annual health statistics report (HMIS)
Responsibility HMIS Office
Objective 3: Effective early warning and surveillance system at all levels by 2016.
IDRS system is in place, but not
integrated into the HMIS,
Nutrition surveillance system is up and
running, but is managed by FSNAU
and remains vertical,
The data collection is mainly
Review and integrate the existing
disease and nutrition surveillance
systems at all levels,
Develop a plan of action of
strengthening disease and
nutrition surveillance system across
Implement the action
plan of strengthening
IDRS and nutrition
surveillance
Pilot the
implementation of the
Continue the
implementation
of the action plan
Expand the
implementation
of community-
32
managed through phone calls from
facility staff to the regional and
national surveillance officers,
There are separate surveillance
officers in EPI, nutrition and other
notifiable diseases,
There is no demographic surveillance
system and sentinel sites in the country,
the country,
Develop a strategy of community-
based IDSR and nutrition
surveillance,
Develop demographic
surveillance strategy in
collaboration with Department of
Statistics of Ministry of Planning
and Academic Institutions,
Introduce and pilot hotlines
(mobile reporting) eDEWS
community-based IDSR
and nutrition
surveillance system,
Implement eDEWS in
selected regions,
Pilot the demographic
surveillance strategy in
selected sentinel sites,
based IDSR and
nutrition
Review and
rollout eDEWS
across the
country,
Review and
expand the
demographic
surveillance
system,
Indicator Number of IDSR weekly reports submitted and disseminated on time;
Source Monthly, quarterly and annual health statistics report (HMIS)
Responsibility HMIS
Indicator Availability of community-based IDSR and nutrition surveillance strategy
Source Policy and Strategy Documentation Centre
Responsibility Director of Planning
Objective 4: Establish a “survey and research agenda” by 2016.
There is research unit in Somaliland
under the department of planning,
Research and survey related activities
are not coordinated, results are rarely
shared and used,
there is no agreed upon research
agenda and plan for the sector,
There are no standard protocols, tools
and methodologies used for research
activities across the country,
The capacity of the research
institutions in the country is very weak,
Develop a comprehensive
research plan for the health sector
based on the information needs of
the HSSP,
Establish a research coordination
mechanism in the country,
Develop capacity building plan
for the research institutions and
universities in the country,
Undertake priority research and
survey activities according to the
research and survey plan
Undertake priority
research and survey
activities as per the
HSSP,
Implement the
capacity building plan
for research
coordinating
committee, institutions
and universities,
Undertake priority
research and
survey activities as
per the HSSP,
Implement the
capacity building
plan for research
institutions and
universities,
Indicator Availability of prioritized national research agenda and costed survey plan;
Source Policy and Strategy Documentation Centre
Responsibility Director of Planning
Indicator Proportion of planned surveys and research studies carried out;
Source Research and Survey Reports
Responsibility M.E. and Research Unit
Objective 5: Vital registration (birth and death) system in the country by 2016.
There is no vital registration system up
and running in the country,
There is an inter-ministerial committee
working on the development of the
civil registration and vital statistics
system in Somaliland,
Establish a coordinating
committee for civil registration,
Develop strategy for the
introduction of the civil registration
system at health facility and
community level
Pilot the civil registration system in
selected regions,
Review the civil
registration strategy,
Implement the civil
registration system in
the EPHS regions,
Rollout the civil
registration system
in all the regions
Indicator Availability of community and facility-based Vital Registration System linked to
HMIS;
Source HMIS
Responsibility Director of Planning
Indicator Number of births and deaths reported annually;
Source Monthly, quarterly and annual health statistics report (HMIS)
Responsibility HMIS
Objective 6: Enhance governance, institutional capacity, partnership and coordination by 2016
There are HMIS officers at central and
regional levels,
There is a functioning research unit
only in Somaliland,
Develop policy and legal
framework for the health
information system of the country,
Review the current structures and
Print and disseminate
policy and legal
framework documents
to all health facilities,
Strengthen the
capacity of the
management
and coordination
33
There are no policies or legal
framework supporting the functioning
of the system,
There are no clear roles and
responsibilities for the proper
management of the data and
information at all levels,
There is no standard operating
procedures for the health information
system of the country,
functions and develop the
institutional framework of the
health information system,
Establish advisory committee to
oversee the development of the
health information system,
Develop SOP for the proper
management of the system at all
level,
Develop training plan for
managers and health workers in
M.E. disciplines
institutions and
stakeholders,
Strengthen the
capacity of the
management and
coordination structures
at all levels,
Train managers and
health workers on M.E,
structures at all
levels,
Continue training
of managers and
health workers on
M.E.
Indicator # of managers and health workers trained in M.E
Source Training reports
Responsibility Training Department
Indicator Availability of policy and legal framework to govern the health information system
of the country,
Source Policy and strategy documentation centre
Responsibility Director of Planning
Objective 7: Improve data demand, dissemination, communication and use at all levels by 2016
Currently, data demand and
information use is very minimal,
Data is rarely disseminated and
shared,
There is no effective dissemination
plan nor communication strategy,
There are no comprehensive statistical
or performance reports produced and
disseminated,
HMIS offices produce monthly bulletin
on certain indicators of service
statistics,
Develop strategies for data
dissemination and
communication,
Establish websites and email
accounts,
Develop media programs for
public access to information,
Produce periodic reports,
Train health workers and
managers on data demand and
information use,
Implement strategy for
data demand and
information use,
Maintain and update
the website,
Implement media
program using various
channels (printed and
electronic media)
Continue the training
of managers and
health workers on data
demand and
information use
Continue the
implementation
of the strategy,
Maintain and
update the
website,
Continue the
implementation
of the media
program,
Indicator Proportion of statistical and performance reports produced and disseminated
Source HMIS, M.E. reports
Responsibility Director of Planning
Indicator Availability of regularly updated MOH websites
Source MOH
Responsibility ICT unit
Objective 8: Use geographic information system for the health sector planning,
Currently, there is no proper mapping
of the available health facilities and
services in both public and private
sector,
There are no defined demarcations of
the catchment areas and target
populations of each and every health
facility across the country,
There are no master lists or data-base
of the available public and private
health facilities in the country,
Undertake mapping of all public
and private sector facilities and
establish data-base,
Develop a plan of action in
defining the catchment areas and
target populations of health
facilities,
Finalize the data-base
(develop the master
lists of all public and
private health
facilities),
Implement the action
plan of defining the
catchment areas and
target populations of
the health facilities in
the EPHS regions
Expand the work
of defining the
catchment areas
and target
populations of the
health facilities to
all regions,
Indicator Proportion of health facilities with defined catchment areas and target population
Source Annual facility survey
Responsibility Regional Health Officers
34
3.5 The Key M.E Plan Implementation Tasks and Assumptions
The key M.E plan implementation tasks will include:
Ensuring that programs assign positions responsible for statistical production, monitoring and
evaluation,
Ensuring that HMIS and other data collection systems and tools are in place and functioning,
Training of health workers and managers in M.E,
Holding quarterly performance review meetings to determine progress towards output targets,
Ensuring proper coordination and oversight (monitoring and supervision) of M.E activities in the
sector,
Planning and budgeting for monitoring and evaluations of all projects and programs. Minimum of
5% of each project budget will be allocated to monitoring and evaluation,
Utilizing M.E findings to inform program, policy, and resource allocation decisions,
Maintaining a Recommendation Implementation Tracking Plan which will keep track of review and
evaluation recommendations, agreed follow-up actions, and status of these actions,
Ensuring that complete and approved M.E reports and health statistical data are made easily
available to the public in a timely manner,
3.6 The M.E Plan Performance Indicators
The following indicators will be used for monitoring implementation of the M.E Plan for HSSP,
Availability of comprehensive M.E plan for HSSP,
Number of copies of the M.E plan disseminated to the public, private health sector and other
stakeholders,
Number of health managers and workers trained in M.E,
The proportion of implementing partners (CSOs and Private Sector) contributing to periodic reports,
Proportion of planned M.E support supervision visits carried out,
Proportion of planned data quality audits conducted,
Proportion of departments, programs, semi-autonomous institutions, CSOs and health facilities
submitting timely and complete reports,
Proportion of planned periodic reviews that are carried out,
Proportion of planned performance review reports compiled and disseminated,
Proportion of planned surveys carried out,
Proportion of planned survey reports compiled and disseminated,
Number of policy/decision makers oriented in knowledge management methods,
Number of performance improvement, planning and resource allocation decisions made based
on the M.E results,
Number of evidence based policy dialogues or briefs made,
3.7 Roles and Responsibilities for the HSSP M.E Plan Implementation
The M.E framework and plan will follow the effective tracking, evaluation and feedback on HSSP 2013 -
2016 implementation. This implies that all stakeholders will be involved directly or indirectly in the M.E
activities. Consequently, a participatory approach that entails the involvement of all key actors and
primary stakeholders will be adopted. This will enable all key actors to fully internalize and own the system
as well as use the results to inform their actions. All other monitoring plans in the sector should be in line with
and input into the overall M.E plan for HSSP both at national, regional and district levels. In order to avoid
over-laps, role conflicts, and uncertainty in the M.E function during the implementation of the HSSP, roles
and responsibilities of key actors are specified below.
35
PROPOSED STRUCTURE
Cabinet/Parliament The sector shall work closely with the relevant committees of parliament and cabinet
for;
Overall political, and policy oversight;
Review of sector progress in the past year (based on the AHSPR), against the policy imperatives set
out in contribution towards the NHP, HSSP and NDP;
The health sector shall interface with parliament and cabinet whenever necessary but in any case,
following the JAR of the Health Sector.
Top Management: The top management (Minister, DG and Directors of the Departments) of the Ministry will
be responsible for;
Overall political, and policy oversight in the sector;
Providing governance and partnership oversight to the sector;
Reviewing of sector progress in the past year (based on the AHSPR), against the policy imperatives set
out in the NHP, HSSP and NDP;
Monitoring adherence to the policy direction of the sector;
TOP MANAGEM
ENT M.E ADVISORY
COMMITTEE
DEPARTMENT
PLANNING M.E SECTION
SURVEILLANCE
HMIS RESEARCH
TECHNICAL DEPTS/W.GS
REGIONS
DISTRICT
HEALTH FACILITY
36
Mobilizing resources for achievement of the sector policy direction;
M.E. Advisory Committee: is a forum for the MOH, development partners, private sector, CSOs, academia
and related line ministries to discuss and advise on key policy and M.E. issues and to advise on the
implementation of the HSSP:
Establishing the results framework for the HSSP, and for ensuring the development of results indicators
that are consistent with the HSSP;
Ensuring proper coordination and oversight (monitoring and supervision) of M.E activities in the sector,
in relation to the NHP, HSSP and NDP;
Training of health workers and managers in M.E.
Quality assurance of the statistical and other performance monitoring reports and surveys;
Organizing regular HSSP and sector review meetings.
Supporting regions to organize regular performance review meetings;
Overseeing the production of the quarterly and annual health sector performance reports;
Maintaining a Recommendation Implementation Tracking Plan which will keep track of review and
evaluation recommendations, agreed follow-up actions, and status of these actions;
Coordinating focused evaluation on emerging concerns and impact assessment studies;
Utilizing M.E findings to inform program, policy, and resource allocation decisions;
M.E & Research Section:
An M.E. and Research Unit will be established under the Department of Planning, which will be responsible
for the overall coordination and implementation of the HSSP M.E plan. The programs' M.E focal
persons/specialists will work under the overall stewardship of the M.E and Research Unit. They will be
responsible for analyzing data and assembling monitoring reports that will be reviewed and verified by
Heads of Departments before submission to the M.E & Research Unit. The M.E & Research unit will use the
statistical information generated by the HMIS, administrative reports, technical supervision reports to
generate and disseminate relevant sector reports.
MoH departments/national referral hospital/semi-autonomous institutions will be centers for performance
monitoring as well as reporting on progress against the HSSP. They will also be the direct consumers of the
outputs and outcomes of this M.E framework and plan. The focus of the central level M.E activities will be
on service delivery, compliance with national standards, outputs and outcomes. The M.E & Research Unit
will be responsible for:
Ensuring proper coordination of monitoring activities at national level;
Providing timely and quality data on relevant performance indicators to the stakeholders;
Training of health workers and managers in M.E;
Coordinating and operationalizing the Health Sector Statistical System at all levels;
Strengthening capacity for collection, validation, analysis, dissemination and utilization of health
statistical data at all levels;
Generating health statistical data on quarterly and annual basis;
Ensuring that complete and approved M.E reports and health statistical data are made easily
available to the public in a timely manner;
Maintaining a Recommendation Implementation Tracking Plan which will keep track of review and
evaluation recommendations, agreed follow-up actions, and status of these action;
Utilizing M.E findings to inform program, policy, and resource allocation decisions.
Technical Working Groups: Actual technical coordination will be through technical working groups, each
focused on specific technical areas. These will be the forums through which technical issues are debated
and agreed and specific recommendations and actions are implemented. Technical Working Groups will
compose of the following:
Health Systems Strengthening WG;
Mother and Child Health WG;
Environmental Health and Health Promotion WG;
National Disease Control WG;
Nutrition WG;
The TWGs will be responsible for;
37
Tracking and coordinating the implementation of the M.E plan and promoting joint monitoring and
evaluation of the HSSP for the respective program areas;
Participating in the JAR/NHA and preparation of the AHSPR;
Submitting reports for discussion during the quarterly and annual review meetings;
Meeting regularly with partners to track progress of achievement of intended HSSP results;
Conducting joint field monitoring to measure achievements and constraints that impede the
realization of the HSSP targets;
Identifying and documenting lessons learnt;
Identifying capacity development needs, particularly in areas of monitoring and evaluation.
There is need to establish resource centers at national and regional levels which will serve as repositories for
health data and information at the respective levels.
The Planning Department: Will be responsible for;
Establishing a competent M.E & Research Unit under the department of planning;
Plan and budget for monitoring, review and evaluation activities annually. A minimum of 5% of each
program or project budget will be allocated to monitoring, reviews and evaluation activities;
Providing on a quarterly and annual basis, data and explanatory information on progress against
performance indicators to top management and stakeholders through quarterly and annual
performance reports;
Utilizing M.E findings to inform program, policy, and resource allocation decisions;
Human Resources Department: Will be responsible for;
Identifying and facilitating recruitment of human resources required to operationalise the M.E. Plan.
This will include recruitment of M.E specialists, as well as statisticians where they are lacking;
Operationalizing the Human Resource Information System;
Utilizing M.E findings to inform program, policy, and resource allocation decisions.
Other Departments, Programs and Projects: Other departments will be centers for performance monitoring
as well as reporting on progress against the targets and actions set out in the HSSP. They will also be the
direct consumers of the outputs and outcomes of this M.E plan. The focus of the MoH M.E activities will be
on service delivery, compliance with national standards, outputs and outcomes. Head of departments will
be responsible for:
Providing oversight for monitoring implementation of work plans and preparation of quarterly and
annual performance reports;
Training of health workers and managers in M.E.
Providing timely and quality data on relevant performance indicators to the M.E and Research Unit
and relevant stakeholders.
Participating in the review and evaluation of the HSSP as well as the preparation of the AHSPR,
JAR/NHA.
Maintaining a Recommendation Implementation Tracking Plan which will keep track of review and
evaluation recommendations, agreed follow-up actions, and status of these actions.
Utilizing M.E findings to inform program, policy, and resource allocation decisions.
Regional Level: The regional level will be responsible for;
Overall coordination of monitoring and evaluation activities in the region.
Liaison between national level and the districts on M.E;
Supporting the development and implementation of the M.E plans of the districts and in the region;
Monitoring and reviewing the implementation of the M.E plans in the region by compiling and
analyzing quarterly and annual reports.;
Maintaining a Recommendation Implementation Tracking Plan which will keep track of review and
evaluation recommendations, agreed follow-up actions, and status of these actions;
Supporting research activities in their respective regions;
Development Partners (i.e. Donors, International Development Agencies): Will be responsible for;
Providing an external perspective on the health sector performance and results;
Participating in the refinement of indicators, tools and processes;
Integrating development partners' monitoring frameworks into Government systems
38
Providing feedback to domestic and international constituencies on health sector performance and
results;
Assisting the health sector through financial, technical and other forms of assistance to strengthen M.E
performance;
Utilizing M.E findings to inform program, policy, and resource allocation decisions.
Other executing agencies (Private Sector): The role of the private sector in the implementation of the HSSP
M.E Plan will be:
Contributing in the development of and adherence to the necessary M.E standards'
Participating in public sector planning processes at all levels;
Providing quarterly performance reports and quality data to the relevant program managers/focal
persons at national and regional level. These will be compiled as part of departmental reports to be
reviewed by relevant working groups for onward transmission to the M.E. and Research Unit;
Participating in discussion and decision-making processes at program, sector and national levels that
review and comment on public sector performance.
Maintaining a Recommendation Implementation Tracking Plan which will keep track of review and
evaluation recommendations, agreed follow-up actions, and status of these actions.
Utilizing M.E findings to inform program, policy, and resource allocation decisions.