document of the world bank...report no: icr00004624 implementation completion and results report...

Document of

The World Bank

FOR OFFICIAL USE ONLY

Report No: ICR00004624

IMPLEMENTATION COMPLETION AND RESULTS REPORT

4771‐KE, 5034‐KE and 5367‐KE

ON A

CREDIT

IN THE AMOUNT OF SDR 128.8 MILLION

(US$198 MILLION EQUIVALENT)

AND ON A

GRANT

MULTI DONOR TRUST FUND FOR HEALTH RESULTS INNOVATION (TF016027)

IN THE AMOUNT OF US$20 MILLION

TO THE

REPUBLIC OF KENYA

FOR A

HEALTH SECTOR SUPPORT PROJECT (P074091) April 29, 2019

Health, Nutrition & Population Global Practice

Africa Region

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

CURRENCY EQUIVALENTS

(Exchange Rate Effective {October 30, 2018})

Currency Unit = Kshs

Kshs 102.05 = US$1

US$1.38 = SDR 1

FISCAL YEAR

July 1 ‐ June 30

Regional Vice President: Hafez M. H. Ghanem

Country Director: Carlos Felipe Jaramillo

Senior Global Practice Director: Timothy Grant Evans

Practice Manager: Magnus Lindelow

Task Team Leader(s): Gandham N.V. Ramana, Jane Chuma

ICR Main Contributor: Peter Okwero

ABBREVIATIONS AND ACRONYMS

AF Additional Financing

AMREF African Medical Research and Education Foundation

AOP Annual Operational Plan

CPF Country Partnership Framework

DARE Decentralized Reproductive Health and HIV/AIDS Project

DHIS2 District Health Information System 2

DHS Demographic and Health Survey

EMMS Essential Medicines and Medical Supplies

GAVI Global Alliance for Vaccines and Immunization

GFATM Global Fund to Fight AIDS, Tuberculosis and Malaria

GoK Government of Kenya

HFMCs Health Facility Management Committees

HIA Health in Africa Initiative

HISP Health Insurance Subsidy Program

HIV/AIDS Human Immunodeficiency Virus/Acquired Immune Deficiency Syndrome

HRITF Multi Donor Trust Fund for Health Results Innovation

HSSF Health Sector Services Fund

IIFRA Independent Integrated Fiduciary Agent

IPPF Indigenous Peoples Planning Framework

ISR Implementation Status and Results Reports

JFA Joint Financing Agreement

KEMSA Kenya Medical Supplies Authority

KEPH Kenya Essential Package for Health

KMTC Kenya Medical Training College

LMIS Logistic Management Information system

MoH Ministry of Health

M+E Monitoring and Evaluation

MDGs Millennium Development Goals

NEMA National Environment Management Authority

NHIF National Hospital Insurance Fund

NHSSP National Health Sector Strategic Plan

OVC Orphan and Vulnerable Children

PDO Project Development Objective

RBF Results‐Based Financing

SWAp Sector‐Wide Approach

UHC Universal Health Coverage

UNICEF United Nations International Children's Emergency Fund

USAID United States Agency for International Development

TABLE OF CONTENTS

DATA SHEET ............................................................................................................................ 1

I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES ........................................................ 5

A. CONTEXT AT APPRAISAL ........................................................................................................... 5

B. SIGNIFICANT CHANGES DURING IMPLEMENTATION (IF APPLICABLE) ...................................... 10

II. OUTCOME ...................................................................................................................... 14

A. RELEVANCE OF PDOs .............................................................................................................. 14

B. ACHIEVEMENT OF PDOs (EFFICACY) ........................................................................................ 15

C. EFFICIENCY ............................................................................................................................. 21

D. JUSTIFICATION OF OVERALL OUTCOME RATING ..................................................................... 22

E. OTHER OUTCOMES AND IMPACTS (IF ANY) ............................................................................. 22

III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME ................................ 23

A. KEY FACTORS DURING PREPARATION..................................................................................... 23

B. KEY FACTORS DURING IMPLEMENTATION .............................................................................. 24

IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME .. 26

A. QUALITY OF MONITORING AND EVALUATION (M&E) ............................................................. 26

B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE ..................................................... 27

C. BANK PERFORMANCE ............................................................................................................. 29

D. RISK TO DEVELOPMENT OUTCOME ........................................................................................ 30

V. LESSONS AND RECOMMENDATIONS .............................................................................. 30

ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS ............................................................ 32

ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION ......................... 45

ANNEX 3. PROJECT COST BY COMPONENT............................................................................. 47

ANNEX 4. EFFICIENCY ANALYSIS ............................................................................................ 48

ANNEX 5. BORROWER, CO‐FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS ... 51

ANNEX 6. SUPPORTING DOCUMENTS (IF ANY) ...................................................................... 61

The World Bank Health Sector Support (P074091)

1

DATA SHEET

BASIC INFORMATION Product Information

Project ID Project Name

P074091 Health Sector Support

Country Financing Instrument

Kenya Investment Project Financing

Original EA Category Revised EA Category

Partial Assessment (B) Partial Assessment (B)

Organizations

Borrower Implementing Agency

Ministry of Health Ministry Of Health

Project Development Objective (PDO) Original PDO

The objectives of the Project are to support the Program and improve: (i) the delivery of essential health services in theRecipient's territory, especially for the poor; and (ii) the effectiveness of planning, financing and procurement ofpharmaceuticals and medical supplies. Revised PDO

The objectives of the Project are to improve (i) the delivery of quality essential health and nutrition services and utilizationbywomen and children especially among the poor and drought affected populations; and (ii) the effectiveness of planning, financing,and procurement of pharmaceutical and medical supplies. PDO as stated in the legal agreement

The objectives of the Project are to support the Program and improve: (i) the delivery of essential health services in the Recipient’s territory, especially for the poor; and (ii) the effectiveness of planning, financing, procurement of pharmaceuticals and medical supplies.


2

FINANCING

Original Amount (US$) Revised Amount (US$) Actual Disbursed (US$)

World Bank Financing TF‐90490

1,218,875 950,828 950,828

IDA‐47710

100,000,000 99,192,565 100,907,491

IDA‐50340

56,800,000 56,569,559 52,730,084

IDA‐53670

41,000,000 36,686,797 33,168,854

TF‐16027

20,000,000 20,000,000 20,000,000

Total 219,018,875 213,399,749 207,757,257

Non‐World Bank Financing 0 0 0

Borrower/Recipient 0 0 0

Total 0 0 0

Total Project Cost 219,018,875 213,399,749 207,757,256

KEY DATES

Approval Effectiveness MTR Review Original Closing Actual Closing

29‐Jun‐2010 07‐May‐2009 12‐Feb‐2013 31‐Mar‐2015 30‐Jun‐2018

RESTRUCTURING AND/OR ADDITIONAL FINANCING

Date(s) Amount Disbursed (US$M) Key Revisions

20‐Dec‐2011 33.06 Additional Financing

31‐Dec‐2013 113.48 Additional Financing

08‐Sep‐2016 158.94 Change in Loan Closing Date(s)

07‐Feb‐2017 181.08 Reallocation between Disbursement Categories

06‐Dec‐2017 198.37 Reallocation between Disbursement Categories

29‐Jun‐2018 208.56 Cancellation of Financing Reallocation between Disbursement Categories


3

KEY RATINGS

Outcome Bank Performance M&E Quality

Moderately Satisfactory Moderately Satisfactory Modest

RATINGS OF PROJECT PERFORMANCE IN ISRs

No. Date ISR Archived DO Rating IP Rating Actual

Disbursements (US$M)

01 22‐Feb‐2011 Satisfactory Satisfactory 8.64

02 12‐Sep‐2011 Satisfactory Satisfactory 8.92

03 15‐Apr‐2012 Satisfactory Satisfactory 33.56

04 14‐Oct‐2012 Satisfactory Satisfactory 68.26

05 22‐Apr‐2013 Satisfactory Satisfactory 83.96

06 05‐Jan‐2014 Satisfactory Satisfactory 113.90

07 30‐Jun‐2014 Satisfactory Moderately Satisfactory 126.03

08 16‐Dec‐2014 Moderately Satisfactory Moderately Unsatisfactory 130.99

09 27‐May‐2015 Moderately Satisfactory Moderately Unsatisfactory 136.84

10 01‐Dec‐2015 Moderately Satisfactory Moderately Satisfactory 145.98

11 30‐May‐2016 Moderately Satisfactory Moderately Satisfactory 159.12

12 30‐Nov‐2016 Moderately Satisfactory Moderately Satisfactory 181.50

13 23‐Jun‐2017 Moderately Satisfactory Moderately Unsatisfactory 181.50

14 21‐Dec‐2017 Moderately Satisfactory Moderately Unsatisfactory 198.79

15 30‐Jun‐2018 Moderately Satisfactory Moderately Unsatisfactory 208.99


4

SECTORS AND THEMES

Sectors

Major Sector/Sector (%)

Health 100

Public Administration ‐ Health 6

Health 94

Themes

Major Theme/ Theme (Level 2)/ Theme (Level 3) (%) Private Sector Development 10

Public Private Partnerships 10

Human Development and Gender 67

Disease Control 17

HIV/AIDS 17

Health Systems and Policies 50

Health System Strengthening 17

Reproductive and Maternal Health 16

Child Health 17

ADM STAFF

Role At Approval At ICR

Regional Vice President: Obiageli Katryn Ezekwesili Hafez M. H. Ghanem

Country Director: Johannes C.M. Zutt Carlos Felipe Jaramillo

Senior Global Practice Director: Yaw Ansu Timothy Grant Evans

Practice Manager: Eva Jarawan Magnus Lindelow

Task Team Leader(s): Michael Mills Gandham N.V. Ramana, Jane Chuma

ICR Contributing Author: Peter Okwero


5

I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES

A. CONTEXT AT APPRAISAL

Context 1. The preparation of the project spanned several years and took place in a complex environment1. The performance and track record of the health sector had been inadequate and previous attempts at reforming the sector unsuccessful. The Kenyan economy was experiencing a slowdown in growth. After peaking at 6.9 percent in 2007, economic growth contracted to 1.7 percent in 2008 before increasing slightly to 2.5 percent in 2009, which was still below the population growth rate. A combination of factors, including: (a) 2008 post‐election violence; (b) impact of the international financial crisis; and (c) reduced farm production arising from the 2009 drought were said to be responsible for the economic downturn. Persistent high levels of poverty and inequality were reported. Between 1981 and 2009, the poverty rate dropped from 48 percent to 47 percent. The Gini index2 was estimated at 0.693, with most of the poor reported living in rural areas. 2. Project preparation coincided with declines in infant and child mortality. According to the Demographic and Health Survey (DHS, 2008), Kenya recorded remarkable declines in infant and under‐five mortality rates (from 77 to 52, and from 115 to 74, per 1,000 live births, respectively) between 2003 and 2007. However, progress in reproductive, maternal and neonatal indicators, as well as, nutrition was limited. Maternal mortality ratio was estimated at 414 per 100,000 live births; six of out ten infant deaths were reported to occur in the first 28 days of life; and 35 percent of children were reported stunted with 14 percent severely wasted, in 2007. Despite the relative improvement, disparities persisted. Stunting varied from 29 percent in Nairobi to 42 percent in Eastern Province. Progress with respect to the Millennium Development Goals (MDGs) was mixed. While Kenya was poised to achieve three out of the eight MDGs: universal primary education, gender equality and women empowerment and combating major communicable diseases, the health MDGs on child and maternal health were considered unachievable. Similarly, Kenya was not likely to achieve the MDG on eradication of extreme poverty and hunger. 3. At appraisal, the health sector faced several health systems challenges. The effects of the challenges were more pronounced at the lower level facilities where the poor mainly seek care. The low level of overall public spending was compounded by inappropriate resource allocation, with relatively high proportion of spending going to regional and central hospitals coupled with irregular and untimely disbursement of funds. As a result, the lower level health facilities received insufficient operational funds for day‐to‐day operations. Inefficiency was noted in the procurement and supply chain management system for health commodities causing wastage and leading to insufficient supplies of medicines and other commodities. There was shortage of health staff, especially in the rural areas, due to poor incentives and weaknesses in deployment and management of staff. 4. In addition, the health sector previously had experienced several governance challenges. In 2006 the Bank suspended the grants component of the Decentralized Reproductive Health and HIV/AIDS Project (DARE, P066486) due to accounting and reporting deficiencies. A forensic audit and a detailed Implementation Review in

1 Project preparation started in the early 2000s and its concept note reviewed in 2005. The delay was caused by the prevailing challenges including the 2007 post‐election violence. 2 The Gini index is used as a gauge of economic inequality, to measure income or wealth distribution in a population.


6

2006 revealed evidence of corruption in the health sector. As part of the creation of the Coalition Government in 2008, the Ministry of Health was split into the Ministry of Medical Services and Ministry of Public Health and Sanitation. While government instituted steps to strengthen coordination between the two Ministries, the split further compounded the existing governance challenges in the sector. Several actions were undertaken to address the governance challenges, including: development and implementation of a new procurement policy; strengthening Audit, Finance and Facilities Management Committees; improving efficiency of internal audit; and public dissemination and monitoring of performance contracts, but the challenges persisted. At the time, it was considered that, for Kenya to improve quality of health services, it needed to increase investments in health systems strengthening for lower level services, improve regional equity and targeting of the poor, and enhance coordination with non‐state actors. 5. Health services in Kenya were provided by a wide range of actors and operated under a Sector‐Wide Approach (SWAp). The actors included government, faith‐based organizations, other not‐for‐profit organizations, for‐profit service providers and traditional healers. The overall sector framework was outlined in the National Health Sector Strategic Plan (NHSSP II, 2005‐2012), whose goal was to reduce health inequities and reverse the downward trend in the health status of the Kenyan people. The NHSSP defined the Kenya Essential Package for Health (KEPH) for all levels. The package of services by population cohort is included in Annex 1 Part D. As part of the SWAp arrangement, the government and partners signed a Code of Conduct and specified the mechanisms for the implementation of the Joint Program of Work and Funding in the Joint Financing Agreement (JFA). The project was meant to support two out of the five JFA baskets: Health Sector Support Fund (HSSF) 3 and for Essential Medicines and Medical Supplies. The other baskets were for: human resources for health, capacity and systems development, and the hospital management services fund, and as further explained in Annex 1 Part F, these other baskets did not materialize as originally envisaged.

6. The Letter of Sectoral Policy prepared in respect of the project outlined the proposed reform agenda4. The proposed reforms included: (a) implementation of the Kenya Essential Package for Health (KEPH); (b) establishment and implementation of the HSSF; (c) strengthening human resource management; (d) implementation of reforms in the proposed Healthcare Financing Policy and Strategy; (e) reforming supply chain management and strengthening governance of Kenya Medical Supplies Authority (KEMSA); and (f) implementation of the community strategy to empower communities and enhance their participation in the provision of health services. The project implemented through the SWAp modality, would focus on two reform areas: (a) development and implementation of the HSSF with the aim of increasing capacity for health services delivery and funding to the frontline service providers using existing transfer mechanisms and performance based grants; and (b) strengthening capacity for procurement and supply chain management of essential medicines and medical supplies by providing predictable financing for medicines, allocating budgets for medicines to individual facilities, establishing a pull system and creation of buffer stock. In addition, the project was to support (a) development and implementation of the Healthcare Financing Policy and Strategy giving priority to financing health care for the poor and vulnerable through subsidies; (b) innovative projects on public private partnerships

3 The HSSF (co‐financed initially by the Bank and DANIDA before DANIDA later pulled out) was an innovative scheme established by the Government of Kenya to disburse funds directly to the health facilities to enable them improve health service delivery to the communities. It had two main objectives: (i) generating and providing sufficient resources for implementing each facility’s Annual Operational Plan (AOP) to address preventive, promotive and curative services at level 1,2 and 3; and (ii) supporting capacity building in management of health facilities; supporting and empowering rural communities to take charge of improving their health; Providing grants for strengthening of the faith based health facilities through their respective secretariats; and Improving the quality of health services in the health facilities. 4 Dated May 10, 2010


7

and activities to strengthen voice and accountability within the HSSF framework; and (c) activities to strengthen the referral system.

The Rationale for Bank Involvement 7. The project was aligned with the two goals of Country Partnership Strategy (2010‐2013)5, which included: (i) improved social service delivery for vulnerable groups, particularly women, and (ii) better provision of health and sanitation services by the counties. In addition, the project was aligned with the Bank’s Health, Population and Nutrition for Africa and the Bank’s Health, Nutrition and population Global Strategy (2007): specifically, strategic direction 2 on supporting country efforts to strengthen and realize well‐organized and sustainable health systems for HNP results. All these strategies sought to strengthen health systems and to improve health outcomes, especially for the poor. 8. The project design built upon the Bank’s past involvement in the health sector. The Bank in the past supported various projects including: (a) Sexually Transmitted Infections Project (Cr. 2686); (b) Decentralized Reproductive Health and HIV/AIDS Project (Cr. 3440); Kenya HIV/AIDS Disaster Response (Cr. 3415); and Total War Against HIV/AIDS (Cr. 4336). In addition, the Bank has contributed to development of the health sector in Kenya through: (a) policy dialogue; (b) donor coordination; (c) technical assistance in a range of areas including strategic planning, financing, procurement and monitoring and evaluation; and economic and sector work. 9. The project in its design focused on the areas where the Bank had comparative advantage. At the time, the Bank had a solid track record in Kenya supporting health systems strengthening, health financing and procurement. Based on the division of labor, the Bank was delegated as the lead partner for the HSSF and procurement. Other development partners were to provide complementary support in other relevant areas by committing, for example, to provide harmonized support for human resources for health. Theory of Change (Results Chain) 10. The project at appraisal did not prepare a comprehensive Theory of Change. However, a conceptual framework for the HSSF and a results chain were developed to explain the causal pathways of the project’s contribution towards strengthening delivery of health services at the primary level. Building on the two conceptual frameworks and the Theory of Change (Annex 1 Part E) prepared by the project team during project implementation, the author prepared the Theory of Change in Figure 1. The Theory of Change is premised on rolling out HSSF and providing grants (HSSF and Results‐Based Financing, RBF) to front line service providers, building institutional capacity for improved health service delivery at the sub national level, supporting implementation of central level governance and stewardship initiatives, and strengthening supply chain management to ensure delivery of the package of essential health services. A key assumption was that other partners operating under a SWAP arrangement would support other complementary reforms; for example, the areas of human resources for health reforms and the implementation of the community strategy.

5 Report Number 52521‐KE


8

Figure 1. Theory of Change

Strategies/Activities

Outputs Outcomes Long‐Term Outcomes

Rollout HSSF countrywide to the primary HFs. Key activities would involve: (i) HSSF grants; (ii) Introduction and rollout of RBF; (iii) Building capacity of HFs to plan and implement AOPs; (iv) Strengthening Counties to support HFs; (ii) Promotion of social accountability for health service delivery; and (iii) Piloting innovative Public Private Partnership for Health (PPPH) approaches

Number of HFs meeting the required standards for the level (HFMCs, EMMS, staffing, equipment, financing etc.) Number of HFs delivering the package of essential health services for the level Increased share of reported complaints acted upon Number of PPPH initiatives piloted

Improved access to essential health services Increased utilization of the essential health services especially by the poor Increased coverage of the population to essential health services Client Satisfaction

Reduction in Morbidity and Mortality Reduced Burden of diseases

Support implementation of Health Sector Governance and Stewardship initiatives: SWAP Secretariat; Pilot approaches to reach the poor; Management and performance of district hospitals; Referral system; and Health care waste management

Increased number of poor households enrolled for health insurance subsidies HFs meeting health care waste management standards Increased number of cases referred and treated at the hospitals

Increased utilization of the essential health services by the poor households Reduced OOP among the poor

Implementation of initiatives to strengthen the Supply Chain Management for Essential Health Commodities through (i) increased funding for essential commodities and (ii) supporting reforms in KEMSA

Improved availability of essential health commodities at the HFs Improved capacity at KEMSA to procure and distribute essential commodities in a timely fashion

Reduced stockouts of essential health commodities at KEMSA and the HFs

Project Development Objectives (PDOs) 11. The original objectives of the Project are to support the Program and improve: (PDO 1) the delivery of essential health services in the Recipient’s territory, especially for the poor; and (PDO 2) the effectiveness of planning, financing, procurement of pharmaceuticals and medical supplies6.

6 The PDO is taken from the Financing Agreement (dated July 5, 2010) as the PDO statement in the PAD excluded mention of the Program. The PDO in the PAD was stated as follows: “The Project development objectives are to improve: (i) the delivery of essential


9

Key Expected Outcomes and Outcome Indicators 12. Six project outcome indicators and fourteen intermediate outcome indicators were identified to monitor project implementation. The following indicators below were the project outcome indicators:

a. Direct project beneficiaries (number) of which female (percent) b. Health facilities that experience essential drugs stock‐outs for more than two weeks (percent) c. Children immunized (number, disaggregated for Northeast Province) d. HSSF facilities meeting core financial management requirements of the fund (percent) e. Timely payment to suppliers by KEMSA based on agreed procurement contract f. Facilities displaying quarterly information on funds received and availability of drugs at facility‐level (levels 2

and 3) (percent)

Components 13. The project comprised two components. 14. Component 1: Effective and transparent implementation of the KEPH through HSSF grants and performance strengthening (total estimated cost US46 million). This component was aimed at providing direct cash transfers through the HSSF to primary health care facilities (Levels 1, 2 and 3)7 and strengthening governance and stewardship functions to enhance delivery of essential health services. Levels 1, 2 and 3 of the health system were the focus of the HSSF. In addition, the HSSF was an important initiative to empower communities to take charge of improving their health by providing direct cash transfers to Health Facility Management Committees (HFMCs) through enhanced transfers and performance‐based incentives. 15. Sub Component 1.1: Health Sector Services Fund (US$34 million): The project under the subcomponent was to support effective implementation of the KEPH, through HSSF grants targeted at health service delivery levels (Levels 1, 2 and 3) and capacity building in the management of health facilities. The recipients could use the resources for: (a) employment of casual workers, operational costs including facility maintenance; (b) outreach activities; (c) strengthening management and coordination of health services delivery; and (d) supporting activities to strengthen voice and accountability. It was envisaged that innovative activities to improve private public partnerships would be piloted under the subcomponent. 16. Sub Component 1.2: Health Sector Governance and Stewardship (US$12 million): The project was to support selected activities to strengthen governance and stewardship functions of the two ministries for enhanced service delivery. Specific activities were to support: (a) implementation of health financing reforms through selective pilots, capacity building and evaluations with focus on developing and piloting approaches for reaching the poor with health insurance cover; (b) KEPH referral system through capacity building of district hospitals and establishment of five model referral sites; (c) health care waste management; and (d) the SWAp Secretariat.

health services for Kenyans, especially the poor; and (ii) the effectiveness of planning, financing and procurement of pharmaceuticals and medical supplies”. 7 Level 1 is the community for which there is no HF. Level 2 HFs are dispensaries. They are at the lowest level of the public health system and the first point of contact with patients. They provide outpatient services for simple ailments and are staffed by enrolled nurses, public health technicians, and dressers. Level 3 are health Centers. They are staffed with clinical officers as the team leaders and provide a range of services, such as basic curative and preventive services for adults and children, as well as reproductive health services. They also provide minor surgical services such as incision and drainage.


10

17. Component 2: Availability of essential health commodities and supply chain management reform (US$54 million). The project was to provide financing towards essential medicines and support reforms to strengthen supply chain management for essential medicines. This was in light of the fact that the strengthening of levels 1, 2 and 3 above would not have been sufficient without parallel investments in essential commodities to ensure that the quality of services is improved. 18. Sub Component 2.1: Commodities (US$48 million): This component was aimed at improving availability of commodities at the lower levels of the public health system through: (a) financing the budget for commodities and building up a buffer stock of essential medicines; (b) reforming the procurement and supply chain system to become more efficient and effective; and (c) establishment of a commercial window to supply commodities to facilities on a cost recovery basis. 19. Sub Component 2.2: Strengthening governance and institutional reforms to enhance the efficiency and transparency of KEMSA (US$6 million). The project was to support capacity building for KEMSA and the health ministries in the key functions of forecasting, planning, procuring and managing critical commodities with a focus on reforms in the flow of funds to KEMSA, definition of the EMMS list, billing system of the KEMSA, development of supplementary commodity services for hospitals, and to address the key constraints affecting the availability of essential medicines in the public health facilities.

20. Independent Integrated Fiduciary Review Agency (IIFRA). To ensure transparency and effectiveness in carrying out the implementation of the project components, in particular with regard to the HSSF and the procurement through KEMSA, an IIFRA was to be recruited to undertake: (i) compliance verification, (ii) procurement monitoring; (iii) performance auditing of the project; and (iv) continuous financial audits of the project with the objective of ensuring that any systemic governance problems and accountability issues are noted early and corrective measures taken.

B. SIGNIFICANT CHANGES DURING IMPLEMENTATION (IF APPLICABLE)

Revised PDOs and Outcome Targets 21. The PDOs were revised twice during the project lifespan as part of Additional Financing (AF) in November 2011 and December 2013, respectively8. The project in 2011 obtained AF to support the government’s humanitarian response to the needs of the drought affected populations and scale up project interventions. More than 3.7 million people in arid and semi‐arid regions of Kenya were directly affected by the drought with around 385,000 children and 90,000 pregnant women estimated to be suffering from acute malnutrition in the affected areas. The 2011 revision changed the PDOs to ‐ “support the Program and improve: (i) the delivery of essential health services in the Recipient’s territory, especially for the poor and the drought affected populations; and (ii) the effectiveness of planning, financing, procurement of pharmaceuticals and medical supplies”. The revision of PDO 1 in 2011 was intended to reflect the emergency response to deliver essential health and nutrition services to drought affected populations in arid and semi‐arid regions of Kenya. 22. The second revision in 2013 modified the PDOs to ‐ “improve (i) the delivery of quality essential health and nutrition services and utilization by women and children among the poor and drought affected populations; and (ii)

8 The first AF (U$56.8 million) was financed through the IDA Crisis Response Window; while the second AF was financed through an IDA Credit (USS$41 million) and the Multi Donor Trust Fund for Health Results Innovation (HRITF, US$20 million).


11

the effectiveness of planning, financing, procurement of pharmaceuticals and medical supplies.” – The revision was meant to enhance the project focus on improving utilization of health and nutrition services by women and children. The results framework was revised to reflect changes to the project ‐ adding new indicators, dropping others and revising targets9. In addition to specifying the beneficiary population groups, a key change in the PDOs was the addition of the quality dimension to PDO 1. The full list of project indicators is included under Annex 1 C.

Revised PDO Indicators 23. The following were the revised project outcome indicators in 2011:

a. Direct Project Beneficiaries (Number of outpatient visits to levels 2 and 3 disaggregated by females and drought affected areas, new)

b. Children fully immunized (number, disaggregated for Northeast Province) c. Value fill rate for commodities on KEMSA Essential Drug List (Percent, revised) d. HSSF facilities meeting the core financial management requirements of the fund (Percent) e. Fiduciary irregularities reported from HSSF and KEMSA acted upon during the previous quarter (Percent) f. Level 2 and 3 HSSF facilities displaying information as per Government of Kenya (GoK) guidelines (Percent) g. Severely malnourished children under five receiving treatment recovered (Percent, new)

24. The revised project outcome indicators in 2013 were as follows:

a. Direct project beneficiaries (Number of outpatient visits to levels 2 and 3) (i) Female Beneficiaries (ii) Beneficiaries from drought affected areas (Number)

b. Children immunized (number, disaggregated for Northeast Province) c. Value fill rate of orders for commodities for Primary Health Care facilities received by KEMSA either from

counties or facilities (Percent) d. Severely malnourished children under five receiving treatment recovered (Percent) e. Share of planned procurement from Essential Medicines and Medical Supplies (EMMS) pool completed by

KEMSA in the financial year as per GoK guidelines (Percent, new) f. Women receiving free delivery services meeting the quality norms at public health facilities (Number, new) g. Poor households receiving health insurance subsidies (Number, new)

Revised Components 25. Component 1 of the project was revised during the 2011 AF. The AF was to support the emergency response in the drought affected arid and semiarid regions of the country. In 2011, a new activity was added under Component 1 (US$12.8 million) to deliver essential health and nutrition services to populations affected by drought, focusing on the supply of nutrition commodities to manage malnutrition among children under five years, pregnant and lactating women, and Component 2 (US$44 million) scaled up to supply essential health commodities for drought affected areas and strengthened to deepen the on‐going reforms to move towards the “pull” system of supply chain management. The nutrition Interventions focused on infant feeding, micronutrient supplementation and management of acute malnutrition.

9 In 2011, one new PDO indicator was added. Meanwhile, for intermediate outcome indicators, two new ones were added and seven dropped. In 2013, three PDO indicators were added and three were dropped.


12

26. Component 1 was again revised during the 2013 AF. The AF was to finance the cost of the scale‐up activities and restructuring of the project to respond to devolution10 process, scale‐up RBF to further enhance the impact of the project and join government’s efforts towards achieving universal health coverage specifically targeting the poor. The changes included: (a) scaling‐up support to the HSSF (IDA, US$11 million); (b) scaling‐up of RBF (Multi Donor Trust Fund for Health Results Innovation, HRITF, US$14 million) to 21 counties in arid and semiarid regions; (c) supporting implementation of the government policy on free maternity services11 and phase one of the Health Insurance Subsidies for the Poor (HISP, US$20 million – IDA US$14 million and HIRTF US$6 million)12; and (d) capacity building of counties to plan, budget and monitor delivery of health services within the new devolution arrangement, specifically targeting county health teams, health facility in‐charges and members of the HFMCs (IDA, US$16 million). A comprehensive impact evaluation to assess the RBF and health insurance subsidies was envisaged to inform government decision making on future scale‐up. Other Changes

27. The project underwent restructuring as further explained below to better reflect the project’s focus on women and children, ensure the project’s implementation arrangements are aligned with the ongoing devolution reforms, and allow adequate time to complete implementation of project activities.

a) Extension of the Project Closing Date. The project closing date was extended twice. The first extension during the AF in 2013 extended the closing date to December 31, 2016 to facilitate implementation of RBF and HISP and completion of impact evaluation to inform future scale‐up of RBF and health insurance subsidies for the poor. The second extension during a Level 2 restructuring (October 2016) extended the closing date to June 30, 2018 to allow for full implementation of remaining project activities: (a) RBF scale‐up; (b) scale‐up of Phase 1 of the HISP; and (c) training 400 enrolled community nurses13; and to enable the project to fully disburse the remaining project proceeds (IDA Credit of US$ 45. 6 million and US$ 10 million grant).

a) Reallocation between Disbursement Categories. The second Level 2 restructuring (December 2017) reallocated unspent funds for RBF from the IDA Credit (about US$44 million) to finance the procurement of essential medical equipment to fill the critical gaps in the delivery of primary health services and nutritional commodities to respond to the prolonged drought in the country.

b) Revision of the Implementation Arrangements. In 2011, United Nations International Children's Emergency Fund (UNICEF) was chosen as the specialized procurement agency to be contracted to supply

10 Kenya promulgated a new constitution in 2010, which ushered devolution. Under the devolution framework newly created Counties assumed overall responsibility for the coordination and delivery of health services, while the national government remained with the responsibility for stewardship, health policy, technical assistance to counties, and management of national referral health facilities. The Counties took over ownership of the health facilities and became responsible for staff recruitment and deployment, and procurement of equipment as well as medical supplies. Besides KEMSA, Counties could procure drugs from any other source. With devolution, funds to Counties were transferred as conditional grants through the respective County Revenue Fund (CRF) Accounts before being channeled to a County Special Purpose Account for Health for onward disbursement to the beneficiary facilities. 11 The Government of Kenya (GoK) abolished user fees for delivery services in all public hospitals (June 1, 2013) through a presidential directive with the aim of promoting health facility delivery and reducing pregnancy related mortality in the country. The project however did not finance free maternity services. The activity was financed by government. 12 HISP was intended to provide comprehensive health insurance cover for outpatient and inpatient services to five beneficiaries from each enrolled household. 13 The students enrolled to the Kenya Enrolled Community Health Nursing Program run by the Kenya Medical Training College (KMTC) were selected from poor households from the arid and semiarid counties and were expected to return to serve their communities upon completion of the course.


13

the nutrition commodities. In 2013, the overall responsibility for the project implementation was placed under the newly created Directorate for Policy, Planning and Healthcare Financing in the Ministry of Health after the merger of the previous two ministries created by the coalition government. With the advent of the devolution policy, the county governments became responsible for delivering essential health services including overseeing project related activities in the counties. The government budget for health services including essential health commodities was devolved to the county governments. The project implementation arrangements including the financial management arrangements and procurements were updated to reflect the new mandate of the counties to approve workplans, verify performance and report results as well as statement of expenditures14. At the same time, Kenya School of Government was identified to lead capacity building of the Counties in financial management; and African Medical Research and Education Foundation (AMREF) to train HFMCs on their new roles.

c) Revision of the Results Framework. The results framework underwent major revisions during the AF to enhance the project focus on improving the delivery and utilization of health services by women and children and to reflect changes to the project (nutrition and devolution). Some indicators were revised/dropped specifically for lack of relevance in view of the devolution process. Four indicators were dropped, and two new ones added in 2011; meanwhile, five indicators were dropped, and three new ones added in 2013. In the end the project remained with seven PDO outcome indicators, five under PDO 1 and two under PDO 2.

Rationale for Changes and Their Implication on the Original Theory of Change 28. The changes were intended to: (a) scale‐up ongoing activities to deepen the impact of the project; (b) support the government to respond to the health needs of the population severely stricken by drought; and (c) ensure the project is compliant and supportive of the devolution program adopted by the GoK. Considering the implied original Theory of Change, the changes added the quality dimension to the PDO, added nutrition as a core project activity, better clarified the poor and vulnerable populations targeted by the project, and deepened implementation impact of the project in terms of utilization. While changes to the results matrix were meant to strengthen the results chain, the dropping of some indicators weakened the causal pathways of the project’s interventions to attain its objectives. The indicator, “health facilities experiencing drug stockouts” was replaced with “value fill rate of commodities on KEMSA Essential Drug list,” which was considered to better capture the new accountability relationship between KEMSA and the health facilities under the pull system ‐ the main reform agenda said to be supported by the project. The indicator, “health facilities meeting staffing norms” was dropped because Counties were taking over the responsibility for staffing at levels 2 and 3 facilities. Finally, devolution made County Local Government significant stakeholders in the delivery of health services, which was unanticipated in the original implied Theory of Change.

14 The devolution process in Kenya was very ambitious and fast‐tracked devolution of responsibilities to the newly created counties in an environment where the building blocks for devolution were still evolving, including intergovernmental structures and mechanisms for intergovernmental cooperation and transfer of resources to deliver on policy priorities. This resulted in the project mainly playing a catchup or firefighting role.


14

II. OUTCOME

A. RELEVANCE OF PDOs

Assessment of Relevance of PDOs and Rating 29. The relevance of the PDOs is High. The PDOs are aligned with the current national development plans: National Health Policy (2012‐2030), Health Sector and Strategic Plan (2013‐2017), and Vision 2030. The PDOs are aligned with the Social Pillar of Vision 2030 which prioritizes improving the livelihoods of Kenyans by providing efficient, integrated, high quality and affordable health care through a devolved system of governance, and relevant for the current initiatives on Universal Health Coverage (UHC) premised on provision of quality health services and financial risk protection to all Kenyan citizens. The PDOs are also aligned with two out of three domains of engagement in the Country Partnership Framework15 (CPF): human resource development for shared prosperity and delivering a devolution dividend, and specifically two goals of the CPF: improved social service delivery for vulnerable groups, particularly women, and better provision of health and sanitation services by the counties. 30. The PDOs focus on priority interventions. The PDO 1 objectives of improving delivery of quality essential health and nutrition services and utilization of essential health and nutrition services by women and children among the poor and drought affected populations are highly relevant. Women and children represent a large share of the population and at the same time bear a disproportionate burden of disease. The semi‐arid and arid regions of Kenya periodically experience episodes of severe drought and the drought affected population need to be provided with the necessary food supplementation. Kenya, like other countries is faced with the challenge of delivering quality health services commensurate with the needs of the population, and as it embarks on implementing the various health initiatives for the attainment of UHC, it will need to systematically address the quality gaps in health service delivery. Similarly, the PDO 2 objective of improving effectiveness of planning, financing and procurement of essential health commodities is relevant. While capacity of KEMSA was enhanced, insufficient budgetary allocation for drugs still poses a major challenge for ensuring sustainable financing of essential medicines. Furthermore, with the transfer of responsibility for delivery of essential health services to the counties, the need for continued capacity building for health services delivery, and especially supply chain management of health commodities remain of paramount importance. It is necessary that the capacity of the county governments is continually enhanced for them to perform their new roles and mandates within the new devolution dispensation.

15 Kenya Country Partnership Strategy 2014‐2018, 88940 v2


15

B. ACHIEVEMENT OF PDOs (EFFICACY)

Assessment of Achievement of Each Objective/Outcome 31. Rationale for assessment. The PDO includes two main parts (PDO1 and PDO2), each of which are broken down further into separate parts which this ICR analyzes individually, per ICR Guidelines. PDO 1 has two parts: (a) improve the delivery of quality essential health and nutrition services and (b) improve utilization of essential health and nutrition services by women and children among the poor and drought affected populations. PDO 2 on the other hand has three parts ‐ improve effectiveness of: (a) planning, (b) financing and (c) procurement of pharmaceuticals and medical supplies. Each of these five PDO parts are assessed against the PDO outcome indicators, including those dropped or replaced in 2013, given over 70 percent of credit proceeds were disbursed by November 201316. However, as all targets were revised upwards and the scope of the PDOs expanded, this ICR does not use a split rating methodology. The assessment also reviewed the extent of execution of the core reforms supported by the project and the main elements implied in the PDOs and compared utilization of core services in RBF counties against the rest of the country. 32. Overall achievement of project indicators. The project RF includes 13 indicators, of which seven are outcome indicators and six are Intermediate Outcome (IO) Indicators. The project achieved 11 out of the 13 indicators (Figure 2). Four of the five PDO 1 outcome indicators and the two PDO 2 outcome indicators were achieved. The status of achievement of project indicators is captured in the updated table in the results framework, Annex 1 C.

Figure 2. Overall Project Achievement

Rating PDO Indicators

Intermediate Outcome Indicators

Total PDO/IO Indicators achieved

Total

PDO 1 PDO 2 PDO 1 PDO 2 PDO 1 PDO 2

Achieved 4 2 4 1 8 3 11

Not Achieved 1 0 1 0 2 0 2

Total 5 2 5 1 10 3 13

33. Assessment of Indicators dropped at 2013 AF. Three PDO indicators and five intermediate outcome indicators dropped during the 2013 AF were assessed against the original targets. Most of the dropped indicators were under PDO 2. All three PDO outcome indicators had been achieved: two under PDO 1 and one under PDO 2. Regarding the Intermediate Outcome Indicators, the annual targets for two indicators were achieved against the 2013 targets. Assessment of achievement of PDO 1, Part One: improve the delivery of quality essential health and nutrition services – Rating: Substantial 34. The project thrust was on delivering quality health services based on the KEPH. The package comprised cost‐effective interventions that address the major causes of disease burden in Kenya. As further explained below, in addition to improving direct financing to front‐line health facilities (Levels 2 and 3) through the RBF and HSSF to procure necessary inputs in a timely manner, the project improved the supply of the essential medical and health supplies. The project achieved the two PDO indicators on quality (see Figure 3 below). The delivery of free

16 The indicators dropped or replaced in 2011 were excluded from assessment


16

maternity services was based on standard norms promoting delivery of quality services. In total, over 600,000 women delivered under the free maternity program compared with the original target of 550,000. The women received a benefit package which included both outpatient and inpatient services for the mother and newborn for a period of one year as well as antenatal care, delivery, postnatal care and emergency referrals for pregnancy related conditions and complications. This was one of the flagship programs instituted by government in its quest to achieve universal health coverage. Improvement in the care of malnourished children was also recorded. According to the nutrition surveillance and reporting system, over 80 percent of malnourished children treated recovered compared to a target of 60 percent. The number of admissions of children suffering from severe acute malnutrition was also reported to have declined to around 3000 per month in 2013 compared to over 5000 per month in 2011. In addition, over 19,000 health personnel were trained in various disciplines over the project duration. 35. Based on achievement of the two PDO indicators, improvement in the delivery of quality essential health and nutrition services is rated Substantial. Assessment of achievement of PDO 1, Part Two: improve utilization of essential health and nutrition services by women and children among the poor and drought affected populations – Rating: Modest

36. Increased utilization of essential health and nutrition services (Figure 3). The utilization of health and nutrition services increased as demonstrated by the increase in direct project beneficiaries based on outpatient visits. Women and the drought affected population registered a significant increase in utilization of outpatient services which more than doubled the baseline figures. Although not directly financed by the project, a significant increase was also registered in the utilization of free delivery services offered as part of the free maternity program as explained above. However, immunization rate remained persistently low. The annual immunization rate remained low throughout the project duration, and the end project rate achieved was lower than the original baseline. This is confirmed by the GAVI Country Fact Sheet on Kenya (2018), which shows DPT 3 coverage had been slowly sliding downwards since 2012/13 and in 2017 was estimated at 71 percent based on official country estimate. The same report noted that only 15 out of 47 counties achieved a DPT3 coverage of 80 percent or more. While immunization data was meant to be disaggregated for the North East region, the information was not tracked and was therefore unavailable.

Figure 3. Achievement of PDO 1 Indicators

PDO 1 Indicators

Baseline Target Actual Status

Part 1. Delivery of quality essential health and nutrition services

Severely malnourished children under five receiving treatment recovered ‐ New 2011

0% 60% 84% Achieved

Women receiving free delivery services meeting quality norms at public facilities – New 2013

453,857 550,000 604,521 Achieved

Part 2. Utilization of essential health and nutrition services by women and children among the poor and drought affected populations

Direct project beneficiaries Number Female (55%) Number from drought affected areas

25,896,000 14,299,363 0

35,000,000 17,500,100 5,500,000

50,949,459 34,447,148 15,088,032

Achieved


17

Children immunized – National From North East

1,188,698 32,800

1,298,058 65,000

1,105,234 NA

Not Achieved

Poor households receiving health insurance subsidies – New 2013

0 30,000 181,315 Achieved

37. The increase in utilization at the facilities was however not registered at the population level. According to the preliminary findings of the 2018 Household Health Expenditure and Utilization Survey17, there was no change in the total number of outpatient visits made in the four weeks preceding the survey between 2013 and 2018, which remained about 9.1 million visits. Annual per capita utilization dropped from 3.1 in 2013 to 2.5 visits in 2018 with the rate for the lowest wealth quintile dropping from 2.7 to 2.2 visits18. The same scenario was reported in the RBF counties. Out of the 18 RBF counties with data for the 2013 survey, 10 registered a drop in per capita utilization rate, 5 an increase and 3 witnessed no change in the per capita utilization rates between 2013 and 2018. While the project used national outpatient data to estimate number of project beneficiaries, and assuming that the preliminary survey findings are valid, it can be deduced that the sum total of the initiatives undertaken during the period were not able to positively impact on the per capita utilization rate. 38. Emergency Nutrition Support to Drought Affected Areas. While Kenya remains prone to drought and its effects, the project contributed towards the government’s humanitarian nutrition response in the drought affected areas. Over 15 million people (moderately malnourished children under five and pregnant and lactating women) in the drought affected areas received health care services and the relevant food supplements. Because of the persistent household food insecurity in the arid and semiarid regions of Kenya, severe acute malnutrition remains a challenge. In 2017 admissions for children with severe acute malnutrition spiked from 3,000 in the previous years to about 6,000 cases per month in early part of the year. The total number of children 6 to 59 months and pregnant and lactating women requiring treatment for acute malnutrition in the arid and semiarid counties were estimated at 420,674 and 39,068, respectively19. In light of this, the unutilized RBF funds as explained below were reallocated to procure additional food supplements. 39. Health Insurance Subsidy for the Poor Program (HISP). The project, through the National Hospital Insurance Fund (NHIF), successfully supported the HISP20, the government’s flagship program under Vision 2030 to provide the poor and vulnerable households with health insurance cover reaching 181,315 of the 181,968 targeted households21, representing about 50 percent of orphan and vulnerable children (OVC) households countrywide. This is consistent with the increase in health insurance coverage, which increased from 17.1 percent to 19.9 percent between 2013 and 2018, representing coverage of 9.5 million Kenyans out of the estimated 47.8 million population. An impact evaluation of the HISP concluded that despite challenges encountered in its implementation, health insurance subsidies for the poor increased utilization of health services22, especially services provided by the private

17 These was one of main studies financed under the project in part to assess its impact on the delivery health services 18 Ministry of Health, Draft Report, Kenya Household Health Expenditure and Utilization Survey, 2018 19 Ministry of Health. Nutrition Situation Report for Arid and Semiarid Areas, August 2017 20 The Health in Africa Initiative (HIA) Kenya program in collaboration with the project provided the technical support towards the design of HISP both the pre‐test and scale up, as well as implementation through community mobilization and sensitization under the Africa Health Markets for Equity program. In addition, HIA conducted three rounds of process evaluation during the lifetime of HISP. 21 The target households were beneficiaries of the orphan and vulnerable children cash transfer program. 22 The average number of hospitalizations per household per annum increased in the intervention group (from 1.4 to 1.6), while declining in the control group (from 1.6 to 1.3). In addition, there was an overall reduction in outpatient OOP of Kshs 3488.81 and a similar decrease in inpatient expenditure of Kshs 4828.99, which is estimated to have resulted in an annual decrease in mean direct medical costs of Kshs 6,320.08. In addition, at end‐line, levels of spending declined among intervention households to 7.1%, and increased among control households (14.4%). The HISP intervention was thus estimated to be responsible for a reduction by 6.8 absolute percentage points in the share of household expenditure spent on treatment.


18

sector, and reduced out of pocket health expenditure as a share of the overall household expenditure23. The incidence of catastrophic health expenditure was estimated at 4.9 percent (using 40 percent threshold) and 8 percent (using 10 percent threshold). This was a significant improvement from 2013 rates of 6.21 percent and 12.7 percent, respectively. Furthermore, the project used a combination of approaches to target the poor and vulnerable to enhance equitable access to health and nutrition services. The HSSF grants whose allocations were based on equity targeted the primary health facilities predominantly used by the poor. Both the RBF program and nutrition humanitarian support targeted the arid and semi‐arid areas, where most of the poor reside. 40. Implementation of the HSSF reform. The project facilitated reforms to directly transfer funds to primary health facilities by rolling out HSSF grants to all primary health facilities and strengthening community engagement in health service delivery. Project support was instrumental in increasing the operational budgets for lower level health facilities, capacity building for health service delivery at the county level and establishment of Health Facility Management Committees (HFMCs). According to the Independent Integrated Fiduciary Agent (IIFRA) report, irregular use of grants by health facilities reduced from 21 percent to 5 percent and the health facilities maintaining proper cash books increased from 53 percent to 78 percent between 2011/12 and 2017/18, respectively. IIFRA at midterm confirmed that overall quality of service, waiting time, cleanliness and the state of health centers had also improved. Another independent evaluation24 revealed that direct cash transfers to peripheral health facilities led to improved quality of care, staff motivation and patient satisfaction, even though such funds represented less than 1 percent of the total health sector budget, and increased involvement of community members in the running of facilities as part of HFMCs, which strengthened accountability. However, with devolution, the original fund flow arrangements under the HSSF to the health facilities were severely disrupted and curtailed implementation of the essential activities25 previously financed through the HSSF grants. The capacity created through project support, as explained below, continues to partly benefit the devolution program in the health sector; however, because of the severe disruption of the fund flow arrangements to primary health facilities the original principles – especially timely transfers of funds to the HFs ‐ that underpinned the HSSF were no longer tenable under the new arrangements through the counties. 41. Results Based Financing. The project facilitated the introduction of performance‐based financing approaches in the health sector by piloting and rolling out RBF to 21 arid and semi‐arid counties. Because of devolution the program experienced considerable challenges with funds flow and reporting requirements, which constrained implementation of the RBF activities and undermined the program’s potential contribution to improving performance of the health sector. RBF activities started late in 2015/16 and only two transfers were made to the counties in 2015/16 and 2016/1726. Absorption of RBF funds by the facilities remained low and many counties did not procure equipment. Slow implementation of RBF was attributed to funds flow challenges to counties, delays in internal clearances in MoH and protracted health worker’s strikes and national elections. The individual counties experienced diverse challenges and their overall performance was mixed. In the end, the project was restructured to reallocate the savings from the Credit (about US$ 44 million) due to slow implementation of RBF activities to procure essential equipment and nutrition commodities. In addition, the US$ 10 million transferred to counties that remained unspent/unaccounted and of which half remained in the County Revenue Fund were absorbed through re‐costing which involved the upward revision of the RBF indicator payment

23 Ministry of Health, A Report of the Impact Evaluation of the Health Insurance Subsidy Program Pilot in Kenya, November 2017 24 Carried out by the Kenya Medical Research Institute Welcome Trust (2014). 25 Basic repairs and maintenance, installation of water tanks, printing patient treatment log cards, and payment for utility bills and casual staff. 26 Shs 508 million in 2015/16 and Shs 1,167 million in 2016/17, equivalent to US$16.75 million. The second release include funds for equipment.


19

rates. Despite these efforts, about US$ 4 million of the project proceeds had to be cancelled after reconciliation of project expenditures. 42. Capacity Building. In support of the HSSF under component 1, the project included a set of activities to strengthen the governance and stewardship functions of the Ministry of Health at the national level and health teams at the county level to enhance service delivery. The activities included strengthening the Directorate of Policy Planning and Healthcare Financing and County Departments of Health including creation of strategic planning and management units. The project supported capacity building of county health teams through provision of technical assistance before and after devolution: (a) training County teams in financial management and (b) training health facility management committees. In addition, DANIDA in parallel supported the training of HSSF Accountants who were deployed to the Counties. In addition, the project supported the scale up of training midwives under the Kenya Enrolled Community Health Nursing Program. A total of 400 students from poor counties sponsored by the project were in their final year into the course during the ICR mission and upon completion of the studies would return to serve in their respective counties27. Another 800 students sponsored through other financing arrangements are currently enrolled in year 1 and 2 of the course and are also expected to return to serve in their communities. On the other hand, innovative projects to improve public private partnership and activities on strengthening the referral system were not carried out.

Assessment of achievement of PDO 2, Part One: Effectiveness of planning for pharmaceuticals and medical supplies – Rating: Modest

43. Effectiveness of Planning. While there were no specific indicators to monitor effectiveness of planning, the project contributed to reforming the supply chain management system for medicines and medical supplies and making KEMSA become autonomous and more effective and efficient as well as transparent in its operations. With the reforms, all the previous vertical supply chains were integrated into one system under KEMSA. The project among others supported establishment of an electronic record keeping system, and development of an electronic Logistic Management Information system (LMIS), which has improved KEMSA’s ability to monitor and manage the supply chain system for medicines in Kenya. KEMSA is now the medical logistics provider with the widest national coverage of 4,100 health facilities in the country. By 2013, 95 percent of health facilities were able to place orders under pull system the previous quarter28. These reforms were largely focused on KEMSA and did not involve the counties, yet counties with devolution became responsible for overall planning and management of the supply chain system under their jurisdiction. While overall management capacity improved at KEMSA, this was not the case with the counties. Furthermore, the advent of devolution adversely affected the capacity of the counties in areas of quantification, forecasting, execution of orders and ensuring timely payments for commodities. Without capacity for planning and forecasting drug needs at the counties, KEMSA runs a risk of eroding its current level of buffer stock especially if counties fail to make timely payments for orders. Besides, KEMSA still needs to strengthen its administrative processes. Assessment of achievement of PDO 2, Part Two: Effectiveness of financing for pharmaceuticals and medical supplies – Rating: Substantial

27 The training of enrolled community health nurses over a period of three (3) years was conceived in response to the severe shortage of frontline health workers, especially nurses in the arid and semiarid counties. The students are selected from marginalized communities, minority groups and orphans and upon completion of the studies return to their communities. 28 This indicator was dropped in 2013


20

44. Effectiveness of Financing. The project, among others, supported KEMSA’s transformation from an agency that was previously dependent on government resources to an adequately resourced independent procurement agency able to respond to the needs of 47 counties, the national government and other key stakeholders including bilateral and multilateral partners in the country. The timely reimbursement of KEMSA by the MoH also allowed KEMSA to operate a drug revolving arrangement29. KEMSA created a buffer stock and reformed the supply chain management system from a “push” to a “pull” system. Using the new structure, KEMSA now sells medicines and medical supplies to county governments and faith‐based institutions using a not‐for‐profit revolving fund system30. KEMSA is also able to procure commodities for key donor support projects including United States Agency for International Development (USAID), Global Fund to Fight AIDS, Tuberculosis and Malaria (GFATM) and Global Alliance for Vaccines and Immunization (GAVI). In 2015, the US government signed a five‐year US$ 650M contract with KEMSA to procure commodities for its programs. According to the UNICEF publication, KEMSA managed to procure and distribute essential medicines and health supplies valued at about US$900 million over a seven‐year period.

Figure 4. Achievement of PDO 2, Part Three Indicators – On Procurement

PDO 2 Indicators Baseline Target Actual Status

Share of planned procurements from EMMS pool completed by KEMSA in the FY per GoK guidelines – New 2013

0 60% 100% Achieved

Value fill rate of orders for commodities for primary health care received by KEMSA either from counties or facilities – Revised 2013

73% 80% (2013)

83% (2018) Achieved

Assessment of achievement of PDO 2, Part Three: Effectiveness of procurement of pharmaceuticals and medical supplies – Rating: Substantial 45. Effectiveness of Procurement (Figure 4). The project achieved the targets for the two PDO 2 indicators assessed under the PDO, both of which were related to effectiveness of procurement (Figure 5). KEMSA attained 100 percent completion rate of the planned procurements, against the end target of 60 percent. Over the period, KEMSA improved on the handling of orders from the HFs, and by the end of the project, had achieved a value fill rate of 83 percent against a target of 80 percent. The PDO 2 indicator on health facilities that experience essential drugs stock‐outs for more than two weeks was dropped in 2011; however, it would have provided a better picture of the performance of the entire drug supply chain management system31. The indicator was replaced with the indicator “value fill rate for commodities for primary health care received by KEMSA from either counties or facilities” to ensure that the project would be better aligned with the devolved health system where counties would have resources and flexibility to place orders. According to the IIFRA report, after devolution many counties failed to order and pay for essential medicines from KEMSA on time, which increased stock‐outs of commodities at the health facilities in 2013/14 and 2014/15. The shortages were confirmed by the Kenya Household Health

29 The indicator Timely reimbursement to KEMSA documented claims by MoH was dropped in 2013 30 UNICEF Publication. KEMSA after the devolution: Financial sustainability through a supply chain strategy with a strong focus on human resources and leadership 31 The following three PDO 2 indicators dropped in 2011 were not assessed: health facilities that experience essential drugs stock‐outs for more than two weeks, timely payment to suppliers by KEMSA based on agreed procurement contract, and facilities displaying quarterly information on funds received and availability of drugs.


21

Expenditure and Utilization Survey, 2018, which reported 21 percent (similar to 2013) of patients had bypassed the nearest and lowest level health facility for lack of drugs. Justification of Overall Efficacy Rating

46. Overall rating for efficacy is Substantial. This is based on the following ratings for the five parts of the project.

Figure 5. Rating of Efficacy

PDO Parts Rating

PDO1, Part One: improve the delivery of quality essential health and nutrition services

Substantial

PDO1, Part Two: improve utilization of essential health and nutrition services by women and children among the poor and drought affected populations

Modest

PDO2, Part One: Effectiveness of Planning Modest

PDO2, Part Two: Effectiveness of Financing Substantial

PDO2, Part Three: Effectiveness of Procurement Substantial

C. EFFICIENCY

Assessment of Efficiency and Rating

47. Cost Benefit. Project specific economic evaluation was not undertaken for the project during appraisal. The ICRR, however, includes an economic analysis at the end of the project to estimate the extent to which the project was able to yield economic and financial returns to Kenya by supporting the delivery of the basic package of essential health services. Given net estimated cash flow in terms of income of lives saved from mothers, the estimated internal rate of return stands at 9.9 percent. At the given discount rate of 3 percent and the net present value of US$$60,912,114.67 the benefits outweigh the costs. Further, the project has a benefit‐cost ratio of 1.74:1. Sensitivity analysis was conducted by increasing the discount rate to 5 percent still yielding a positive return at US$3,410,258.54 or a benefit‐cost ratio of 1.04:1. This analysis only focused on the reduction of maternal mortality and excluded child health and nutrition. The project contribution to improvements in child health were considered minimal. Nutrition data was unavailable. It is important to note that, while data on the costs and cost‐effectiveness of humanitarian feeding programs are generally lacking, there are concerns regarding the cost‐effectiveness of the programs. A few studies carried out reveal that evidence regarding cost effectiveness of supplementary feeding programs are context specific and cannot be generalized32. Further, the gains realized from increased efficiency in

32 Emergency, Nutrition Network; The impact and effectiveness of emergency nutrition and nutrition‐related interventions: a review of published evidence 2004‐2010; 2011


22

KEMSA’s operation were not considered as separate outcomes by themselves and were treated as inputs to increased access and utilization of basic package and the resultant improvement in maternal health outcomes.

Implementation efficiency

48. Design: The project in its design was meant to support implementation of the KEPH. The package consisting of cost‐effective interventions aimed at addressing the major causes of disease burden in Kenya. Analysis of the package reveals the interventions are consistent with Kenya’s disease burden and the World Health Organization recommended list of cost‐effective interventions. In addition, the project provided nutrition supplements to populations affected by drought, focusing on children under five years, pregnant and lactating women. While provision of food supplements is generally considered less cost‐effective and unsustainable, they are often distributed during emergencies as part of the humanitarian response, which was the case with the project. 49. Implementation: The project was designed to use the HSSF as its main implementation platform. As explained above and in the sections below, both the HSSF and RBF mechanisms suffered major implementation setbacks with the onset of devolution, the effects of which to a large extent were never fully resolved up to the end of the project, despite several measures put in place as well as the capacity building initiatives undertaken by the project. The project was restructured in 2016 extending the closing date to allow for RBF implementation; however, even with the additional time, the problems remained largely unresolved. In addition to the delayed disbursements to the counties, there were significant disbursement delays to virtually all the other major implementing entities including KEMSA, UNICEF, AMREF and KMTC. Furthermore, the changes as result of project restructuring added to the complexity of the project. Finally, while IIFRA diligently conducted reviews, the reports were not shared with stakeholders and could not therefore inform project implementation.

50. Efficiency is rated as Modest. This is based on the fact that while the project based on the economic analysis yielded a positive return, the rate was borderline. Secondly, the analysis did not factor the implementation challenges which beset the project with the advent of devolution.

D. JUSTIFICATION OF OVERALL OUTCOME RATING

51. Overall project outcome is rated Moderately Satisfactory. This is based on the High rating for relevance of the PDOs, Substantial rating for efficacy, and Modest rating for efficiency.

E. OTHER OUTCOMES AND IMPACTS (IF ANY)

Gender

52. The project directly targeted women providing a range of essential reproductive and maternal health services. In doing so, the project contributed to improving access to health services for the poor and, particularly, poor women. In addition to the 34,447,148 women beneficiaries, the project supported over 600,000 women to receive free delivery services that met the quality norms at public health facilities. Institutional Strengthening 53. The project supported various institutional strengthening initiatives. These include the accountants recruited to facilitate HSSF implementation and the HFMCs established with project support in the 21 Counties. As


23

explained earlier, the accountants were absorbed to continue serving in the counties. While devolution disrupted the project’s fund flow arrangements, the project contributed to building capacity of counties to manage fiscal transfers from the government. The reforms of the supply chain management system were to a large extent supported by the project which have enhanced KEMSA’s institutional capacity. With support of the project and other partners, KEMSA successfully adopted and is currently using the Enterprise Resource Planning System, Logistics Management Information System and an e‐filing system, which have boosted its capacity to procure and supply medical supplies and commodities in the country.

Mobilizing Private Sector Financing

54. The HIA Kenya program in collaboration with the project provided technical support to the NHIF towards the design and implementation of HISP. In addition, HIA supported NHIF to develop instruments to enhance recruitment of informal sector employees into the health insurance scheme.

Poverty Reduction and Shared Prosperity

55. The project was pro‐poor. It (a) supported lower level health facilities utilized mainly by the poor, (b) targeted drought affected populations supplying nutrition commodities to manage malnutrition among children under five years, pregnant and lactating women, and (c) supported the rollout of the government flagship program providing health insurance subsidies to the poor. Furthermore, the RBF program was implemented in the 21 semiarid and arid counties where poverty levels are highest in Kenya. Other Unintended Outcomes and Impacts

56. Not applicable.

III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME

A. KEY FACTORS DURING PREPARATION

57. Project preparation was protracted33, was participatory and involved major stakeholders of the health sector. The preparation took account of the country and sectoral context as well as the contribution of other development partners including experience of the Bank’s past involvement in the health sector in Kenya, especially the DARE project. Project preparation benefited from several diagnostic studies and assessments including a comprehensive pharmaceutical study which informed the design of Component 2. Careful attention was given to effectiveness conditions, which enabled the operation to become effective on October 1, 2010 shortly after approval in June 2010. The HSSF, a major reform supported under the project was designed based on the model successfully piloted in two provinces. 58. The PDOs while broad and complex were aligned with national and sectoral priorities. PDO 1 referred to “delivery of essential health services” without specifying actual services. PDO 1 also referred to the poor in terms of level of health service delivery (Level 1‐3), but the design did not describe mechanisms to target the poor as the project was designed to support a national program and project beneficiaries were not explicitly mentioned in the appraisal document. PDO 2 on the other hand was more transactional in its description. To ensure clarity, PDO 1 was later revised during the AF to exclude reference to the Program and to specify project

33 The concept note for the project was reviewed in 2005, and by 2007 a full package of documentation was prepared for a Decision Meeting. However, due to governance concerns about the outbreak of post‐election violence, the split of the MOH as part of the political agreement relating to the establishment of the Coalition Government, and the subsequent institutional problems facing the MOMS and MOPHS, World Bank management decided to suspend Project processing throughout 2008.


24

beneficiaries; and in the process the quality dimension was added. The results framework was designed to use the regular government information system which eased the collection of the data. The intermediate outcome indicators were quite many, and some could have been excluded as was later done during the restructuring of the project. 59. The project was designed as part of the SWAp to support the government’s program to reform the health sector. This was meant to ensure synergy and complementarity with other partner programs. Coordination mechanisms were put in place such as a SWAp secretariat which ensured improved collaboration and information sharing across key actors thereby avoiding duplication of interventions and allowing for a more overall strategic engagement. However, the scope of project activities was broad, and the institutional and implementation arrangements complex given the governance arrangements between the two Ministries and large number of entities involved in project implementation. 60. A comprehensive risk assessment was conducted during project preparation and a mitigation plan adopted. This included a governance assessment of the overall health sector, and both components of the project were designed to include specific activities that address governance and corruption concerns. The project was essentially ready for implementation by Bank approval. Its implementation arrangements were in place and it was declared effective within two months of approval.

B. KEY FACTORS DURING IMPLEMENTATION

Factors subject to Govt/implementing agency control. 61. The introduction and rollout of the HSSF. The HSSF was rolled out successfully. The HSSF was a home‐grown reform program developed and piloted before national scale‐up. It marked the first‐time direct transfers of funds were made to frontline health facilities to meet day‐day operational and maintenance costs. The HSSF was successful and by 2013 the funds allocated to the HSSF were almost fully disbursed. 62. Strong commitment and political will to reform the supply chain management system for medicines and medical supplies. While KEMSA experienced operational challenges, because of the strong support from government and development partners, it was able to transform into an autonomous agency, establish a buffer stock, and establish more efficient and effective systems for the planning and management of the supply chain system for medicines and medical supplies. The reforms enabled KEMSA to effectively carry out its mandate in the supply chain management of medicines. 63. Planned and targeted capacity building activities. While there were delays with disbursements, and disruptions with the transfer of funds to the health facilities, the institutions contracted to carry out targeted capacity building activities under the project were successful in enhancing capacity of the sub‐national level entities. Kenya School of Government trained over 2,900 County officers out of 4,000 in public financial management. AMREF trained members of the Health Facility Management Committees in 21 RBF Counties. Most of the trained officials continue to support the various departments in the counties including the health department. In addition, most of the district accountants were absorbed by the counties.

64. The government decision to abolish user fees at primary level health facilities and introduce free maternity services. In support of the two reforms, the government increased the budget to the health sector. The abolition of user fees contributed to the increased utilization of health services including maternity services.


25

While a statistically significant increase in the number of facility‐based deliveries was identified, there were no significant changes in the ratio of maternal mortality and the rate of neonatal mortality34.

65. Impact of devolution on transfer of funds to the health facilities. The devolution process greatly impacted project implementation, especially the HSSF and RBF fund flow arrangements to the health facilities. Prior to devolution HSSF and RBF grants were directly transferred from the center to the health facilities. However, following devolution, conditional grants and project disbursements were channeled through the respective County Revenue Fund (CRF) Accounts. The grants were then transferred from the respective CRF Account to a County Special Purpose Account for Health for onward disbursement to the beneficiary facilities. This resulted in delays of between 3‐6 months in releasing the RBF and conditional grants to the facilities. According to the IIFRA, the main causes for the delay were diversion of funds and slow processing and approval of funds by the counties. Secondly, because of the changes, RBF disbursements to counties were made based on the County Allocation of Revenue Act provisions rather than performance. Attempts to ensure the transfers were based on performance were unsuccessful, as it was deemed that performance‐based transfers would contravene the existing Act. The introduction of devolution also greatly disrupted the supply of medicines. Counties were no longer obliged to source medicines from the government‐run KEMSA, which led to mismanagement and scarcity of drugs at health facilities. It is reported that nearly all counties are faced with challenges ranging from capacity gaps, human resource deficiency, lack of critical legal and institutional infrastructure, and a conflictual relationship with the national government, the net effect of which has led to stagnation and in some cases a reversal of some of the previous gains in the sector35. Originally, devolution was planned to be rolled out over a three‐year period; instead the pace was hastened. Besides some of the enabling policy frameworks were not in place. 66. Merger of the Ministry of Medical Services and Ministry of Public Health and Sanitation. The decision to merge the two ministries resulted in changes to the project’s institutional and implementation arrangements. New coordination and implementation arrangements for the project had to be established. Partly, as a result, there was a high turnover of staff responsible for coordination of project implementation and loss of institutional memory. Furthermore, the changes impacted on the execution and follow‐up of some core tasks necessary for project implementation leading to delays in: (a) release of funds by the MoH to implementing entities; and (b) signing/extension of key agreements/memorandum of understanding with KEMSA, AMREF, KSG, KMTC and UNICEF.

67. Delayed disbursements of funds. While delay with disbursement of funds was most pronounced at the health facilities, especially after the advent of devolution, all other entities including KEMSA, KMTC, NHIF, AMREF and Kenya School of Government suffered major disbursement delays. The implementation of activities by these entities not only started late due to delay with signing memoranda of understanding, implementation was further slowed down by the delayed disbursements by the MoH. After devolution, the Counties through the Council of Governors, took a long time in the 2013/14 and 2014/15 financial years before signing the memorandum of understanding (MOU) for the KHSSP conditional grants. This adversely affected the release of conditional grants and led to non‐implementation of some planned activities by the health facilities.

34 C. M. Gitobu, P. B. Gichangi, and W. O. Mwanda. The effect of Kenya’s free maternal health care policy on the utilization of health facility delivery services and maternal and neonatal mortality in public health facilities. BMC Pregnancy and Childbirth. 27 March 2018. 35 Leah Kimathi, Challenges of the Devolved Health Sector in Kenya: Teething Problems or Systemic Contradictions? Africa Development, Volume XLII, No. 1, 2017, pp. 55‐77 © Council for the Development of Social Science Research in Africa, 2017 (ISSN: 0850 3907)


26

Factors outside Govt/implementing agency control 68. Industrial action by health workers. The doctors 100‐day strike in 2016 and nurses’ industrial action of 2017 which lasted over 5 months severely disrupted delivery of health services. The impact of the industrial action was particularly severe for front line services supported by the project. The nurses’ strike gravely impacted the provision of health and nutrition services country‐wide. Over 50 percent of primary health facilities were closed in six drought‐affected Counties. Analysis of District Health Information System 2 (DHIS2) data for Turkana and Garissa counties revealed dramatic declines in immunization rates from 80 percent in January ‐ May 2017 to 30 percent in June – July. Similarly, Ante‐natal Care attendance dropped from 40 percent to 10 percent and Skilled Birth Attendance dropped from 60 percent to 30 percent36. 69. Drought. Kenya experienced severe drought which affected the most impoverished areas of the country populated by nomadic and semi‐nomadic pastoralists and subsistence farmers. As part of the support towards the government’s response the project obtained AF to help Kenya address more effectively the emergency health and nutrition needs of most vulnerable populations, especially pregnant and lactating women and children under five in the drought affected northern arid and semi‐arid regions.

Factors subject to WB control 70. Additional Financing and project restructuring. Through the AF in 2011 and 2013 the project supported the government’s humanitarian response to drought affected populations; supported the free maternity care following abolition of user fees for delivery services; and expanded the supply of medicines and other medical supplies through KEMSA. The restructuring (a) refocused the PDO to support the drought affected populations; (b) targeted women and children as project beneficiaries; (c) adapted the implementation arrangements to the ongoing reforms in the country; and (d) provided time for implementation of delayed project activities. But in the process, the new initiatives added made the project more complex. 71. The Bank team’s proactivity and resilience. The project was implemented in a complex environment amid many challenges; but the task team remained proactive in their engagement with government and partners which enabled the project to undergo restructuring and receive the AF to address the adverse effects of drought and scale‐up project interventions.

IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME

A. QUALITY OF MONITORING AND EVALUATION (M&E)

M&E Design 72. Quality of M&E design is rated modest. The project’s results framework was designed with the majority of indicators collected from routine data sources and complemented by surveys where appropriate. All the indicators had baseline data and the expected targets to be achieved. The intermediate outcome indicators were many. Owing to the SWAp approach, the results framework was designed to measure national level implementation, although the indicator on immunization was disaggregated to the North East Province for equity consideration. The design

36 UNICEF, Kenya Humanitarian Situation Report – 1 September 2017


27

included the use of IIFRA to monitor the execution of the project. A major shortcoming of the results framework was that the design did not include the appropriate PDO indicators to adequately measure the delivery of essential health services. With exception of immunization, the results framework consisted of mainly input and process indicators. Indicators on coverage of specific program interventions or targeting specific population groups were lacking. This was however partly rectified during the AF with incorporation of indicators targeting children and women and utilization of nutrition services. M&E Implementation 73. Quality of M&E implementation is rated modest. The results framework was regularly populated and the Implementation Status and Results Reports (ISRs) updated. The ISR ratings were realistic and generally reflected the implementation status of the project. The results framework was revised to reflect changes to the project and to take account of implementation experience with devolution. With counties becoming responsible for health services delivery, the monitoring of such indicators would be difficult from the center. The changes included dropping some indicators, adding new indicators and revision of the targets with the aim of simplifying the results framework and ensuring that it is aligned with the revised project objectives. However, in the process, the project became more complex. Furthermore, these efforts did not rectify the weaknesses around the PDO outcome indicators. Instead relevant PDO indicators (staffing norms and drug shortages) were dropped. In addition, immunization data for the North East region were not reported. Two major studies were conducted during project implementation: household expenditure and utilization survey and the service delivery indicator survey. Owing to delays, both studies had not been concluded by project closing and as such could not directly inform project implementation. It is, however, envisaged that the findings will inform future developments of the overall health sector program in Kenya. IIFRA on its part conducted periodic reviews of the project and issued the reports. In addition, the project contributed towards the Kenya Demographic Health Survey and STEPwise approach to Surveillance for non‐communicable diseases and the evaluation of the HISP. M&E Utilization 74. Quality of M&E utilization was modest. The project generated a considerable volume of data. The use of the data was however mixed. KEMSA built an effective electronic management information system, which was used to monitor execution of its programs and to drive the reform agenda towards the “pull” system. With devolution, the quality of reporting by the counties deteriorated considerably. For various reasons, many of the counties failed to provide timely progress and financial reports to the Ministry of Health. In addition, while IIFRA submitted its reports to the Ministry of Health, the reports were not shared with the key stakeholders including the Bank team. As a result, use of the IIFRA findings was limited. This was a missed opportunity since IIFRA was responsible for assessing compliance with approved work plans; verifying activities in the approved work plans were executed and planned results achieved, and linking the expenditures incurred to successful achievement of the envisaged results.

Justification of Overall Rating of Quality of M&E 75. Overall rating for quality of M&E is modest. This is based on the modest quality of M&E design, modest quality of M&E implementation and modest quality of M&E utilization.

B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE


28

76. Environmental safeguards. The project triggered Operational Policy 4.01 relating to Environmental Assessment and was classified as environmental Category B. The environmental safeguard policy issue related to handling and disposal of medical waste generated by the health facilities. The project procured incinerators for five hospitals and supported the training of health providers on the handling and disposal of medical waste and infection control. The project overall was compliant with the environmental safeguard policy. Health facilities segregated medical waste using color coded bins. The environmental and social audit report on the installation of the incinerators was approved by National Environment Management Authority (NEMA). By the time of project closure, the Ministry of Health was following up on the implementation of the audit findings and recommendations by NEMA on decommissioning of old incinerators and ensuring that hospitals obtain a license from NEMA to operate the incinerators. 77. Social safeguards. The project was designed to address inequities in access to, and utilization of, health services, which was a central goal of the NHSSP II. The project triggered OP 4.10 on Indigenous Peoples, and for which an Indigenous Peoples Planning Framework (IPPF) was prepared at appraisal. During implementation, and especially after the AF, attention was paid to increasing access and utilization of health services by the poor and the vulnerable populations. Emphasis was made to target drought affected population, and women and children from the arid and semiarid regions of Kenya. The poor were also targeted through HISP. It is important to note that deliberate efforts were made to enhance the inclusion and participation of the vulnerable and marginalized groups through community empowerment, promotion of good and desirable health practices, making services affordable, and ensuring the health facilities serving the vulnerable and marginalized groups obtained sufficient medicines. The HFMCs served as a key link with the communities to channel their grievances. Service charters were translated into local languages. Facilities also kept locked feedback boxes, where grievances can be submitted anonymously. In addition, the NHIF implementing the Health Insurance Subsidies for the Poor (HISP) operated a 24‐hour toll‐free hotline and a dedicated email address to handle client complaints. 78. Financial management. At midterm review (August 2013), financial management arrangements at the MoH central level (Health SWAp and HSSF Secretariats) and counties were reported as adequate. There were no outstanding audit issues and interim financial reports were submitted in a timely manner and in a form acceptable to the Bank. However, the advent of devolution severely disrupted the HSSF financial management arrangements and adversely affected the transfer of grants to the health facilities. Many counties were no longer submitting financial returns despite repeated follow up by the Ministry of Health. By project closure, several issues raised in the audit reports and other reviews had not been addressed including ineligible expenditures of Kshs 295,785,262 and potential ineligible expenditure of Kshs 1,166,120,469. The audit report for the period ending June 30, 2018 also noted that the statement of financial assets and liabilities reflected a bank balance of Kshs 1,029,248,879 as at 30 June 2018. Included in this balance was Kshs 657,875,266 in respect of bank balances from 47 counties that were not supported with bank confirmation certificates, bank reconciliation statements and cashbook. Furthermore, the National Hospital Insurance Fund implementing the HISP did not submit quarterly reports to the MoH as required. As reported above, while IIFRA diligently conducted the reviews, the reports were not shared with stakeholders and could not therefore inform project implementation. The FM issues are being followed up under the new follow‐on Project. 79. Procurement. The bulk of procurements under the project were carried under component two by KEMSA. Prior to the project, procurements by KEMSA experienced major shortcomings, but with support of the project, KEMSA made substantial progress in the procurement of essential medicines and implementation of the procurement plan. Improvements were noticed with respect to compliance in terms of procurement planning, bid preparation, publication of contracts and filing and records management. However, KEMSA needs to improve the


29

quality of bid evaluation reports, strengthen contract administration and management, and pay suppliers in a timely manner. While few, procurements by the MoH were slow and experienced considerable delays. In the project design, health facilities were meant to handle small procurements. Because of the changes brought about by devolution, these procurements, including those for essential medical equipment could not go ahead and were abandoned.

C. BANK PERFORMANCE

Quality at Entry

80. The Bank’s performance with respect to quality at entry is rated moderately satisfactory. The project was aligned with the strategic development objectives of the sector and country as described in the HSSIP II (2005‐2012) and Vison 2030. The preparation and design of the project involved broad stakeholder consultations and several analytics, took cognizance of the Bank’s prior experience with the health sector, and reflected the country and sectoral context. Great attention was paid to assessment of the governance environment including fiduciary arrangements and putting in place appropriate mitigation measures. The majority of the QER recommendations were incorporated during the project design. The project had robust implementation arrangements and was able to become effective two months after Board approval. The long project preparation duration meant team members kept changing, but the overall team composition remained comprehensive. The major shortcomings were with the broad scope and complex design of the project. The project apart from mentioning the poor did not explicitly identify its beneficiaries, nor were the actual services to be delivered mentioned. These as explained earlier were only partly addressed during the restructuring of the project. Besides the broad PDOs and complex nature of the project, the project design did not include appropriate PDO indicators to adequately measure the delivery of essential health services. Quality of Supervision 81. The Bank’s performance with respect to quality of supervision is rated satisfactory. The majority of the task team members including the task team leader were based in the Kenya Country Office and had been involved in project preparation. The team leader during Board approval stayed throughout the project duration as team leader. The team jointly with other partners and the MoH held regular biannual implementation support missions. In addition to assessing progress with project implementation and achievement of the project objectives, the missions assessed compliance with fiduciary and safeguards policies as well as the legal covenants. The outcomes were captured and adequately reflected in the aide memoires and ISRs. The ISR ratings were generally consistent with the mission findings and reflected the status of project implementation. 82. The team was proactive in addressing the project’s shortcomings and responding to the changing country dynamics. This was demonstrated by the team’s effort in obtaining AF and restructuring the project. The team was proactive and established strong rapport with partners and government officials which provided them a good opportunity to access MoH and other government agencies and to understand the changing country context and dynamics. Their understanding of the local context helped them to effectively supervise the project and respond to the changing dynamics around project implementation. Despite the team’s efforts, the disruptions to the fund flow arrangements persisted and severely impacted the implementation of RBF activities. As noted above (M+E section), the weaknesses around the PDO outcome indicators were only partially addressed. Furthermore, the opportunity to use IFRAA findings to monitor project execution were missed.


30

Justification of Overall Rating of Bank Performance 83. Overall Bank performance is rated moderately satisfactory.

D. RISK TO DEVELOPMENT OUTCOME 84. Overall risk to development outcome is moderate. The Project was executed using country systems, did not finance new health infrastructure outside the existing health facilities and focused on building institutional capacity at county and facility levels. Most staff (technical officers in financial management and HFMCs) trained under the project continue to serve in their respective capacities. The 400 nurses who were under training during the mission were expected to return and serve in their communities. With the systems for procurement and supply chain management using automated Enterprise Resource Planning Program, KEMSA has become more successful as a public‐sector procurement agency. While KEMSA’s reliance on release of funds by the MoH has reduced there is the increased risk of fragmentation of the budget for medicines with the onset of devolution. The devolution disrupted the RBF and HSSF fund flow arrangements, but created a new mechanism for intergovernmental fiscal transfers, which provides a good opportunity for government and partners to make direct transfer of funds to health facilities. However, this will require the Ministry of Health to proactively engage in the ongoing discussions on the devolution process. Building on the project, the Bank prepared a follow‐on project “Transforming Health Systems for Universal Care Project” (P152394, $191.1 m) to continue support to the Kenya Health system. With project closing, the government has taken over the funding of the HISP. The new project is also using a performance‐based approach to allocate resources to the counties based on improved PHC results.

V. LESSONS AND RECOMMENDATIONS

85. The challenge of projects responding to major government reforms. The devolution process in Kenya was very ambitious, fast‐tracked and unfolded in an environment where the building blocks for devolution were still evolving. For such major reforms, like the Kenya one, while flexibility was needed, it is necessary to exercise caution, otherwise, the project risks losing focus and playing a catchup and/or firefighting role. 86. Direct funding of health facilities. Initiatives and programs aimed at directly transferring funds to health facilities like the HSSF and RBF can potentially improve service delivery and promote transparency as well as accountability. Implementation of both the HSSF and RBF programs faltered in Kenya. To ensure success of such initiatives, it is necessary to ensure: (a) proactive and continuous dialogue between the central and county levels of the Ministry of Health to clarify respective roles and responsibilities: (b) a supportive regulatory framework is in place anchored within an appropriate legal instrument; (c) fund flow arrangements are clarified and clearly enshrined in the devolution framework; and (d) health facilities are adequately capacitated to perform the necessary fiduciary roles.

87. The reforms undertaken by KEMSA were successful and demonstrate the importance of: (a) a comprehensive diagnostic assessment conducted in a consultative manner; (b) securing consensus among partners and with government on the proposed reforms, (c) ensuring the reforms are institutionalized within a clear institutional framework; and (d) provision of the necessary technical assistance and adequate resources to support the reform process.


31

88. In‐country presence of the Task Team. The project was a complex one implemented in a challenging context. The continuity and in‐country presence of the task team and especially the task team leader was critical in the context of the fast‐tracked devolution which introduced complex reforms in the health sector. This allowed the project to address implementation gaps as they arose and to work together with central and county government towards effective solutions. 89. The importance of supporting home grown initiatives. The HSSF was a home‐grown reform program developed and piloted before national scale‐up. While its implementation suffered with devolution, it was a successful program, demonstrating the value of building on home grown initiatives, piloted and executed within the government institutions. On the other hand, the HISP, a government flagship program, supported by the project in the first phase is now fully funded under the UHC umbrella through the National Treasury after the closure of the project.

90. The use of government institutions to implement capacity building activities. The project used established GOK institutions to implement key capacity building activities in many areas, which has enabled continuity. Financial management training was undertaken by Kenya School of Government. The Enrolled Community Nursing Program by KMTC, has ensured continuity in provision of the curriculum and courses offered by the institution to date. .


32

ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS

A. RESULTS INDICATORS A.1 PDO Indicators

Objective/Outcome: The delivery of essential health services in the Recipient’s territory, especially for the poor;

Indicator Name Unit of Measure Baseline Original Target Formally Revised

Target

Actual Achieved at Completion

Direct project beneficiaries Number 25896000.00 29300000.00 35000000.00 50949459.00

05‐Jul‐2010 31‐Mar‐2015 29‐Jun‐2018 29‐Jun‐2018

Female beneficiaries Percentage 14229369.00 16100100.00 17500000.00 34447148.00

05‐Jul‐2010 31‐Mar‐2015 29‐Jun‐2018 29‐Jun‐2018

Beneficiaries from drought affected areas

Number 0.00 332500.00 5500000.00 15088032.00

05‐Jul‐2010 31‐Mar‐2015 29‐Jun‐2018 29‐Jun‐2018

Comments (achievements against targets): Target Achieved


Target


Children immunized Number 1188698.00 1800000.00 1298058.00 1105234.00


33

(number) 05‐Jul‐2010 31‐Mar‐2015 29‐Jun‐2018 29‐Mar‐2018

Comments (achievements against targets): Target Not Achieved


Target


Severely malnourished children under five receiving treatment recovered

Percentage 0.00 60.00 60.00 84.00

30‐Jun‐2011 31‐Mar‐2015 29‐Jun‐2018 29‐Jun‐2018



Target


Women receiving free delivery services meeting the quality norms at public health facilities

Number 453857.00 550000.00 550000.00 604521.00

01‐Jul‐2013 30‐Dec‐2016 29‐Jun‐2018 29‐Jun‐2018



Target


Poor households receiving health insurance subsidies

Number 0.00 30000.00 30000.00 181689.00

01‐Jul‐2013 30‐Dec‐2016 29‐Jun‐2018 29‐Jun‐2018


34

Comments (achievements against targets): Target Achieved. In total, 6 out of 7 PDO Outcomes were Achieved

Objective/Outcome: The effectiveness of planning, financing, procurement of pharmaceuticals and medical supplies


Target


Value fill rate of orders for commodities for Primary Health Care facilities received by KEMSA either from counties or facilities

Percentage 73.00 90.00 80.00 83.00

05‐Jul‐2010 31‐Mar‐2015 29‐Jun‐2018 29‐Jun‐2018



Target


Share of planned procurement from EMMS pool completed by KEMSA in the financial year as per GoK guidelines

Percentage 0.00 60.00 60.00 100.00

31‐Dec‐2013 30‐Dec‐2016 29‐Jun‐2018 29‐Jun‐2018


A.2 Intermediate Results Indicators

Component: The delivery of essential health services in the Recipient’s territory, especially for the poor


35


Target


People with access to a basic package of health, nutrition, or reproductive health services (number)

Number 0.00 30495000.00 39400000.00 47143791.00

05‐Jul‐2010 31‐Mar‐2015 29‐Jun‐2018 29‐Jun‐2018



Target


Facility management committees having quarterly meetings with minutes of meetings (levels 2 and 3) (percent)

Percentage 27.00 90.00 90.00 83.00

05‐Jul‐2010 31‐Mar‐2015 29‐Jun‐2018 29‐Jun‐2018

Comments (achievements against targets): The Target was Substantially Achieved (92%)


Target


Health personnel receiving training (number)

Number 0.00 35120.00 14899.00 19271.00

05‐Jul‐2010 31‐Mar‐2015 29‐Jun‐2018 29‐Jun‐2018



36


Target


Level 2‐3 facilities implementing results based financing initiatives (number)

Number 0.00 250.00 1331.00 1770.00

05‐Jul‐2010 31‐Mar‐2015 29‐Jun‐2018 30‐Mar‐2018



Target


Number of beneficiaries receiving supplementary feeding in the drought affected areas. (Moderately malnourished children under five and moderately malnourished pregnant and lactating women)

Number 157413.00 285210.00 285210.00 450582.00

01‐Jul‐2011 31‐Mar‐2015 29‐Jun‐2018 29‐Jun‐2018


Component: The effectiveness of planning, financing, procurement of pharmaceuticals and medical supplies


Target


Counties placing orders for Essential Medicines and Medical Supplies with KEMSA

Number 16.00 30.00 30.00 47.00

31‐Jan‐2014 29‐Jun‐2018 29‐Jun‐2018 30‐Mar‐2018


37

Comments (achievements against targets): Target Achieved In total, 5 out of 6 Intermediate Outcome Indicators were Achieved.


38

B. KEY OUTPUTS BY COMPONENT

Objective/Outcome 1: Effective and transparent implementation of the KEPH through HSSF grants and performance strengthening (total estimated cost US119.68 million)

Outcome Indicators

1. Direct project beneficiaries (Number of outpatient visits to levels 2 and 3) Female Beneficiaries Beneficiaries from drought affected areas (Number)

2. Children immunized (number, disaggregated for Northeast Province) 3. Value fill rate of orders for commodities for Primary Health Care facilities received by KEMSA either from counties or facilities (Percent) 4. Severely malnourished children under five receiving treatment recovered (Percent) 5. Share of planned procurement from Essential Medicines and Medical Supplies (EMMS) pool completed by KEMSA in the financial year as per GoK guidelines (Percent) 6. Women receiving free delivery services meeting the quality norms at public health facilities (Number) 7. Poor households receiving health insurance subsidies (Number)

Intermediate Results Indicators

1. People with access to a basic package of health, nutrition and reproductive health services 2. Facility management committees having quarterly meetings with minutes of meetings 3. Health personnel receiving training 4. Level 2‐3 facilities implementing results‐based financing initiatives 5. Number of beneficiaries receiving supplementary feeding in drought affected areas 6. Counties placing orders for essential medicines and supplies with KEMSA

Key Outputs by Component (linked to the achievement of the Objective/Outcome 1)

1. Contributed to the delivery of primary health care services to 50 million direct project beneficiaries, including 34.4 million women, 0.6 million women receiving free delivery services and 15 million beneficiaries from the drought affected regions; 2. Provided health insurance cover to 181,968 poor households ‐ included outpatient and inpatient services; 3. Improved treatment outcome for children severely malnourished, achieving 84 percent recovery rate; 4. Rolled out HSSF to all public health facilities and RBF to 21 counties reaching 1,771 primary health facilities;


39

5. Supported training of 19,271 beneficiaries, including training 400 students as enrolled community health nurses with focus on midwifery to address human resource shortages in underserved counties; 6. Trained Health Facility Management Committees (HFMCs) in 21 RBF counties; 7. Trained 2,994 county executives, administrative and technical officers in financial management; 8. Supported the service delivery indicators survey targeting 3,281 health facilities and a new round of household expenditure and utilization survey (HHEUS) targeting 37,500 households

Objective/Outcome 2: Availability of essential health commodities and supply chain management reform (US$98.11 million)

Outcome Indicators 1. Share of planned procurements from EMMS pool completed by KEMSA in the FY per GoK guidelines 2. Value fill rate for commodities on KEMSA Essential Drug List 3. Fiduciary irregularities reported from HSSF and KEMSA acted upon

Intermediate Results Indicators

1. Number of beneficiaries receiving supplementary feeding in drought affected areas 2. Facilities under pull system sending orders in previous quarter 3. Timely reimbursement to KEMSA documented claims by MoH 4. Counties placing orders for essential medicines and supplies with KEMSA

Key Outputs by Component (linked to the achievement of the Objective/Outcome 2)

1. Supplied essential health and nutrition commodities, cold chain equipment and basic equipment to the 21 RBF counties; 2. Improved KEMSA’s performance to achieve 100 percent execution of planned procurement for the FY; 3. Supported establishment of an electronic record keeping system, development of a procurement manual and an electronic Logistic Management Information system; 4. Supported KEMSA’s transformation from an agency that was dependent only on government resources to an adequately resourced independent procurement agency responding to the needs of 47 counties and the national government; 5. Supported KEMSA to create a buffer stock and to reform the supply chain management system from a “push” to a “pull” system.


40

C. Author’s Results Framework – Kenya Health Sector Support Project: Reflects original and revised baselines and targets from 2011 and 2013 AF Indicators Original

Baseline 2011 Baseline

2013 Baseline

Original Target 2011 Target 2013 Target Achieved

Project Outcome Indicators

PDO 1

Direct project beneficiaries Number Female (55%) Number from drought affected areas

25,896,000 14,299,369

0

29,300,000 16,100,100

332,500

35,000,000 17,500,100 5,500,000

50,949,459 34,447,148 15,088,032

Children immunized From North East

1,188,698 32,800

1,800,000 65,000

1,298,058 1,105,234

HSSF facilities meeting core financial management requirements of the fund – Dropped 2013

0% 90% 85% 97% (2013)

Level 2 and 3 HSSF facilities displaying information as per GoK guidelines ‐ Dropped 2013

0% 90% 90% 80% (2013) 77% (2013)

Severely malnourished children under five receiving treatment recovered ‐ New 2011

‐ 60% 84%

Women receiving free delivery services meeting quality norms at public facilities – New 2013

453,857 550,000 604,521

Poor households receiving health insurance subsidies – New 2013

0 30,000 181,968

PDO 2

Health facilities that experience essential drugs stock‐outs for more than two weeks ‐ Replaced 2011

18% 5% Not Assessed

Timely payment to suppliers by KEMSA based on agreed procurement contract – Replaced 2011

60% 80% Not Assessed

Facilities displaying quarterly information on funds received and availability of drugs – Replaced 2011

0% 90% Not Assessed

Share of planned procurements from EMMS pool completed by KEMSA in the FY per GoK guidelines – New 2013

0% 60% 100%

Value fill rate of orders for commodities for primary health care facilities received by KEMSA either from counties or facilities – New 2011and revised in 2013

73% 90% 80% (2013) 83% (2018)

Fiduciary irregularities reported from HSSF and KEMSA acted upon (last quarter) – new 2011 and Dropped 2013

0% 100% 90% (2013) 100%

Intermediate Outcome Indicators (PDO 1)

People with access to a basic package of health, nutrition and population services

0 30,495,000 39,400,000 47,143,791


41

Facilities receiving HSSF Funds within 15 days of the beginning of the quarter – Dropped 2013

0% 90% 60% (2013) 0

Facility management committees having quarterly meetings with minutes of meetings

Level 2 27% Level 3 46%

90% 83%

Health personnel receiving training 0 35,120 6,500 14,899 19,271

Level 2 and 3 facilities meeting staffing norms – Dropped 2013


40% 40% (2013) Not Available

Level 2 and 3 facilities providing reports on services to OVCs ‐Dropped 2011

0% 50%

Dropped Not Assessed

Successful completion of innovative initiatives to improve referral to district hospitals and for improved health financing – Dropped 2011

0 5

Dropped Not Assessed

Level 2‐3 facilities implementing results‐based financing initiatives

0 40%

250 1,331 1,770

Number of beneficiaries receiving supplementary feeding in drought affected areas – New 2011

157,413 285,210 450,582

Intermediate Outcome Indicators (PDO 2)

Performance contract between KEMSA and MOMS, and a client service agreement between KEMSA and MOPHS, established and monitored on a quarterly basis ‐ Dropped 2013

N0 Yes Dropped Draft

Complaints reported, and actions taken through an accessible complaints mechanism on service delivery, fraud and corruption on a quarterly basis – Dropped/Replaced 2011


70% Not Assessed

Tenders and awarded contracts advertised on the KEMSA website and meeting the national and international standards for dissemination – Dropped 2011


On‐time delivery of drugs and medical supplies to level 2 and 3 facilities ‐ Dropped 2011

80% Not Assessed

Availability of pre‐shipment inspection certificate within KEMSA ‐ Dropped 2011


Timely completion of the Annual Procurement Plan – Dropped 2011

No Yes Not Assessed

Facilities under pull system sending orders in previous quarter – New 2011 and Dropped 2013

TBD TBD TBD 95%

Timely reimbursement to KEMSA documented claims by MoH –New 2011 and Dropped 2013

‐ 100% 75% 75%

Counties placing orders for essential medicines and supplies with KEMSA

16 30 47


42

D. The Kenya Essential Package for Health Cohort Service package

Cohort 1 Pregnancy and newborn (up to 2 weeks of age)

ANC and nutritional care, IPT, TT2. Use of skilled birth attendants, clean delivery, BCG. Postnatal care, breast feeding support, supplementary feeding. Family planning services. ITN promotion and use. IPT and indoor spraying. PMTCT. Micro‐nutrient supplements (iron). Hygiene, water and sanitation.

Cohort 2 Early childhood (2 weeks to 5 years)

Community IMCI, ITN promotion and use. Appropriate nutrition, extended breastfeeding, growth monitoring, EPI, provision of vitamin A, zinc. Psychological stimulation, physical / cognitive development. Exercise and recreation.

Cohort 3 Late childhood (6 ‐ 12 years)

Essential school health program. Adequate nutritional care. ITN promotion and use. Exercise and recreation.

Cohort 4 Adolescents (13 ‐ 24 years)

TT2 in schools. RH and HIV/AIDS/STI counseling. Substance abuse counseling. Adequate nutritional care. Accident prevention. RH/FP services. Exercise and recreation.

Cohort 5 Adults (25 ‐ 59 years)

Annual screening and medical examinations. Accident prevention. FP/RH services. Healthy lifestyles (exercise, recreation, nutrition, etc.).

Cohort 6 Elderly

Annual screening and medical examinations. Exercise and the promotion of general hygiene. Social/emotional/community support.


43

E. Theory of Change Prepared by Project Team During Implementation

Activities Outputs PDOs/Outcomes Long‐Term

Outcomes

Component 1: Delivery of quality essential health and nutrition services and utilization by women and children especially among poor and drought affected populations

Reduction in Maternal and Child Mortality Reduced Burden of diseases Universal Health Coverage

HSSF grants to primary health facilities (US$ 34 M)

Primary health facilities receiving HSSF grants in time

Effective delivery of essential health services

Health Sector Governance and Stewardship (US$ 12 M)

Facility management committees in place and meeting regularly Health personnel trained

Health facilities meeting core staffing and financial management requirements

Supply of Nutrition Commodities in Drought affected districts (US$ 12.8 M)

Delivery of nutrition services to malnourished children and pregnant mothers

Severely malnourished children receiving treatment recovered

Introduction of Results Based Financing in Arid and Semi‐Arid Counties (US$ 25 M)

Facilities implementing RBF initiatives

Improved Outpatient attendance; Institutional deliveries; Immunization coverage Women receiving free delivery services meeting the quality norms at public health facilities

Piloting Health Insurance Subsidy Program (HISP, US$ 36 M)

Poor households enrolled for health insurance subsidies


Component 2: Effectives of planning, financing and procurement of pharmaceutical and medical supplies

Improved availability of Essential Health Commodities (US$ 48 M)

Timely reimbursement to KEMSA documented claims by Ministries of health

Primary health facilities placing orders during previous quarter

Strengthening reforms to enhance efficiency and transparency of KEMSA US$ 6 M

KEMSA signs Performance contract Reforms in KEMSA – Procurement Manual; On‐line LMIS

Share of planned procurement completed by KEMSA in the financial year

Increased availability of essential health commodities to drought affected areas (US$ 44 M)

On‐time delivery of drugs and medical supplies to primary health care facilities

Value fill rate of orders for commodities for Primary Health Care facilities received by KEMSA either from Counties or facilities


44

F. HSSP SWAp Arrangements

The project was designed for implementation under a Sector‐Wide Approach (SWAp) arrangement. As part of the arrangement, the government and partners signed a Code of Conduct and specified the mechanisms for the implementation of the Joint Program of Work and Funding in the Joint Financing Agreement (JFA). The JFA established the joint provisions and procedures for financial support through pooling of resources from partners and the GOK for the implementation of the NHSSP II. It also served as a coordination framework for consultation between Signatories. The main partners were: World Bank, USAID, DANIDA, JICA, the German Development Cooperation and the UN Agencies. The SWAp coordination structures included the Joint Interagency Coordinating Committee (JICC) at the Ministers’ level, the Health Sector Coordinating Committee (HSCC) at the Permanent Secretaries’ level, and the HSCC Steering Committee (HSCC‐SC). The day‐to‐day work was coordinated by a SWAP Secretariat supported by different Interagency Coordination Committees (ICCs) established as technical‐level and systems‐support forums for discussions of programs in various areas including: Procurement and Supply Chain Management ICC, Community Services ICC, Human Resources for Health ICC, Health Sector Financing ICC, and Public‐Private Partnership ICC. Under the framework partners were to pool funding using any of the five JFA baskets: Health Sector Support Fund, Essential Medicines and Medical Supplies, Human Resources for Health, Capacity and Systems Development, and the Hospital Management Services Fund. While several partners signed the code of conduct, only the World Bank and DANIDA signed the JFA. The project was meant to support two out of the five JFA baskets: Health Sector Support Fund (HSSF) and for Essential Medicines and Medical Supplies. USAID on the hand had committed to: (i) strengthening of KEMSA, through: (a) leadership, management and governance, (b) infrastructure and systems, including LMIS, (c) technical capacity for commodity management and coordination with programs, (d) operational financing for distribution and warehousing; (ii) strengthening of MOH commodity management, through (a) technical capacity at national level, (b) coordination for commodity management at national level, (c) technical capacity at provincial (supervision), district (supervision and management), and facility level (management); and (iii) strengthened pharmaceutical regulation. Apart from the HSSF and essential medicines and medical supplies baskets, the other baskets: human resources for health, capacity and systems development, and the hospital management services fund did not materialize as originally envisaged. With the onset of devolution DANIDA also pulled out of the HSSF leaving the Bank as the sole partner financing the HSSF. Many of the other SWAp structures were equally affected and could not operate as previously envisioned.


45

ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION

A. TASK TEAM MEMBERS

Name Role

Preparation

Supervision/ICR

Gandham N.V. Ramana, Jane Chuma Task Team Leader(s)

Grace Nakuya Musoke Munanura, Dahir Elmi Warsame, Joel Buku Munyori

Procurement Specialist(s)

Josphine Kabura Kamau Financial Management Specialist

Lucie Muchekehu Team Member

Eva K. Ngegba Team Member

Suryanarayana Satish Social Safeguards Specialist

Yi‐Kyoung Lee Team Member

Kimberly Vilar Team Member

Joyce Cheruto Bett Team Member

Racheal Njeri Mwaura Team Member

Ocheng Kenneth Kaunda Odek Team Member

Toni Lee Kuguru Team Member

Angelina Darini Musera Team Member

Lilian Wambui Kahindo Social Safeguards Specialist

Diana Jemutai Sirma Environmental Safeguards Specialist


46

B. STAFF TIME AND COST

Stage of Project Cycle Staff Time and Cost

No. of staff weeks US$ (including travel and consultant costs)

Preparation

FY04 0 11,060.00

FY05 13.425 77,635.56

FY06 67.885 396,156.72

FY07 87.232 430,310.45

FY08 94.990 494,722.31

FY09 43.255 202,695.43

FY10 35.847 220,508.91

FY11 0 2,980.47

FY12 0 7,500.00

Total 342.63 1,843,569.85 Supervision/ICR

FY08 0 0.00

FY11 37.997 214,688.28

FY12 33.496 245,494.34

FY13 21.613 250,544.53

FY14 54.278 344,981.42

FY15 63.625 450,324.13

FY16 33.952 314,242.50

FY17 35.156 220,074.24

FY18 29.833 312,184.22

FY19 10.150 71,422.23

Total 320.10 2,423,955.89


47

ANNEX 3. PROJECT COST BY COMPONENT

Components Amount at Approval

(US$M) Actual at Project Closing (US$M)

Percentage of Approval (US$M)

Effective and transparent implementation of the Kenya Essential Package for Health (KEPH) through HSSF grants and performance streng thening.

46 109.66 237

Availability of essential health commodities and supply chain management reform

54 98.10 181

Total 100.00 207.76 207


48

ANNEX 4. EFFICIENCY ANALYSIS

Implementation Efficiency 1. Essential Package of Cost‐Effective Interventions. The project was designed to support implementation of the KEPH, a package consisting of cost‐effective interventions aimed at addressing the major causes of disease burden in Kenya. The interventions were consistent with Kenya’s disease burden and the World Health Organization recommended list of cost‐effective interventions. In addition, the

project was later restructured to provide nutrition supplements to populations affected by drought,

focusing on children under five years, pregnant and lactating women. While provision of food supplements is generally considered less cost‐effective and unsustainable, their provision is often justified during emergencies as part of the humanitarian response, which was the case with the project. Finally, the KEPH was designed as a national package, and as such, did not specify the category of services to be delivered by the different HF levels. 2. Equity. The project was designed to support service provision at Levels 2 and 3 HFs, where the majority of the poor received care, but did not specify the actual target beneficiaries. This was later

rectified when the project implementation was restructured to focus on children under five and women

as well as the drought affected populations. Furthermore, through the National Hospital Insurance Fund, the project supported the HISP, the government’s flagship program under Vision 2030 to provide the poor and vulnerable households with health insurance cover reaching 181,315 of the 181,968 targeted households, representing about 50 percent of orphan and vulnerable children (OVC) households countrywide. 3. Use of existing systems. The project was designed to operate under a SWAp and to use existing

national health systems including both public and private providers. Unfortunately, the SWAp did not materialize as originally envisaged. DANIDA and the Bank were the two partners to pool funds under the HSSF window. The arrangement however was short‐lived as DANIDA pulled out in 2014. Similarly, the initiatives to promote public private partnership did not take off and as such faith‐based providers were not brought on board. 4. Significant Implementation Delays. The project experienced significant disbursement delays of

funds. While was most pronounced at the health facilities, especially after the advent of devolution, all other entities including KEMSA, KMTC, NHIF, AMREF and Kenya School of Government suffered major disbursement delays. The implementation of activities by these entities not only started late due to delay with signing memoranda of understanding, implementation was further slowed down by the delayed disbursements by the MoH. After devolution, the Counties through the Council of Governors, took a long time in the 2013/14 and 2014/15 financial years before signing the memorandum of understanding (MOU) for the RBF and KHSSP conditional grants. These adversely affected the release of grants and led to non‐implementation of planned activities by the health facilities. While the project closing date was extended twice, implementation of the RBF continued to be marred by challenges, which led to reallocation of a significant share of funding ($44 million) originally earmarked for RBF for procurement of essential equipment to fill critical gaps in the delivery of primary health services and nutrition commodities to respond to the protracted drought.


49

5. HSSF and RBF Implementation Challenges. The two reforms were the hallmarks of the project. The

devolution process greatly impacted implementation of the HSSF and RBF causing significant delays of between 3‐6 months in the release of the RBF and conditional grants to the facilities. Attempts to ensure the transfers were based on performance were unsuccessful. The introduction of devolution also disrupted the supply of medicines. Counties were no longer obliged to source medicines from the government‐run KEMSA, which led to mismanagement and scarcity of drugs at health facilities. Cost benefit analysis 6. Project specific economic evaluation was not undertaken for the project during appraisal. The ICRR however includes an economic analysis at the end of the project to estimate the extent to which the project was able to yield economic and financial returns to Kenya by supporting the delivery of the basic package of essential health services. Since the major focus of the project was maternal health, the analysis assessed the extent to which the project could potentially have averted maternal deaths over its implementation duration. The interventions related to management and availability of essential medicines and supplies were assessed among the inputs towards reducing maternal mortality. Reduction in child mortality was excluded because the project’s contribution was considered minimal. Similarly, the provision of food supplements was excluded from the analysis for lack of data and the fact the rationale for the provision of supplements was mainly based on humanitarian concerns. The assumptions guiding the analysis are included in the Table below.

Assumptions Guiding the CBA

1. Discount rate 3% 2 Sensitivity analysis (discount rate) 5%

3 constant relative rate of reduction of

maternal mortality per year 7%

4 Probability of deliveries with twins live births

14/1000

5 Annual per capita for the bottom 50%

US$1200

6 Real wage growth 5.85%

7. To estimate the number of deaths averted, the analysis compared maternal deaths between 2010 and 2018. The base case scenario of the number of maternal deaths without new interventions was compared against number of deaths upon instituting new interventions. Without any new interventions, it is estimated that about 64,096 maternal deaths would have occurred between 2010 and 2018. However, with the new interventions implemented since 2010, it is estimated that only 48,625 maternal deaths could have occurred during the period thus saving 15,470 women. This is however only partially attributable to the project, given other financing mechanisms including from government and other donors. Attribution was estimated at 7.1 percent, or the project’s share in total public reproductive and maternal health expenditures as per 2016 national health accounts estimate.

8. Lives saved from these interventions were put into monetary value to be able to compare to total project cost and arrive at an internal rate of return and a benefit‐cost ratio. The average age of women giving birth in Kenya is 22 years. Women who survived were expected to earn an annual income of USD


50

1,200 which is the average income of the bottom 50 percent which is associated to the project target group rather than per capita GDP or GNI. Real wage growth was estimated to be 5.85 percent annually, or at the rate of long‐term economic growth. Furthermore, it is about half the value of the official minimum hourly wage at 1.15 (which would result in an annual income of USD 2,208). Women are expected to earn this income until their life expectancy of 67 years. This figure was discounted by 3 percent to arrive at a net present value.

9. Given the estimated net cash flow in terms of income of lives of mothers whose lives would have been saved, the estimated internal rate of return stands at 9.9 percent. At the given discount rate of 3

percent, the net present value of the total investment equals to USD $60,912,114.67 or a benefit cost

ratio of 1.74:1. Sensitivity analysis was conducted by elevating the discount rate even further up to 5 percent, which still yields a positive return at USD$3,410,258.54 or a benefit cost ratio of 1.04:1. Sustainability 10. The Project was executed using country systems, did not finance new health infrastructure outside

the existing health facilities and focused on building institutional capacity at county and facility levels. Most staff (technical officers in financial management and HFMCs) trained under the project continue to serve in their respective capacities. With the systems for procurement and supply chain management using automated Enterprise Resource Planning Program, KEMSA has become more successful as a public‐sector procurement agency. The devolution disrupted the RBF and HSSF fund flow arrangements, but created a new mechanism for intergovernmental fiscal transfers, which provides a good opportunity for government and partners to make direct transfer of funds to health facilities. It was noted that the government and partners are continuing to engage in discussions on how best to support the health sector in the current decentralized context. 11. GoK has substantially increased funding to the health sector to support the flagship programs under the UHC agenda. This is reflected in the share of the budget allocated to health by the Counties, which has increased from an average of 13 percent in FY2013/14 to 25.2 percent in FY2014/1537. Building

on the project, the Bank prepared a follow‐on project “Transforming Health Systems for Universal Care Project” (P152394, $191.1 m) to continue support to the Kenya Health system. To obtain funding from the new Project, counties are required to allocate at least 20 percent of the county budget to health and maintain sustained increases in budgetary allocations to the health sector in the subsequent years.

37 MoH. 2017. 2016/2017 National and County Health Budget Analysis Report.


51

ANNEX 5. BORROWER, CO‐FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS

The Client’s Report

Project Development Objectives

1. The objective of the project was to improve: (i) the delivery of essential health services for Kenyans, especially the poor; and (ii) the effectiveness of planning, financing, procurement and logistics of pharmaceuticals and medical supplies. Key Expected Outcomes and Outcome Indicators 2. The project identified seven key performance/outcome indicators and five intermediate outcome (IO) indicators. The key project indicators were: (i) direct project beneficiaries (number) of which female

(percent); (ii) health facilities (level 2‐4) without one of 14 tracer drugs at any point between delivery intervals (percent); (iii) children immunized (number, disaggregated for Northeast Province); (iv) HSSF facilities meeting core FM requirements of the fund (percent); (v) timely payment to suppliers by KEMSA, based on agreed procurement contracts; and (vi) facilities displaying quarterly information on funds received and availability of drugs at facility level (levels 2 and 3) (percent). The expected outcomes were primarily improved maternal, neonatal and child care as well as health systems strengthening. Project Components 3. The project consisted of two components:

a) Effective and transparent implementation of the Kenya Essential Package for Health (KEPH) through HSSF grants (US$32.6 million) and performance strengthening (US$11.8 million) totaling US$44.4 million for Component 1; main activities supported were generating and providing sufficient resources for implementing each facility’s AOP to address preventive, promotive and curative services at levels 1,2 and 3, and to account for them in an efficient, and transparent manner according to current government systems; supporting capacity building in the management of health facilities; Strengthening voice and accountability, linked especially to the HSSF; Supporting the KEPH through the phased implementation of health financing reforms through selective pilots, capacity building and evaluations; Supporting the KEPH referral system; Strengthening environmental standards through support to waste management; and Strengthening sector governance structures through support to the SWAp Secretariat.

b) Availability of essential health commodities US$48 million and supply chain management reform US$7.6 million totaling US$55.6 million for component 2. Main activities were to support implementation of initiatives to strengthen the Supply Chain Management for Essential Health Commodities through (i) increased funding for essential commodities and (ii) supporting reforms in KEMSA.

Status of implementation at completion


52

4. Most of the scheduled activities had been completed by June 2018 and no new activities have since been initiated. The Ministry has focused on finalizing payments and continues to implement priority activities using government resources as the project objectives remain relevant to date. The following key

outputs have been attained: Primary health facilities receiving HSSF grants on time, Facility management

committees in place and meeting regularly, Health personnel trained, delivery of nutrition services to malnourished children and pregnant mothers, facilities implementing RBF initiatives, poor households enrolled for health insurance subsidies, timely reimbursement to KEMSA documented claims by MOH, on time delivery of drugs and medical supplies to primary health care facilities and reforms in KEMSA. Achievement of Project Development Objectives and Analysis of the Results Framework 5. 6 out of the 7 Project Development Objective indicators surpassed the end‐of‐target and 4 out of 5 Intermediate objective indicators surpassed the targets. Two indicators that did not achieve target: i)

number of children immunized; and ii) Facility management committees having quarterly meetings with

minutes of meetings (levels 2 and 3) (percent) never achieved the target due to issues relating to governance and management in a devolved system resulting inequities in access and distribution. The detailed Results Framework (RF) is presented separately below.

Project Development Objective Indicators

Indicator Name Baseline 2013‐2014 2014‐2015

2015‐2016 2016–2017

2017–2018

End Target

Direct Project beneficiaries

25,896,000 38,327,054

39,499,288

34,670,439 44,275,435

50,949,459

35,000,000

(a) Female beneficiaries

14,229,369 21,263,422

22,076,154

24,065,319 18,356,514

34,447,148

17,500,100

(b) Beneficiaries from drought affected areas

0 7,820,890 5,806,659

9,359,722

14, 365,881

15,088,032

5,500,000

Children Fully immunized

1,188,698 1,090,751 1,110,212

1,142,374 1,107,604

1,105,234

1,298,058

Value order fill rate of orders for commodities from Primary Health Care facilities received by KEMSA

73% 95% 87% 90% 87% 83% 80%

Severely malnourished children under five receiving treatment recovered

60% 79% 67% 58% 56%

84% 60%

Share of planned procurement from EMMS pool completed by KEMSA in the financial

0% 75% 100% 100% 100% 60%


53

year as per GoK guidelines

Women receiving free delivery services meeting the quality norms at public health facilities

453,857 630,355 717,442 784,846 670,575

604,521

550,000


0 Indicator introduced in 2014

21,546 158,925

181,968 30,000

Key Intermediate Results Indicators

Indicator Name Baseline 2013/2014 2014‐2015

2015‐2016 2016 –2017

2017–2018

End Target

People with access to a basic package of health, nutrition, or reproductive health services

0 38,935,800

41,800,000

41,800,000 47,114,397

47,848,479

39,495,000

Facility management committees having quarterly meetings with minutes of meetings

27% 92% 90.1% 90% 88%

83% 90%

Health personnel receiving training (number)

0 9,884 10,431 15,284 18,655

19,271

14,899

Level 2‐3 facilities implementing results‐based financing initiatives

0 89 89 1,627 1,770

1,990 1,331

Beneficiaries receiving supplementary feeding in drought affected areas.

157,413 337,346 450,582 Support on this ceased

Support on this ceased

Support on this ceased

285,210

Counties placing orders for Essential Medicines and Medical supplies

16 16 47 47 47 47 30

KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME

A. KEY FACTORS DURING PREPARATION

6. The project set realistic objectives for improving maternal, neonatal and children services along with

health systems strengthening and improving planning, financing and procurement of pharmaceutical &

medical supplies aligned with the national priorities. This was fully aligned with the MOH’s strategic plans (NHSSP II) as well as the sectoral context during appraisal which ensured smooth implementation of the


54

PDO. The clear and compelling rationale for Bank involvement was adequately reflected in the Project Appraisal Document. The project used national systems for monitoring, which ensured regular updates and sustainability.

B. KEY FACTORS DURING IMPLEMENTATION

7. Several factors affected project implementation. These factors are sub‐divided into five main

sections: (i) factors that positively affected project implementation; (ii) factors within MOH control that

adversely affected implementation; (iii) factors beyond MOH’s control; and (iv) factors beyond government’s control. 8. Factors that positively affected project implementation:

a. The domiciling of project implementation responsibility under the HSSF/SWAp Secretariat helped in mainstreaming administrative and coordination processes.

b. The deployment of a project coordinator and other key staff (which was a legal covenant), ensured adequate attention, administration, supervision and monitoring of the project;

c. The Bank’s independent audits helped establish credibility and ensure greater accountability, while Bank support for fiduciary functions helped strengthen capacity;

d. MOH’s proactive disclosure of information (for example, disclosing the names of the people who received scholarships and naming the suppliers providing defective equipment) promoted transparency and accountability;

e. The project financed procurement of vehicles and medical equipment along with training of relevant technicians. This greatly benefitted the project as it avoided the need for the equipment to be transported to the manufacturer/supplier and then taken back after repair;

f. Use of SWAp coordination mechanisms, particularly the thematic technical working groups, ensured greater collaboration amongst the stakeholders, avoided duplication of interventions, resolved implementation bottlenecks, and promoted greater harmonization;

g. Recruitment of HSSF and KHSSP accountants increased accountability for funds reaching the primary health facilities and the District Health Management Team;

h. The bi‐annual review missions carried out by the Bank jointly with the MOH, helped to establish credibility, enhance accountability and address any implementation gaps and issues at an early stage, while Bank support for fiduciary functions helped strengthen capacity;

i. The direct cash transfers to level 2 and 3 primary health facilities through the Health Sector Services Fund (HSSF) for employment of casual workers (clerks, guards, etc.), operational costs, and the maintenance and improvement (e.g. small repairs), and the provision of EMMs through the “drawing rights” system (leading to full “pull” system for EMMS) helped significantly to improve overall functionality, general cleanliness and service delivery at the facilities;

j. The active involvement of the beneficiary communities through quarterly HFMCs meetings and annual meeting of stakeholders to review past performance, identify and analyze current problems, identify priorities, and plan (i.e. AOP) for outputs and activities to be undertaken during the next financial year ensured project ownership (making sure the project delivered on actual needs of the communities) and helped monitor progress;


55

k. Citizen engagement through committees linked to Health Facilities ensured ownership (making sure the project delivered on actual needs of the communities) and helped monitor progress;

l. The project’s social accountability interventions and MOH’s proactive disclosure of information (such as the requirement for health facilities to display quarterly information on funds received and availability of drugs at facility level, i.e. levels 2 and 3 helped in promoting transparency and accountability;

m. Efficient and prudent use of project proceeds. Cost savings realized from project activities, especially at KEMSA, were reallocated to other critical and relevant project activities, for example, the reallocation of funds for hire of a pre‐shipment inspection firm for quality assurance testing and development of a Marketing Plan for Supplementary Services Division (SSD) (which was developed with the support of USAID);

9. Factors within MOH control that adversely affected implementation:

a. (a) inadequate planning for project activities, which led to delays and cost overruns, eventually resulting to the project not disbursing fully

b. (b) Weak plans for procurement, due to inadequate specifications and insufficient participation of the user departments meant that equipment supplied to some of the facilities were not needed. Due to limited capacity in the MOH, particularly for preparing comprehensive cost estimates some of the supplies had awards given despite the contract price being high. The lack of a proper distribution plan as well as the need to do pre‐installation work at the facilities meant distribution and commissioning was almost always delayed and worsened by the weak coordination in the devolved system

c. (c) Greater efforts were needed to strengthen the health facility inventory systems, train staff in the use of new equipment procured, and to continue the mentorship program of health workers in the areas of Financial management

d. (d) Frequent change for social safeguards officer in the project led to limited implementation of the VMPP

e. (e) Absence of a communication officer in the Ministry/Project led to limited updates to the public about the project

f. (f) Implementation of the two components was particularly hampered by devolution of the health system leading to delays in funds transfer to health facilities and stock‐out of essential medicines and commodities

g. (g) Inadequate capacity and under‐staffing for the unit coordinating KHSSP procurements for KHSSP caused the slow start to the rolling out of the HSSF, late disbursement of Project funds and lengthy delays in the implementation of the planned procurement activities, especially at the start of the project and

h. (h) Inadequate management and coordination of the M&E functions as the hiring of a Technical Assistance to strengthen M&E capacity of KHSS Project was delayed and later dropped.

10. Factors beyond MOH’s control: Lack of collaboration between MoH and Council of Governors resulted to reduced allocation and transfer of funds to County Governments as accountability and utilization of funds could not be assured.


56

11. Factors beyond government’s control: unusual rain patterns disrupted implementation and delayed

completion of some activities e.g. surveys and distribution of equipment that resulted to contracts being amended. EVALUATION OF PERFORMANCE AND COMPLIANCE ISSUES COMPONENT 1: EFFECTIVE AND TRANSPARENT IMPLEMENTATION OF THE KENYA ESSENTIAL PACKAGE FOR HEALTH (KEPH) THROUGH HSSF GRANTS AND PERFORMANCE STRENGTHENING (US$44.4 MILLION). 12. Health Sector Services Fund: The Project supported the effective implementation of the KEPH, through financing the HSSF targeted at health service delivery levels 1, 2 and 3. The Government has

established the HSSF as part of its efforts to delegate power and responsibility for health service provision to the community, as envisaged by the NHSSP II. The HSSF improved the delivery of quality essential services, especially at the sub‐district and community levels, in an equitable and efficient manner, through: (i) generating and providing sufficient resources for implementing each facility’s AOP to address preventive, promotive and curative services at levels 1,2 and 3, and to account for them in an efficient, and transparent manner according to the current government systems; and (ii) supporting capacity building in the management of health facilities. 13. The Results Based Financing was implemented in 20 ASAL Counties plus Migori. Overall, Counties

reported an increase in utilization of the 10 key maternal and child health services with progressive improvement in quality of health services. Further, the project procured medical equipment to ensure Health Facilities are ready to provide the essential health services as per the KEPH. 14. Through additional financing and project restructuring, the project procured nutritional commodities that reduced significantly the effects of drought through prevention and management of malnutrition. The support provided since 2013 has significantly averted child deaths in the affected areas. 15. HSSF and Health Sector Governance and Stewardship: The two health ministries were strengthened

selectively in some of their main governance and stewardship functions, especially at the district level, to enhance service delivery and ensure a strong poverty focus. This was done in conjunction with other DPs through: 16. Strengthening voice and accountability, linked especially to the HSSF. This included various activities,

such as information dissemination and use of report cards; the close monitoring of the demand side

indicators in the results framework, including for the performance of the KEMSA; the establishment of effective complaints mechanisms; and independent external monitoring and auditing of performance and results. An independent integrated fiduciary implementation review agent to check compliance with procedures, monitor procurement and carry out performance audits throughout the project life mitigated several fiduciary risks especially where internal controls lacked and the high turnover of government officers where handover was not properly done. 17. Supporting the KEPH through (i) The phased implementation of health financing reforms through piloting and scale‐up of Health Insurance Subsidy Program for the Poor that resulted to at least 177,815 Households accessing outpatient and inpatient services in accredited public and private health providers. In addition, pre‐screening for various ailments was introduced as a result of the support and is being applied in all insurance schemes provided by the NHIF to ‐date; (ii) selected capacity building and


57

evaluations related to: (a) training for Kenya Enrolled Community Health Nursing that provided full

scholarship for 400 students from ASAL counties and marginalized populations to address critical service delivery gap; (b) training of Health Facility Management Committees initially planned for in 47 Counties but due to logistical challenges and budget constraints, the project focused on the 21 RBF Counties that saw an increased knowledge and skills on various areas and processes in running of a Health Facility in the selected Counties; and (c) Capacity building of health financing institutions in the provision of Financial Management course in the wake of devolved functions. This enabled for better accountability and transparency in the administration of resources in the Health Sector. These also included activities related to analytical studies to support the detailed planning and initial implementation of the health financing strategy as well as the collection of data on expenditure to inform utilization of health services. COMPONENT II: AVAILABILITY OF ESSENTIAL HEALTH COMMODITIES AND SUPPLY CHAIN MANAGEMENT REFORM (US$55.6 MILLION) 18. This component was implemented in parallel with the HSSF. Generally, there was improved

availability of commodities at the lower levels of the public health system. This further enhanced the quality of care provided as frequent shortages and stock‐outs have been minimized. The constraints related to flow of funds from MoH have reduced significantly with KEMSA being capitalized at over Kshs 6 Billion. The order fill rate received from counties constantly remained above 85% and all 47 Counties are now using Logistic Management Information System to place orders on‐line. The order turnaround time for primary health care facilities is now around 8 days. 19. KEMSA has sustained key reforms implemented as part of KHSSP. The KEMSA Board continues to have independent members and institutional integrity mechanisms put in place. To support the procurement of EMMS commodities and the reform process, the Project will further strengthen KEMSA and the Ministry of Health in the key functions of planning, procuring and managing critical commodities, in conjunction with other DPs. 20. Environmental: The project was classified as environment category “B” and it triggered one of the World Bank’s Operational Policies, 4.01 relating to Environmental Assessment due to the potential impact caused by medical waste. The project complied with the World Bank’s environmental and social

safeguards requirements. The MOH prepared an HCWM plan to guide health facilities to manage medical

waste effectively. While most of the health facilities segregated medical waste using color‐coded bins as required, final disposal was mainly carried out using open burning by the health centers as incinerators are only located in the hospitals. However, the inadequate capacity at the health facilities means the HCWM plan is not being adhered to. More support is needed to ensure the HCWM plan is adequately implemented and monitored. 21. Social accountability: Client charters were developed and translation into Kiswahili and local vernacular done. The printing of the service charters by the graphic designer was not done due to long procurement procedures against limited time. Guidance notes on translating client charters were issued to county VMG focal persons, the health facility in charges and the primary school teachers from adjacent

schools who use the language as they teach. Focal persons were invited for client charters who coordinated these activities conducted in three concurrent workshops held in Nakuru, Kilifi and Marsabit. Besides, civil society organizations were engaged to strengthen citizen participation and engagement as they could also participate in activities like RBF community verification of results. The monthly health


58

facility meeting of HFMC were also used for grievance redressal as suggestion boxes contents could be checked and discussed. In some HFs the MOH set‐up a helpdesk (customer Service desk) for Triage which have also capacity to serve as a “call center” for grievance redressal and client referral services. At present, the system has not developed adequate capability to record complaints and direct them to the relevant people and do not regularly inform the aggrieved of the action(s) taken. 22. Financial management (FM): Quarterly interim unaudited FM reports of acceptable quality were submitted in a timely manner. The external audit reports, to a large extent, have been clean. However, there were delays in processing payments mainly due to under‐staffing of the MOH’s accounting unit. Per agreements under the project, annual audits were undertaken by the Office of the Auditor General (OAG). Largely, the audit reports related to the HSSF have been questionable as potential ineligibles and disclaimer of opinion have been expressed. This mostly arising due to lack of human capacity by the office of the Auditor General to carry out audits in the Counties, while expecting MOH to have all the returns in the wake of devolved health functions and compounded further by the lack of HSSF accountants to

provide returns as most of them had either been absorbed by the Counties or terminated after devolution. Under RBF, disbursements to counties were based on County Allocation of Revenue Act (CARA) provisions rather than performance. The initial disbursement to counties in February 2016 for RBF was based on estimates. Subsequent disbursements were to be based on output; however, the MOH was not able to do this as it was considered non‐compliance with the CARA. The NHIF implementing the HISP program failed to share quarterly reports with MoH as required by the program’s monitoring and evaluation framework and plan. This ultimately affected transfer of funds to the institution. The NHIF should reconcile the HISP claims payment data per the operations cash book and the utilization data. The payments for all the inpatient and special benefits package claims should be made and accounted for through the HISP operations account. Issues raised in audit reports and other reviews were addressed especially ineligible expenditure amounting to KShs 229,785,262 and potentially ineligible expenditure of Kshs 1,166,120,469. Further, MoH was unable to substantiate and thus refunded the US$ 802,888.37 being outstanding undocumented advances under IDA Credit 4771.

23. Procurement: There were overall delays in procurement throughout the life of the project. By the end of the project, all procurement contracts signed both goods and consultancies were fully executed despite the MOH having to amend several contracts towards the project closure date as a result of challenges of either failure to deliver consignment to final destinations due to flooding in hard to reach areas or delayed manufacturing that would result in equipment reaching final destinations after project closure date. The MOH has ensured the quality of goods received. LESSONS 24. The project provides important lessons that are considered good practices, most of which have been incorporated in follow‐on operations:

a. Ensure use of SMART indicators for which regular updates are available. Wherever possible, the RF should include indicators that are already being tracked by the government to avoid the need to rely on additional player(s) for reporting on project’s outcomes. These help in project monitoring and decision‐making.


59

b. Continuity and in‐country presence of the task team aids project implementation and ensures supportive supervision. The World Bank’s implementation support used independent validation, which ensured project efficiency and helped the MOH be on track.

c. Embedding new initiatives within the existing government structure, helps ensure continuity and broad‐based ownership. In the case of HISP, the project helped advocate for allocation of funds under the UHC umbrella through the National Treasury after the closure of the project.

RECOMMENDATIONS 25. Effective planning and ensuring adequate capacity of the implementation unit to deliver on project activities early on can accelerate implementation. During fiduciary assessment of the project, attention should be paid to the client’s performance and capabilities in implementing the activities and interventions and include appropriate mitigation in the project design, for example, provision of technical support early on to help with planning, design and realistic costing. Similarly, focal persons should be identified early, and adequate capacity put in place to deal with social and environmental issues.

26. When significant investments in equipment are made, adequate budgetary provisions—from

government—should be made available for maintenance and incremental costs. Similarly, government should make provisions for the required HR to be trained in handling the equipment. This would ensure optimal use as well as longer‐term sustainability and continue with development impacts of project interventions. 27. Subsequent projects should collaborate and work closely with the Council of Governors since the health sector is devolved and implementation of the KEPH is mainly by the county Governments. ARRANGEMENTS FOR SUSTAINABILITY OF RESULTS 28. The preparation, design and implementation of the new project with related development objectives

that have potential to propagate the roll out of the best practices e.g. THS‐UCP is using the Performance

Based Financing approach to allocate resources to County Governments. 29. Riding and use of established GOK institutions to run components of the program like capacity building in Financial Management done by Kenya School of Government, Enrolled Community Nursing by KMTC has ensured continuity in provision of the curriculum and courses offered by the institutions to date. 30. The health interventions financed by the project are the Government of Kenya priorities which have

some existing budgetary support e.g. Health Insurance Subsidy Program for the Poor.

31. The use of the in‐Country systems and staff for implementation and monitoring of the project

enables for system strengthening and capacity building e.g. hosting of the new innovations in existing established portal e.g. the Web portal for RBF in the MOH DHIS and use of same staff for managing. 32. One of the Health Financing Strategies being employed under strategic purchasing towards achieving universal health care is Results Based Financing. This strategy will improve the quality of health services provided.


60

The Government Comments on the ICRR

1. The GoK appreciates the assessment review of the extent of execution of the core reforms

supported by the project.

2. The ICRR rating of Moderately unsatisfactory is unacceptable for a project whose performance

throughout the course of its implementation has mostly been satisfactory / moderately

satisfactory. Further, the banks performance rating is satisfactory while M&E quality is modest.

How is that possible? The Project disbursed up to 98%. How is the ICRR rating determined? The

Governments view is that the rating is not consistent, it’s unfair despite most results having been

achieved as per the Results Framework.

3. The Project was implemented in a devolved system. What was the effect of devolution on the

rating, performance etc.? An analysis of these should be provided to GoK.

4. Use of the IIFRA findings being limited is agreeable especially in the devolved Health System.

However, the IIFRA only towards the end of the project closure did exit meetings with County

Governments.

5. Involvement of Council of Governors was not as effective as the County Government

representatives then did not appreciate the activities implemented under the Project which was

ongoing. The reason being the Project was designed and Implemented before devolution hence

County Engagement was minimal.

6. GoK/MoH team in force at closure could be annexed

7. Replace KEHP with KEPH (Kenya Essential Package for Health) in all areas in the document.

8. Elaborate further on the effect of data quality issues and lack of data on the cost benefit analysis

9. Considering the initiatives undertaken during the period were not able to positively impact on the

per capita utilization rate, what measures were instituted to the follow‐on project (THS‐UCP) to

ensure that increased access and utilization is registered at population level (vis a vis reporting on

merely coverage of essential health services).


61

ANNEX 6. SUPPORTING DOCUMENTS (IF ANY)

document of the world bank...report no: icr00004624 implementation completion and results report...

Documents