document of the world bankdocuments.worldbank.org/curated/en/... · document of the world bank...

Document of The World Bank

Report No: ICR00003380

IMPLEMENTATION COMPLETION AND RESULTS REPORT (IBRD-75870)

ON A

LOAN

IN THE AMOUNT OF US$40 MILLION

TO THE

REPUBLIC OF PANAMA

FOR A

HEALTH EQUITY AND PERFORMANCE IMPROVEMENT PROJECT

June 16, 2015

Health, Nutrition and Population Global Practice Latin America and the Caribbean Region

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

ii

CURRENCY EQUIVALENTS (Exchange Rate Effective: June 16, 2015)

Currency Unit = Panamanian Balboas (PAB) PAB1.00 = US$1.00 US$1.00 = PAB1.00

FISCAL YEAR

April 1 – March 31

ABBREVIATIONS AND ACRONYMS

CCT Conditional Cash Transfer Program CODIPRO Project Executive Council (Consejo Directivo de Proyectos) DHS Demographic and Health Survey CPS Country Partnership Strategy EEC Health Coverage Expansion Strategy (Estrategia de Extensión de Cobertura) FMA Financial Management Assessment GOP Government of Panama HEPI Health Equity and Performance Improvement Project IAF Independent Audit Firm RDH Regional Department of Health IADB Inter-American Development Bank MEF Ministry of Economy and Finance MIDES Ministry of Social Development MHT Mobile health team MOH Ministry of Health NGO Non -Governmental Organization OE External Organizations (Organizaciones Externas) OI Internal Organizations (Organizaciones Internas) PAISS Basic Health Services Package (Paquete de Atención Integral de Servicios de Salud) PHC Primary Health Care PMTISS Multi-Phase Institutional Transformation Project (Proyecto Multifase de Transformación

Institucional) PSPV Health Protection Program for the Poor and Vulnerable Populations (Protección en Salud

para Poblaciones Vulnerables) RO Panama’s Conditional Cash Transfer (Red de Oportunidades) RBF Results-Based Financing RDH Regional Department of Health UGSAF Financial and Administrative Health Management Unit (Unidad de Gestión de Salud,

Administrativa y Financiera)

Senior Global Practice Director: Timothy Grant Evans

Practice Manager: Daniel Dulitzky

Project Team Leader: Carmen Carpio

ICR Team Leader: Renzo Sotomayor

ICR Author: Renzo Sotomayor

iii

PANAMA

HEALTH EQUITY AND PERFORMANCE IMPROVEMENT PROJECT

CONTENTS A. Basic Information .......................................................................................................... iv B. Key Dates ...................................................................................................................... iv C. Ratings Summary .......................................................................................................... iv D. Sector and Theme Codes................................................................................................ v E. Bank Staff ....................................................................................................................... v G. Ratings of Project Performance in ISRs ..................................................................... viii H. Restructuring (if any) .................................................................................................... ix I. Disbursement Profile ...................................................................................................... x 1. Project Context, Development Objectives and Design ............................................... 1 2. Key Factors Affecting Implementation and Outcomes .............................................. 7 3. Assessment of Outcomes .......................................................................................... 13 4. Assessment of Risk to Development Outcome ......................................................... 21 5. Assessment of Bank and Borrower Performance ..................................................... 22 6. Lessons Learned ........................................................................................................ 23 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners ........... 25 Annex 1. Project Costs and Financing .............................................................................. 26 Annex 2. Outputs by Component...................................................................................... 27 Annex 3. Economic and Financial Analysis ..................................................................... 28 Annex 4. Bank Lending and Implementation Support/Supervision Processes ................. 31 Annex 5. Beneficiary Survey Results ............................................................................... 33 Annex 6. Stakeholder Workshop Report and Results ....................................................... 34 Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR ......................... 35 Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders ........................... 51 Annex 9. List of Supporting Documents .......................................................................... 52 Annex 10. MAP IBRD 33462R ........................................................................................ 53

iv

A. Basic Information

Country: Panama Project Name: Health Equity and Performance Improvement Project

Project ID: P106445 L/C/TF Number(s): IBRD-75870

ICR Date: 06/22/2015 ICR Type: Core ICR

Lending Instrument: SIL Borrower: GOVERNMENT OF PANAMA

Original Total Commitment:

USD 40.00M Disbursed Amount: USD 40.00M

Revised Amount: USD 40.00M

Environmental Category: B

Implementing Agencies: MINSA

Cofinanciers and Other External Partners: B. Key Dates

Process Date Process Original Date Revised / Actual

Date(s)

Concept Review: 11/28/2007 Effectiveness: 12/08/2008 12/08/2008

Appraisal: 04/16/2008 Restructuring(s): 04/17/2012 10/23/2013

Approval: 08/05/2008 Mid-term Review: 09/05/2011 12/05/2011

Closing: 06/30/2013 12/31/2014 C. Ratings Summary C.1 Performance Rating by ICR

Outcomes: Moderately Satisfactory

Risk to Development Outcome: Low or Negligible

Bank Performance: Moderately Satisfactory

Borrower Performance: Moderately Satisfactory

C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings

Quality at Entry: Moderately Satisfactory Government: Moderately Satisfactory

Quality of Supervision: Moderately SatisfactoryImplementing Agency/Agencies:

Moderately Satisfactory

Overall Bank Performance:

Moderately SatisfactoryOverall Borrower Performance:


v

C.3 Quality at Entry and Implementation Performance IndicatorsImplementation

Performance Indicators

QAG Assessments (if any)

Rating

Potential Problem Project at any time (Yes/No):

No Quality at Entry (QEA):

None

Problem Project at any time (Yes/No):

No Quality of Supervision (QSA):

None

DO rating before Closing/Inactive status:


D. Sector and Theme Codes

Original Actual

Sector Code (as % of total Bank financing)

Health 5 1

Non-compulsory health finance 65 89

Public administration- Health 30 10

Theme Code (as % of total Bank financing)

Child health 22 22

Health system performance 22 22

Nutrition and food security 22 22

Population and reproductive health 22 22

Social Safety Nets/Social Assistance & Social Care Services

12 12

E. Bank Staff

Positions At ICR At Approval

Vice President: Jorge Familiar Calderon Pamela Cox

Country Director: J. Humberto Lopez Laura Frigenti

Practice Manager Daniel Dulitzky Keith E. Hansen

Project Team Leader: Carmen Carpio Fernando Montenegro Torres

ICR Team Leader: Renzo Efren Sotomayor Noel

ICR Primary Author: Renzo Efren Sotomayor Noel F. Results Framework Analysis

Project Development Objectives (from Project Appraisal Document) The Project Development Objectives of the proposed project are to: (i) increase access of targeted underserved rural communities to quality basic health services known to

vi

improve mother and child health; and (ii) support the development of strategic planning, regulatory, and monitoring mechanisms known to improve health system performance. Revised Project Development Objectives (as approved by original approving authority) The project development objectives were not revised. (a) PDO Indicator(s)

Indicator Baseline Value

Original Target Values (from

approval documents)

Formally Revised Target Values

Actual Value Achieved at

Completion or Target Years

Indicator 1 : Percentage of pregnant women with at least 3 prenatal controls (one in each trimester).

Value quantitative or Qualitative)

20% 70% 86.1%

Date achieved 06/27/2010 08/06/2008 12/18/2014 Comments (incl. % achievement)

Target surpassed. Baseline elaborated in June 2010 based on the EEC-PSPV beneficiary roster

Indicator 2 : Percentage of children below 1 year with a complete vaccination scheme for their age.


26% 95% 85% 96.1%

Date achieved 06/27/2010 08/06/2008 12/11/2012 12/18/2014 Comments (incl. % achievement)

Target surpassed. Baseline elaborated in June 2010 based on the EEC-PSPV beneficiary roster. Target was revised during the first Project restructuring in April 2012.

Indicator 3 : Percentage of women delivering children with the assistance of trained personnel from MOH.


6% 70% 65% 92.76%


Target surpassed. Baseline elaborated in June 2010 based on the EEC-PSPV beneficiary roster. Target was revised during the first Project restructuring in April 2012.

Indicator 4 : National strategy with baseline and targets on improving equity in access to health care services established using DHS data.


0

First report of monitoring progress completed.

TOR for DHS finalized.


Target not achieved. TOR finalized but DHS was not implemented due to delays with Project implementation and lack of funds.

vii



approval documents)

Formally Revised Target Values



Indicator 5 : Percentage of individuals from rural areas diagnosed with diabetes and hypertension in need of medication receiving prescribed drugs by primary health care units according to MINSA's protocol.


77% Diabetes and Hypertension

55% Diabetes 55% Hypertension

54.37% Diabetes 78% Hypertension


Target achieved. Baseline elaborated in August 2012 based on all patients receiving treatment regardless on whether the treatment is the one set in the MOH protocols.

(b) Intermediate Outcome Indicator(s)



approval documents)

Formally Revised

Target Values



Indicator 1 : Component 1: Percentage of health regions that achieve more than 80% of agreed intermediary outcome targets.

Value (quantitative or Qualitative)

0% 90% 70% 20%


Target not achieved. MOH decided to stop using agreed intermediary outcome targets. Baseline elaborated based on MOH administrative systems. Target was revised during the first Project restructuring in April 2012.

Indicator 2 : Component 2: Percentage of health regions that complete survey and mapping of human resources, equipment, and infrastructure.


0% 90% 100%


Target surpassed.

Indicator 3 : Component 2: Percentage of health regions that have successfully complied with accreditation plan of primary health care units in communities targeted by Component 1.


0% 80% 0%

Date achieved 08/06/2008 08/06/2008 12/18/2014 Comments (incl. %

Target not achieved since funding needed by each Regional Department of Health (RDH) to comply with accreditation was allocated based on achievement

viii



approval documents)

Formally Revised

Target Values



achievement) of IO Indicator 1.

Indicator 4 : Component 3: Establishment of a baseline on Access to primary health care services by the poor in rural areas using data from the demographic and health survey.


DHS never done in Panama.

DHS results disseminated.

TOR for DHS finalized.


Target not achieved. TOR finalized but DHS was not implemented due to lack of internal coordination and funding.

Indicator 5 : Component 3: Percentage of drugs from the Primary Health Care Level Essential Drug List that is available at health centers.


69% 80% 80%


Target achieved.

Indicator 6 : Component 3: Percentage of health regions using automated monitoring and information system for assessing achievement of results of primary health care providers.


0% 80% 100%


Target surpassed.

G. Ratings of Project Performance in ISRs

No. Date ISR Archived

DO IP Actual

Disbursements (USD millions)

1 12/05/2008 Satisfactory Satisfactory 0.00 2 06/19/2009 Satisfactory Satisfactory 3.63 3 12/02/2009 Satisfactory Satisfactory 6.91 4 06/28/2010 Satisfactory Satisfactory 6.91 5 02/19/2011 Satisfactory Moderately Satisfactory 12.43 6 06/29/2011 Satisfactory Moderately Satisfactory 13.88 7 12/26/2011 Satisfactory Moderately Satisfactory 18.37

ix

No. Date ISR Archived

DO IP Actual

Disbursements (USD millions)

8 07/11/2012 Satisfactory Moderately Satisfactory 20.47 9 02/26/2013 Satisfactory Moderately Satisfactory 22.97

10 09/30/2013 Satisfactory Moderately Satisfactory 29.26 11 04/21/2014 Satisfactory Moderately Satisfactory 31.36 12 11/24/2014 Moderately Satisfactory Moderately Satisfactory 37.30 13 12/29/2014 Moderately Satisfactory Moderately Satisfactory 40.00

H. Restructuring (if any)

Restructuring Date(s)

Board Approved

PDO Change

ISR Ratings at Restructuring

Amount Disbursed at

Restructuring in USD millions

Reason for Restructuring & Key Changes Made

DO IP

04/17/2012 N S MS 20.47

1. Extended the Closing Date from June 30, 2013 to December 31, 2014 to ensure provision of health services, completion of beneficiary roster, and conducting of health survey. 2. Simplified the disbursement conditions to trigger disbursements. 3. Revised the Results Framework for a more accurate measurement of progress. 4. Introduced flexibility in the percentage of funds financed by the loan versus counterpart funds while maintaining the global percentages.

10/23/2013 N S MS 29.26

1. Transferred funds from Components 2 and 3 to Component 1. Component 1 total funds increased from US$25.8 to US$35.6 million, representing 89 percent of the total Loan. The transfer enabled the financing of the provision of health services to the target population during 2013.

x

I. Disbursement Profile

1

1. Project Context, Development Objectives and Design 1.1 Context at Appraisal

1. Country and Sector background. At Project appraisal in 2008, Panama’s economy was one of the fastest growing in Latin America, with an average GDP growth of 8.7 percent from 2004-06, and more than 10 percent in the first nine months of 2007. With an open and expanding trade regime, the Government of Panama (GOP) in 2006 achieved a fiscal surplus in the non-financial public sector for the first time in ten years and was on track to reduce the overall deficit to 1 percent by 2010. Despite strong economic performance, poverty in Panama remained persistently high; with more than one third of Panama’s population living in poverty and more than one sixth in extreme poverty. The GOP decided to address this situation by working on a comprehensive plan to improve its conditional cash transfer (CCT) program known as Red de Oportunidades (RO). The RO, program promoted the coordination with key ministries, such as health and education, and boosted demand for basic health and education services. 2. Main Health Sector Issues. The main challenge faced by the health system in Panama was related to inequality. Panama achieved important health outcomes (under 5 mortality declined from 34 to 20 deaths per 1,000 births between 1990 and 2004) and devoted a substantial amount of its GDP to health expenditures (7.7 percent in 2004), well above the regional average of 6.5 percent. However, due to geographical, financial and cultural barriers to access key preventive services for mothers and children, poor and indigenous rural households experienced lower health outcomes. Immunization rates increased for the non-poor but decreased between 4 and 5 percentage points in poor households (except for the anti-tuberculosis vaccine BCG). Mortality rates for children under five years of age in Bocas del Toro and the indigenous areas (comarcas) Ngobe Bugle and Guna Yala were respectively 2.4, 1.9 and 1.6 times higher than the national average of 19.9 deaths per 1,000 live births. Maternal mortality in the comarcas Guna Yala and Ngobe Bugle were 5 and 3 times higher than the national average.

3. Since 1995, the Ministry of Health (MOH) had implemented a strategy to expand coverage of primary health care services and to reach the rural and indigenous communities - Health Coverage Expansion Strategy (Estrategia de Extensión de Cobertura - EEC). From 2003 to 2008, the strategy received financial and technical support through an Inter-American Development Bank (IADB) loan implemented through the Multi-Phase Institutional Transformation Project (Proyecto Multifase de Transformación Institucional – PMTISS). By means of this loan and public funds, the MOH delivered a basic package of 19 interventions to poor households and indigenous communities denominated the Basic Health Services Package (Paquete de Atención Integral de Servicios de Salud - PAISS). In support of delivering this package, the MOH introduced the use of payments on capitated basis, performance agreements with MOH teams, and contracts with non-governmental organizations (NGOs).

2

4. In 2007, the MOH decided to continue financing the EEC through two new programs. The first one to finance and reach rural communities: the Health Protection Program for the Poor and Vulnerable Populations (Protección en Salud para Poblaciones Vulnerables - PSPV); the second one which built on PAISS, but focused on scaling up nutrition services in the indigenous comarcas: Strengthened Basic Health Services Package (Paquete de Atención Integral de Servicios de Salud Fortalecido con Atención Integral de la Niñez en la Comunidad – PAISS+N).

5. The new PSPV received technical and financial support from the World Bank through the Health Equity and Performance Improvement Project (HEPI). Under the PSPV, and based on the previous five years’ experience implementing PAISS, the MOH decided to improve the capitation payments and include the results-based financing (RBF) approach. PSPV also included the sub national health levels –Regional Department of Health (RDH) - in the scheme in order to improve RDH responsiveness and to strengthen MOH stewardship. In addition, PSPV allowed contracts with private sector providers for the delivery of the health services package by mobile teams in the rural areas. 6. The new program targeting indigenous comarcas, PAISS+N, was supported by the IADB health project and the World Bank Social Protection Project1, which also supported the CCT RO program, which included the strengthening, design and delivery of preventive and nutritional services in indigenous jurisdictions. 7. Country Partnership Strategy and Rationale for the World Bank involvement. The GOP objectives for the health sector were fully aligned with the World Bank’s commitment to support poverty reduction and inclusion efforts within the framework of the World Bank Group’s Country Partnership Strategy (CPS) 2008-2010 (Report # 41338) discussed by the Executive Directors on October 30, 2007. The Project supported two of four CPS pillars: Pillar I - “Reduce Poverty and Inequality” and Pillar IV - “Develop Human Capital”. In addition, the Project was consistent with GOP’s “Strategic Vision of Economic and Employment Development Towards 2009” and World Bank’s 2007 Strategy for Health, Nutrition and Population.

8. The World Bank was particularly well-positioned to support Panama´s effort to launch the new PSPV, given its experience in supporting countries in the region to address inequality by using innovative financing mechanisms such as capitation payments and RBF mechanisms, and developing monitoring and evaluation systems to measure results among vulnerable populations (i.e. Argentina, Brazil, Mexico, and Uruguay). Consequently, PSPV was designed with features similar to other World Bank-financed projects that used provider incentives to foster accountability and RBF. Finally,

1 Panama Social Protection Project (Red de Oportunidades), Loan # 7479 (P098328).

3

the World Bank was already supporting EEC for delivering health services in the indigenous jurisdictions through another project: Social Protection Project (P098328).

1.2 Original Project Development Objectives (PDO) and Key Indicators (as

approved) 9. The PDO consisted of two sub-PDOs which were to: (i) increase access of targeted underserved rural communities to quality basic health services known to improve mother and child health; and (ii) support the development of strategic planning, regulatory, and monitoring mechanisms known to improve health system performance. 10. According to the PAD, the PDO indicators were:

1. Percentage of pregnant women with at least 3 prenatal controls with at least one control in each trimester of the pregnancy.

2. Percentage of children younger than 1 year with complete vaccination scheme. 3. Percentage of births attended by trained personnel from MOH. 4. Development of a national health strategy includes a baseline and targets to reduce

inequalities in access to health services constructed with data from the national and demographic health survey (DHS).

5. Percentage of individuals from rural areas diagnosed with diabetes and hypertension in need of medication receiving prescribed drugs by ambulatory primary health care units according to MOH’s protocol.

1.3 Revised PDO (as approved by original approving authority) and Key Indicators,

and reasons/justification. 11. The PDO was not revised. However, the Project’s Results Framework was revised during project restructuring (April 2012) allowing for a more accurate measurement of Project progress and impact. The revisions introduced were as follows: (i) modified some annual and final Project targets to align with the revised closing date (which was extended by 18 months to December 2014), (ii) revised the baseline of one indicator; and (iii) revised data sources and clarified the measurement for indicators 1, 2, 3 and 5. The Project restructuring did not exclude, add or change the indicators. 1.4 Main Beneficiaries 12. The project loan, specifically Component 1, would provide resources to finance the average units cost of ensuring access to basic health services to approximately 225,000 poor individuals living in rural areas. The Project did not include the communities living in the indigenous jurisdictions (comarcas indígenas) as they were already covered by other financial sources.

1.5 Original Components (as approved)

4

13. The Project consisted of the following three components: 14. Component 1: Ensuring access to primary health care services by the poor in rural areas (US$36.8 million – World Bank financing US$25.8 million). The objective of this component was to increase access by targeted underserved communities to basic health services known to improve mother and child health care.

15. The following table presents the package of prioritized health interventions that this component financed:

Table 1. Package of Basic Health Services Preventive Health Services

Prenatal control Post-partum control Provision of Folic Acid and Iron for women in fertile age Development and growth monitoring for children under 5 years old Family Planning Pap smear and breast cancer preventive examination

Health Care Services Acute Respiratory Infections (ARI) treatment Acute Infectious Diarrhea Screening, diagnosis and treatment of malnutrition for children under five Screening, diagnosis and treatment of malnutrition for women during pregnancy and post-partum Provision of supplementary food for women with low weight during pregnancy and post-partum Screening and treatment of micronutrient deficiency in pregnant women and children under 5 years old Triage and referral to appropriate health care unit for institutional delivery Screening and treatment of tuberculosis Screening, diagnosis and treatment of malaria Screening, diagnosis and treatment of leishmaniasis Screening, diagnosis and treatment of hypertension Ambulatory treatment and follow up of other adult chronic conditions and child morbidity (primary health care)

Education and Health Promotion Preventive nutritional education Education and training on evaluation of water sources, treatment and quality control Education and training on sanitation activities in rural areas Education and training for solid waste management and disposal in rural areas Health promotion

16. This component aimed to use capitation payments as a disbursement mechanism and finance health regions to implement PSPV. Loan proceeds for capitation payments were disbursed against a certified list of enrolled eligible individuals to be submitted to the Bank by the MOH. The certification of the list of beneficiaries was provided by the National Comptroller’s Office (Contraloría General de la Nación) and by an independent audit firm (IAF). The MOH in turn transferred loan proceeds to the Regional

5

Departments of Health (RDH), all of which have been decentralized, with capitation payments adjusted according to achievement of agreed-upon result targets. 17. Concurrent medical audits confirmed that: (i) services were in fact rendered to eligible individuals; (ii) services were rendered following established quality standards; and (iii) reported levels of target-achievement are verifiable and accurate. Social audits aimed at engaging with beneficiary communities to provide information for accountability mechanisms.

18. The package of health services was provided by mobile health teams. Mobile health teams could be: i) private sector providers contracted by MOH central level to deliver the health package, referred to as external organizations (organizaciones externas - OE), or ii) mobile health teams integrated by health professionals hired by the RDH, referred to as internal organizations (organizaciones internas - OI). 19. Component 2: Increase responsiveness of primary health services networks to vulnerable populations in rural areas (US$9.8 million – World Bank financing US$6.2 million). The objective of this component was to increase the capacity of primary health care networks to respond to health care service demand by poor rural communities. There were two subcomponents:

20. Subcomponent 2.1. Strengthening health regions’ management capacity to develop and implement primary health care interventions in rural areas. This subcomponent aimed at financing investments, consultancies and training for activities including: development of an accreditation system for primary health care facilities in rural areas; technical assistance to carry out assessments of available human resources, equipment and infrastructure; technical assistance to develop and implement annual operational plans at the regional level to improve the management of the supply chain of pharmaceuticals; technical assistance to develop and implement annual operational plans to reduce access barriers in rural areas; and activities to improve cultural appropriateness of health care delivery to rural communities.

21. Subcomponent 2.2. Improving the delivery of health services to vulnerable populations living in remote rural areas. This subcomponent aimed at financing investments, technical assistance and training for activities including: refurbishment of existing primary health care units in rural areas; acquisition of medical equipment and vehicles for primary health care units; development and initial implementation of a continuous training program for community health workers; development of strategies for early identification and transportation of critical-condition patients; implementation of a pilot program of telemedicine in primary health services in rural areas. 22. Component 3: Strengthening MOH’s institutional capacity to exercise stewardship in the health sector (US$11.6 million – World Bank financing US$7.0 million). This component aimed at strengthening the MOH’s institutional capacity for strategic planning, management of information and monitoring results to improve the

6

health system performance with a focus on fostering equitable access to quality health services. The component was structured under three subcomponents:

23. Subcomponent 3.1. Improving the MOH’s institutional capacity for strategic analysis and planning to improve performance in the health sector. Activities to be financed under this subcomponent included conducting national surveys and studies such as the first Demographic and Health Survey; technical assistance and training to improve the MOH’s technical capacity to analyze data and trends in the health sector; consultations for a new National Health Strategy to improve equitable access to health services; development of national pharmaceutical and nutrition strategies; and strengthening IT systems for data analysis and strategic planning.

24. Subcomponent 3.2. Update health regulations and revise enforcement mechanisms. This subcomponent aimed at financing technical assistance, goods, and services needed to overhaul health regulatory framework, including the revision and updating of the health code, and strengthening enforcement mechanisms.

25. Subcomponent 3.3. Strengthening management and monitoring, and information systems of national programs to improve access to health services by the poor. This subcomponent aimed at financing technical assistance, goods, and services to implement various activities, including strengthening information systems to monitor implementation of the new model for primary health care; information systems to monitor results and financing for the PSPV, including automated system at the central and regional level; concurrent financial, technical, and social audits; upgrading information technology at the regional level and training for data management; co-financing an impact evaluation of PSPV; and the management of fiduciary aspects of project implementation. 1.6 Revised Components

26. Components were not revised. 1.7 Other significant changes 27. Project restructuring. The Project went through two restructurings, both Level Two. The significant changes of the first restructuring (April 17, 2012) were:

The Closing Date was extended from June 30, 2013 to December 21, 2014 to ensure the complete implementation of the three Components.

Simplification of disbursement conditions related to the certification of the beneficiary roster that resolved initial disbursement delays.

Introducing flexibility in the percentage of funds financed by the loan versus counterpart funds for each disbursement request, while maintaining the global agreed upon percentages for all expenditures categories. Due to national budget norms, at the beginning of the year the MOH did not have enough counterpart funds for large disbursement.

7

28. The second restructuring (October 23, 2013) transferred funds from Component 2 and Component 3 to Component 1. Based on a MOH – PMTISS analysis, an original cost of US$27.3 per capita payments was established at appraisal. However, during the implementation phase the per capita increased to US$40.88. In addition, in 2012 the per capita was recalculated and increased to US$49.00 for the most accessible rural areas. As a consequence, Component 1, that financed the per capita payments, increased its total funds from US$25.8 to US$35.6 million, representing at the end, 89 percent of the total project loan. The restructuring allowed to finance the provision of the health service package by the mobile teams during 2013 and 2014.

2. Key Factors Affecting Implementation and Outcomes 2.1 Project Preparation, Design and Quality at Entry 29. Soundness of the background analysis. The Project significantly incorporated the lessons learned from the five years of experience implementing the IADB´s Multi-Phase Institutional Transformation Project – PMTISS that supported the MOH in the implementation of the PAISS in the following areas: i) the utilization of performance agreements and capitation payments for improving efficiency and quality; ii) the need for monitoring and auditing systems with spot checks; iii) the development of an individualized roster for the poor in rural areas. Finally, the Project addressed inequality by targeting the poor rural areas (that have lower health indicators than urban areas) and recognized the cash incentives which beneficiaries from poor households received from the CCT program RO to access and utilize health services. 30. Assessment of the Project design. Based on the analysis of Panama’s health sector, the PDO was important, clear and realistic for the country. In addition, the Project was responsive to Panama’s long-term strategy of expanding health services to rural areas and to global development objectives. Given the duration, resources, approach and previous experiences, the Project could be held accountable for the achievement of the outcomes. 31. The three Components were reasonable in relation to achieving the PDO. The MOH had a positive experience using capitation payments under the IADB loan. However, despite a careful assessment of the health sector needs during Project preparation, a significant increase in the per capita cost was unforeseen, which eventually caused the reallocation of funds from Components 2 and 3 to Component 1 which substantially increased Component 1’s final allocation from US$25.8 to US$35.6 million (from 64.5 percent to 89 percent of total project loan).

32. In order to strengthen and improve the health decentralization process, the RBF design called for a necessary but challenging coordination among several organizations: Ministry of Economy and Finance (MEF), MOH, UGSAF, Regional Departments of Health, mobile health teams hired by the RDH (OIs), private sector providers (OEs), the National Comptroller’s Office and the Independent Audit Firm. In addition, it involved coordination with other national agencies and programs such as the Ministry of Social

8

Development (MIDES) and the RO program. Nevertheless, the Project had a relatively strong and experienced implementing agency - the MOH-UGSAF. Despite the involvement of several organizations, the innovations introduced by the Project, and the different geographical areas covered, UGSAF managed to achieve positive results at the end as demonstrated in the achievement of 4 of 5 PDO indicators.

33. Adequacy of Government´s commitment. The MOH appointed the UGSAF, a relatively strong and capable implementing agency with more than five years of experience in a similar role under IADB and World Bank loans, as the unit responsible for the overall coordination of all technical, financial and administrative aspects of the Project. The UGSAF is responsible for priority health projects and it is composed of regular MOH staff. In addition, the MOH established a Project Executive Council to oversee overall implementation that reported directly to the Minister of Health. However, the Project costs included borrower’s counterpart funds in the total of US$18.73 million. During Project implementation, the provision of counterpart funds took much longer than expected due to compliance with national laws and internal control procedures. Despite the effort of the MOH, the UGSAF and the actors involved, internal procedures generated delays in payments to mobile health teams and gaps in the provision of health services. 34. Assessment of risks. Risks were properly considered, with the most important being: (i) weak country and sectoral level strategic planning as a basis for budget allocation; (ii) overlap among donors. However, despite mitigation measures correctly identified for these main risks, execution was not carried out. For instance, a policy framework to clarify the division of responsibility among development partners was suggested but not fulfilled. In addition, no sub-national operational risks were considered, only country and national sectoral level, and during the implementation phase sub-national weaknesses (in the Regional Departments of Health for instance) jeopardized overall implementation.

2.2 Implementation 35. The per capita payments and RBF model created the incentives for the health sector and mobile health teams to successfully achieve four of the five PDO indicators. In addition, the Project maintained a Moderately Satisfactory rating during implementation. However, there were implementation challenges during the operation:

36. Despite having an RBF approach, the Project had a complicated verification process – established in the Project’s Operations Manual - for the review and approval of the per capita payments that involved several actors performing repetitive tasks. First, at the sub-national level, reports were prepared and reviewed by private sector provider and the Regional Department of Health. Then, the reports were transported to the central national level, where they were reviewed by the National Directorate of Health Service Delivery of the MOH, the UGSAF, the Comptroller’s Office and the IAF. In many cases the information reviewed by these offices was similar such as, the name and existence of a beneficiary to prevent false reporting, the correct recording of each beneficiaries’ information and the health service delivered, and the results indicators. In addition, not

9

only did the review cycle involve several public offices, the Project also required a high number of reports per rural community: one every two, six and twelve months. In total, approximately 720 reports had to be reviewed every year by the offices mentioned above. Key personnel from the MOH and UGSAF dedicated several hours to oversee this procedure. To complete the verification process for each report took at least three months if there was no discrepancy. However, in cases with discrepancies, the process could take up to a year or more. In any case, the complicated review process resulted in significant delays in payments which subsequently affected the provision of health services in the rural areas by the mobile teams. For instance in 2010, from June to August, services were not delivered. Finally, the review process focused on reporting and did not fully guarantee that health services were actually delivered in the field.

37. In order to strength the stewardship role of the MOH and support the decentralization process, the Project involved the sub-national levels of health, the Regional Departments of Health (RDHs), and established that funds were transferred from the central level MOH to the RDH and finally to the private sector provider. Although it was a good initiative, implementation showed that it added an additional layer of compliance with laws and control mechanisms for the national and sub national levels. For instance, it increased the number of administrative steps from 16 to 32, contributing to the delays in the review process and the per capita and RBF payments.

38. All the procedures mentioned above heavily relied, at the beginning of the Project, on paper reports and manual procedures. Later, as part of Project’s activities to improve the MOH information system, a software solution was implemented that allowed the use of CDs which helped reduce the number of paper reports produced. However, the reporting process still demanded a significant amount of time from the Project’s human resources. The Project could have invested in the development of automatized and online solutions early on during implementation to support the coordination among the many verification layers and reduce the paper-based reporting which can lend itself to natural, human error. In addition, the mid-term review –September, 2011- could have been used to rethink the whole payment process and how the processes related to implementing the RBF scheme could have been made more efficient.

10

2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization 39. M&E design. Adequate indicators were identified to monitor progress towards the PDO. There were five PDO-level indicators, not an excessive number, which contributed to proper monitoring. Indicator 1 (prenatal controls), 2 (vaccination) and 3 (births attended by trained personnel) were directly related to the first Project objective: to increase access of rural communities to basic health services for improving mother and child basic health. These indicators are frequently used in similar programs, and therefore are easy to understand by professionals in the health sector and in project management. In addition, these indicators are linked to international goals and commitments hence they are already constantly monitored by national health systems. 40. Indicator 4 (national strategy) and Indicator 5 (rural patients receiving proper treatment for diabetes and hypertension) were connected to the second PDO. On one hand, Indicator 4 had a direct and clear connection with the second PDO and was very straightforward because it involved the implementation of a National and Demographic Health Survey, the first one in Panama. On the other hand, there is no clear connection between Indicator 5 and the second PDO. In addition, Indicator 5 is not yet widely used in developing countries; however, Panama has a chronic diseases program for these two pathologies hence it had experience to measure and monitor them. Finally, according to the PAD, the Project had a quality of health service dimension that was not captured nor measured by the indicators. The indicators focused on access to health services and not into quality of these services.

11

41. The Project had six intermediate indicators to monitor components’ progress. These indicators were adequate in number and in nature because they were in clear line with components objectives and had a right focus on monitoring the progress of implementation. No baseline data were defined at design stage, asking the related government institutions to carry out this task during Project Implementation. The innovation on the indicators was that the denominator was defined as a roster of beneficiaries and not the estimated population, allowing to a more precise monitoring of progress towards the targets. However, the indicators’ definition was not explicit in mentioning that the indicators only referred to the target population included in the roster. 42. M&E implementation. The UGSAF managed the collection of data from all agencies, data monitoring and evaluation. However, as described above, the process of data collection involved several institutions (MOH, UGSAF, RDH and OEs, OIs) and was manually intense, which complicated the monitoring process and delayed the production of information. This situation improved with the introduction of a software for data collection and reporting. According to stakeholders interviewed, there was also lack of training and supervision of the professionals registering the information. As established at the project design stage, the baseline was planned to be produced by the end of the first year of implementation. However, due to initial delays in Project implementation that also affected the prompt execution of the baseline, the baseline was produced during the second half of the second year of implementation. 43. M&E utilization. The long process that was required for collecting, analyzing and verifying the data and indicators severely jeopardized M&E utilization. Information about progress on the indicators was available several months (at least three months) after the actual provision of services in the rural area. Hence, the system did not allow for timely and constant measurement and feedback to inform Project progress.

12

2.4 Safeguard and Fiduciary Compliance

44. The safeguard policies triggered by the Project were two: (i) Environmental Assessment (OP 4.01) and (ii) Indigenous Peoples (OP 4.10). 45. The Project triggered OP 4.01 due to possible increases in the production of healthcare waste and minor rehabilitation works. The Project was not expected to generate any large scale, significant, or irreversible impacts, but localized impacts were possible from the improper transport, treatment or final disposal of healthcare waste by mobile teams or regional health facilities. As a result, the MOH generated a unified Environmental Assessment and Healthcare Waste Action Plan in order to mitigate the impacts of the Project. The Action Plan was implemented and included detailed checklists of procedures (from generation to disposal) tailored to the mobile team situation. The compliance of the checklist by mobile teams was supervised by the subnational health teams. The MOH also conducted an environmental screening and developed procedures for contractors outlining environmental, health and safety guidelines. However, because no intervention on primary health care facilities was executed such procedures were no utilized.

46. Approximately 9.6 percent of Panama’s population are indigenous peoples, the majority (82 percent) live in rural areas where health outcomes are lower than the rest of the population. The Project did not target indigenous jurisdictions (comarcas), nevertheless it triggered OP 4.10 because it included activities to support the implementation of the Indigenous Peoples Plan. The Project, in collaboration with other funds, partially financed trainings of health professionals about good practices of cultural adaptation of heath care services 47. Fiduciary compliance and Procurement. The MOH, through the UGSAF, was responsible for the financial management tasks of the Project, including: (i) budget formulation and monitoring; (ii) cash flow management; (iii) maintenance of accounting records; (iv) preparation of financial reports; (v) administration of underlying information systems; and (vi) arranging for the carrying out of external audits. In accordance with OP/BP 10.02 and the "Financial Management Practices in World Bank Financed Investment Operations", LCSFM completed a Financial Management Assessment (FMA). The FMA concluded that UGSAF had prior experience with managing the administrative and financial management aspects of externally financed projects; therefore, the unit already had in place the capacity necessary to monitor the financial aspects of the Project, as well as a basic administrative structure and financial management systems.

48. The Project finished with a Moderately Satisfactory Financial Management (FM) rating. Based on the FM supervisions carried out during the project life, it was concluded that: (i) the FM provided reasonable assurance that the loan proceeds were used for the intended purposes; (ii) the FM-related arrangements allowed an appropriate level of transparency that facilitated oversight and control and supported smooth Project implementation; and (iii) the legal FM-related covenants were met. The major FM-related

13

issues identified and followed-up during the final year of project implementation included: (i) occasional delays in submission of project Interim Financial Reports (IFRs) and audit reports, including the additional information requested by the Bank; (ii) delays in submission and low quality of the audit and operational audits of the transfers made under component/disbursement category 1, taking into consideration that the acceptance of such audits was a condition for disbursement under afore mentioned category 1; (iii) delays in execution of the local counterpart funds, agreed in the PAD. Procurement was rated “Satisfactory” during the life of the project. In a few cases MOH’s internal procedures caused delays to final award and contract signing and implementation.

2.5 Post-completion Operation/Next Phase 49. Presently, Panama’s MEF is considering a request from the MOH for a new health operation to be financed by the World Bank. This operation will finance the implementation of the DHS and continue the RBF model to address challenges related to maternal mortality.

3. Assessment of Outcomes 3.1 Relevance of Objectives, Design and Implementation 50. The rating for relevance of objectives, design and implementation is Substantial. The Project contributed to increased access of underserved rural communities to basic health services which was a country priority. Rural communities have always experienced poverty and the worst health indicators and for Governments in Latin America it has been a challenge to improve their situation. For instance, in Panama, the national percentage of women delivering children with the assistance of trained personnel has been above 90 percent since the year 2000 (it was 86.3 percent in 1991). And, also since the year 2000, the percentage of children below 1 year receiving with MMR and BCG immunization has been above 85 percent. However, for the target population of the Project, only 6 percent of women delivery with the assistance of a trained personnel and only 26 percent of children had complete vaccination. The target population had a very low starting point that reflects the challenge to reach and deliver health services in rural areas, difficulties such as the lack of proper infrastructure and equipment, the lack of proper transportation, cultural barriers, financial barriers, and the lack of health professionals willing to work in these remote areas, and so on. This difference between the progress in national averages and the marginalized target population, and the challenge involved in changing this situation, made the PDO highly relevant and in line, at that time, with the Millennium Development Goals. And reducing inequality will continue being a priority in the Sustainable Development Goals that the World Bank Group - including the Health, Nutrition and Population Global Practice - is currently supporting. Finally, it should be emphasized that the Project supported the delivery of a complete package of health services - and not additional or new health services- and that without the EEC supported by the World Bank rural communities wouldn’t have access to health services at all.

14

51. The design of the Project, that included a per capita and RBF model, was key for generating the incentives to deliver health services to poor rural populations. The causal chain was clear, convincing and supported by evidence from previous interventions. The Project generated new evidence about how successful per capita and RBF mechanisms can be for achieving results and closing inequality gaps. Despite the implementation difficulties, the Project provided a valuable learning experience for the MOH about how to implement RBF mechanisms, lessons that are now incorporated in new initiatives in the health sector. 52. In the public sector, the MOH frequently faced problems for delivering health services in rural communities such as lack of proper transportation to reach rural areas (to mobilize both health personnel and supplies); lack of medicines, supplies and instruments; difficulties to quickly hire and replace health human resources that decided to move to another rural or urban positions. By contrast, the private sector providers, through the Project, had the flexibility to quickly adapt to the demands and difficulties of working in rural areas. For instance, they arranged the proper transportation for health personnel and supplies, provided health professional with proper equipment for executing health exams in the field such as control of blood pressure, glycaemia, urinary tract infections, otoscopic examination, Papanicolaou test among others. In addition, the Project per capita RBF mechanisms created the incentives for OEs and OIs to achieve results on the ground and, in some cases, the OEs had strong financial bases allowing them to maintain the delivery of services despite the payment delays. 53. In conclusion, the Project’s objectives, design and implementation were relevant for tackling health inequalities in rural areas where a large part of the poor and extreme poor live.

3.2 Achievement of Project Development Objectives 54. Achievements against each project development objective are described and assessed below: (i) PDO 1 - Increased access of targeted underserved rural communities to quality basic health services known to improve mother and child health. Rating: Substantial. 55. This first PDO was supported through Component 1 and Component 2 which aimed to ensure access to a basic health service package and increase responsiveness of primary health services networks for the poor in rural areas. According to the audited data for the Project, a total of nearly 180,000 beneficiaries per year from rural non-indigenous areas received regular access to a basic package of health services through the mobile health teams financed under Component 1. This first PDO was measured by a total of six indicators, three of which were PDO-level indicators and three were intermediate indicators. Of the six indicators, a total of four were either surpassed or achieved, and two were not achieved. These are described below.

15

PDO Indicator 1. Percentage of pregnant women with at least 3 prenatal controls (one in each trimester). Surpassed.

56. The target of 70 percent was surpassed with an actual achievement of 86.1 percent. The factors contributing to the target having been surpassed were the emphasis given to early pregnancy monitoring and the coordination of the mobile health teams with the MOH fixed health facilities in the rural areas to avoid the loss of registered cases and ensure proper follow up of pregnant women.

PDO Indicator 2. Percentage of children below 1 year with a complete vaccination scheme for their age. Surpassed.

57. The formally revised target of 85 percent was surpassed with an actual achievement of 96.1 percent. The factors contributing to the target having been surpassed relate to the coordination of the mobile health teams with the MOH fixed health facilities to avoid the loss of recorded data for cases in which mothers may have first sought immunizations in the health clinic and not with the mobile health teams. In addition, the increase in demand for health services (promoted by CCT RO) in the first year of life provided ample opportunities for applying the vaccinations and the mobile health teams conducted frequent visits (at least every two months), both directly contributing to the target having been surpassed.

PDO Indicator 3. Percentage of women delivering children with the assistance of trained personnel from MOH. Surpassed.

58. The formally revised target of 65 percent was surpassed with an actual achievement of 92.76 percent. The Project provided a clear incentive (capitation payment per beneficiary) for mobile health teams to enroll pregnant women into the MOH prenatal control program and refer women for delivery to the closest MOH facility. In addition, CCT RO also provided incentives to beneficiaries for seeking care with the mobile health teams and health clinics.

Intermediate Indicator 1. Component 1 – Percentage of health regions that achieve more than 80 percent of agreed intermediary outcome targets. Not Achieved.

59. Due to political pressure, the MOH transitioned to performance monitoring system in which targets were negotiated with each Regional Department of Health to one where the MOH set very high common targets across all indicators for all the regions. This decision did not take into consideration the different challenges and baselines in each region. For instance, all regions had to achieve 100 percent of pregnant women covered with at least three prenatal controls. This resulted in only one region having achieved more than 80 percent of the agreed targets, resulting in an overall achievement of 20 percent.

16

Intermediate Indicator 2. Component 2 – Percentage of health regions that complete survey and mapping of human resources, equipment, and infrastructure. Surpassed.

60. All the Regional Departments of Health were able to complete the diagnostic and mapping of existing human resources for health, equipment, and infrastructure. The Project financed workshops where the national, regional, and local levels participated on how to carry out the diagnostic and mapping.

Intermediate Indicator 3. Component 2 – Percentage of health regions that have

successfully complied with accreditation plan of primary health care units in communities targeted by Component 1. Not Achieved.

61. Intermediate Indicator 3 was not achieved since funding needed to comply with accreditation was transferred to Component 1 to continue financing the provision of health services in the rural areas. This indicator is directly linked to Intermediate Indicator 1 which was also not achieved. Intermediate Indicator #1 required all Regional Departments of Health to achieve a common set of targets across all regions which did not take into account the varying challenges and contexts of each region. The idea of Intermediate Indicator #3 was that once each RDH achieved the common set of indicators, which would trigger each Region accessing funding which would allow it to purchase needed inputs to comply with accreditation. Since the common targets were only fully achieved by one region, the funding needed by the RDHs was not triggered. 62. In addition to the six indicators under the Results Framework, which informed the achievement of PDO 1, a set of RBF-specific indicators also monitored the Project’s progress and were the basis for determining the amounts of the capitation-based payments made by the MOH to both, the OIs and OEs. These RBF indicators monitored, on the one hand coverage, levels and, on the other hand, performance related to the delivery of priority health services. Under the RBF scheme, OIs and OEs were paid based on a per capita amount (US$49.00 for the most accessible rural areas). The potential maximum payment an OI or OE could receive was the per capita amount multiplied by the number of enrolled, eligible beneficiaries. The OI and OE mobile networks received a 20 percent advance which was deducted from the first payments in order to ensure sufficient cash flow for operations. The payments were then made in three parts. A total of 65 percent of the potential maximum was made every two months based on coverage levels achieved; 30 percent made every four months based on performance achieved in the priority health services delivered; and 5 percent paid annually based on a beneficiary survey. Table 1 and 2 below provide an overview of the Project’s achievement with regard to the coverage and performance related indicators.

Table 1. Component 1 RBF Coverage Indicators (2013-2014 Period) Indicator Definition Result

Communities visited

MHT visit at least 85 percent of the communities in a population group during each health round.

99%

Population groups protected

MHT should protect a minimum of 80 percent of the resident beneficiary population in the communities belonging to the

99%

17

population group that they are responsible for during each round.

Protected population effectively treated

MHT should treat a minimum of 50 percent of the protected population through the different services of promotion, prevention or attention established in the portfolio of incentivized services.

62%

Protected children under 24 months treated

MHT should treat a minimum of 80 percent of the registered population younger than 24 months old, through growth and development check-ups outlined in the PSPV.

97%

Days of services MHT should provide the PSPV with 38 days of direct medical services to the population, per health round.

100%

Source: MOH-UGSAF Progress Report for the Period July 1, 2014 to December 31, 2014

63. The first payment was based on coverage and was made every two months (at the end of each health round) and could be a maximum of 65 percent of the per capita amount. Payment was based on achieving target coverage levels for the 5 indicators in Table 1 above. If these targets were not met, the percentage of the per capita amount that the RDH or private sector provider received was determined by a table that defined a percentage level according to the degree of achievement. The rule used to determine the payment was choosing the lowest percentage achieved for the five indicators.

Table 2. Component 1 RBF Performance Indicators (2013-2014 period) No. Indicator OEs OIs

1 Percentage of pregnant women with at least three antenatal check-ups (one per trimester) by the end of the third trimester

78% 17%

2 Percentage of pregnant women registered out of estimated total

80% 22%

3 Percentage of pregnant women with second dose or booster of TT or TD

89% 21%

4 Percentage of births attended by trained staff 88% 23%

5 Percentage of women 20 years or older who have had pap smears

67% 11%

6 Percentage of children under one year who have had four or more growth-and-development check-ups

92% 16%

7 Percentage of children under one year with a complete vaccination record

94% 20%

8 Percentage of children aged four with at least two growth-and-development check–ups

87% 13%

9 Percentage of children aged one to four years old with a complete vaccination record

89% 15%

10 Percentage of symptomatic respiratory diseases recorded out of estimated total

13% 3%

Source: MOH-UGSAF Progress Report for the Period July 1, 2014 to December 31, 2014 64. The second tranche payment, paid every 4 months, was based on performance levels achieved and could be for a maximum of 30 percent of the per capita amount. The results from the previous period were supposed to be used as the basis for setting the

18

targets for the next period; and the MOH, UGSAF and the RDH would meet every four months to establish the threshold levels indicator by indicator and by population group. However, the complexity of the Project generated delays in the flow of information that made this planning exercise difficult. As Table 2 above shows, the OEs had a better performance in achieving the performance indicators comparted to the OIs. (ii) PDO 2 - Support the development of strategic planning, regulatory, and monitoring mechanisms known to improve health system performance. Rating: Modest. 65. The second PDO was supported through Component 3 which aimed to strengthen the MOH institutional capacity to carry out its stewardship role in the health sector. It was measured by a total of five indicators, two PDO indicators and three intermediate indicators of which a total of three were either surpassed or achieved, and two were not achieved.

PDO Indicator 4. National strategy with baseline and targets on improving equity in access to health care services established using DHS data. Not Achieved.

66. The national strategy was based on DHS data; however, the DHS was not carried out. The DHS was not carried out because: a) delays in initiating Project implementation, b) lack of funding. At the beginning of Project Implementation there was an initial delay of almost 15 months. According to the mid-term review (September 2011) documentation, the delay was due to change in government authorities after the elections, the review of Project commitments, Project legal approval, among other reasons. Once Project started implementation and preparation of DHS, funds were transferred from Component 3 to Component 1 in order to continue financing the increasing costs of the provision of health services in rural areas which was considered the priority. Public sources were going to be used to finance this activity, consequently the indicator was not formally revised. Terms of Reference were jointly developed by the MOH in partnership with the Instituto Conmemorativo Gorgas de Ciencias de la Salud and the National Institute for Statistics. However, funds were not obtained and at the end of the Project the DHS was not implemented. The MOH and Bank team are presently exploring financing the DHS under the new Social Protection Project under preparation.

PDO Indicator 5. Percentage of individuals from rural areas diagnosed with diabetes and hypertension in need of medication receiving prescribed drugs by primary health care units according to the Ministry of Health Protocol. Achieved.

67. The original target of 55 percent for both diabetes and hypertension was achieved for diabetes with an actual value of 54.37 percent and surpassed for hypertension with an actual value of 78 percent. Despite diabetes not having reached the full 55 percent, the difference between what was achieved (54.37) and the target of 55 percent is statistically insignificant and considered achieved. The Project promoted and financed the identification and registration of chronic disease patients and the proper treatment.

19

Intermediate Indicator 4. Component 3 – Establishment of a baseline on access to primary health care services by the poor in rural areas using data from the demographic and health survey. Not Achieved.

68. PDO Indicator 4 (above) and Intermediate Indicator 4 are directly linked as the latter (establishing the baseline) depended on the former (DHS) being completed. Since the DHS was not carried out, the Project was unable to establish a national baseline on aces to primary health care services for the entire country. This national baseline was different from the Project baseline focus on selected communities and based on an individualized roster.

Intermediate Indicator 5. Component 3 – Percentage of drugs from the Primary Health Care Level Essential Drug List that is available at health centers. Achieved.

69. This target was achieved with 80 percent of drugs from the Primary Health Care Level Essential Drug List being available at the health centers. The Project financed training which strengthened the procurement function of the basic health units that provide health services and who were required to carry out evidence-based planning based on an identified and registered population. The coordination with the MOH health network was strengthened to provide these basic health units with the essential medicines through rounds and by providing trainings and technical assistance.

Intermediate Indicator 6. Component 3 – Percentage of health regions using automated monitoring and information system for assessing achievement of results of primary health care providers. Surpassed.

70. The target of 80 percent was surpassed with an actual achievement of 100 percent. The key factors contributing to surpassing this indicator target was an intense training program with each Regional Department of Health which included basic elements such as software installation, activities with practical examples, and the continuous support of the system. The Project financed the training program and also financed the procurement of the work stations. 71. The Project surpassed its targets for three PDO indicators, achieved its target for one, and only one was not achieved. The four indicators surpassed or achieved are directly connected to very relevant results for the poor. They were related to the delivery of health services in the rural communities in order to tackle inequality. Like in many other countries in Latin America, Panama’s health outcomes in rural areas have been low for decades. In the six years the Project was implemented, it achieved a remarkable improvement of rural health situation: the Project contributed to increase the percentage of pregnant women with at least 3 prenatal controls from 20 percent (2010) to 86.1 percent (2014); the coverage with the full vaccination scheme for less than 1 year children increased from 26 percent (2010) to 96.1 percent (2014); and the delivery of children with the assistance of trained personnel increased from 6 percent (2010) to 92.76 percent (2014). Even for chronic health problems like diabetes and hypertension, patients receiving medication reached 54.37 percent and 78 percent, respectively. These

20

remarkable results were possible due to the work of the MOH, the UGSAF, and the mobile health teams from the public and private sector contracted and supervised under the scheme of per capita and RBF payments. This was a goal difficult to achieve it is still at the core of Panama and World Bank development objectives. 72. On the basis of ratings for the two parts of the PDO, the overall rating is weighed in favor of PDO1, also taking into account total project funds spent to achieve the intended outcomes under PDO1 (89% of total project costs). In addition, the Project achieved 80% of PDO indicators, and all of the indicators achieved were related to the delivery of health services to the poor in remote rural areas. Hence, overall rating for Efficacy is Substantial.

3.3 Efficiency 73. The rating for efficiency is Modest. Based upon the costs and measurable economic benefits of the Project, results of the cost-benefit analysis suggested that Project benefits exceeded Project costs in term of net present value. The MOH, with the support of a consulting firm and IADB’s PMTISS, estimated the costing of capitation payments to ensure access to a basic package of primary health care services. This estimate considered indicators such as rates of use of the services by the beneficiary population, cost differences across the regions, difficulties in accessing the services, and the epidemiological profile of the population covered. The average unit cost of the package was estimated to be US$27.3 per capita. However, during the implementation phase of the Project, the per capita was recalculated and estimated at US$40.88 per capita for the years 2009-2011 and US$49.00 per capita for the years 2012-2014. 74. For the ICR, two types of benefits for the target population of the Project were estimated for the cost-benefit analysis: i) deaths avoided during the period of implementation of the Project, ii) lives with disability avoided as a result of the Project. The criteria adopted to evaluate the economic efficiency of the Project included the Net Present Value (NPV) and the Internal Rate of Return (IRR). In addition, a sensitivity analysis was included to consider different hypotheses on the effectiveness of the Project. Assumptions made for the economic analysis include:

75. The productive life for the poor was considered to be from the period of 20 to 60 years of age. Benefits of the lives saved were calculated using the salary of US$188.74 monthly, which was the average salary for rural areas reported by the Ministry of Finance for the year 2011. A discount rate of 10 percent, which represents the investment opportunity cost of project resources, was used to discount total benefits generated by the Project.

76. Taking all this into consideration, the Project’s proposed interventions generated a net economic benefit of US$85 million, in present value terms. The internal rate of return of the Project was calculated at 24.97 percent, which exceeded the 10 percent discount

21

rate. In other words, other alternatives would have obtained 10 percent versus the 24 percent generated by investing in this health Project. Even under a 20 percent reduction scenario, the 16 percent IRR was higher than the discount rate. Finally, it should be emphasized that the Project supported the delivery of a complete package of health services and not additional or new health services. And that without the EEC supported by the Bank rural communities wouldn’t have access to health services at all. 77. Despite the Project has a favorable economic analysis based on the NPV and IRR, the operation generated an undesirable burden on the public sector (MOH, UGSAF, RDH, Comptroller’s Office), associated with an unintended waste of humans resources time, capacities, and efforts. 3.4 Justification of Overall Outcome Rating Rating: Moderately Satisfactory 78. The Project’s overall outcome is based on the achievement of the majority of the PDO indicators (four out of five), its relevance and efficiency. 3.5 Overarching Themes, Other Outcomes and Impacts (a) Poverty Impacts, Gender Aspects, and Social Development 79. The Project targeted excluded populations, particularly the rural poor. In addition, the focus was on maternal and child health. See previous section 3.4 for results. (b) Institutional Change/Strengthening 80. The achievement of PDO indicators was possible because the RBF approach created the incentives for service delivery on the ground and allowed to contract, pay and manage mobile health teams under a results framework. The RBF institutional arrangement is now being used to improve the delivery of services in primary health care clinics and has the potential to scale to other levels of care. (c) Other Unintended Outcomes and Impacts (positive or negative) N/A. 3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops N/A

4. Assessment of Risk to Development Outcome Rating: Moderate 81. The EEC has been financed by international donors since 1995. After Project closure the GOP continued financing the provision of health services using an IADB loan that will conclude in 2016. The GOP is currently preparing a new project to continue financing and improving the health services provided, a project that World Bank aims to support.

22

5. Assessment of Bank and Borrower Performance

5.1 Bank Performance (a) Bank Performance in Ensuring Quality at Entry Rating: Moderately Satisfactory 82. During Project preparation, a group of professionals with vast experience in health programs in general, and with experience in RBF in particular, were gathered to prepare the Project design and tackle the clients’ sectorial problems. The team relied heavily on lessons learned from previous World Bank financed and IADB financed projects. In addition, the Project was consistent with the Government’s strategy and priorities in the health sector and with globally agreed goals. However, the per capita cost for Component 1 was underestimated and this situation negatively influenced the implementation of Components 2 and Component 3. (b) Quality of Supervision Rating: Moderately Satisfactory 83. Adequate budget and staff resources were allocated. World Bank supervision missions took place on regular basis and the Project was closely monitored. In addition, restructurings prove that the World Bank addressed problems that emerged in a timely manner. Aide memories provide evidence that the World Bank offered advice and observations. There were two major problems that were discussed with MOH authorities during supervisions but not properly addressed by the World Bank: a) the pronounced increase in capitation costs that at the end compromised the achievement of the second PDO. Possibilities to address this issue could have included additional funding or the restructuring of component 3; b) the Project verification process for the innovative payment mechanisms were very complex and put an excessive administrative burden on public offices that affected the operation. It caused delays on payment to mobile health teams that interrupted the provision of basic health services. This complex process – established in the Project’s Operations Manual - was in place during the six years the Project was implemented. Despite there were discussions about this problem no solution was implemented, nor was a review conducted of the valued added by the verification process at each office, nor was the Project’s Operations Manual modified to simplify the process. 84. Fiduciary and safeguards policies were properly followed and reported.

(c) Justification of Rating for Overall Bank Performance Rating: Moderately Satisfactory 85. The World Bank provided a moderately satisfactory quality at entry and offered a proper and close supervision. However, the Project had complex administrative processes under the RBF component that jeopardized the operation and was not properly addressed by the World Bank.

23

5.2 Borrower Performance

(a) Government Performance Rating: Moderately Satisfactory 86. The GOP supported this operation during its design and implementation. Evidence of its support is the willingness and agreement to adopt an innovative payment mechanism such as per capita payments and RBF. In addition, this is exemplified by the GOP willingness to contract private sector organizations to deliver public services. In addition, the GOP was successful in achieving full loan disbursement. However, due to the complex administrative process for reviewing the results and approving the per capita payments, there were severe delays in per capita payments to RDH and MHT. (b) Implementing Agency or Agencies Performance Rating: Moderately Satisfactory 87. The MOH, through the UGSAF, was the lead implementing agency for the Project. Despite the complexity of the Project: several different actors, several different regions, new payment mechanisms, the MOH and the UGSAF managed to achieve four of the five PDO indicators. However, the implementation of the DHS, an activity related to one PDO indicator, was not carried out because the funds were transferred to cover the increasing per capita costs related to the delivery of prioritized health services under Component 1. The terms of reference for the DHS were elaborated and the MOH tried to cover the cost of its implementation using public funds, however, it was not possible. 88. Despite a delayed beginning, during the implementation, the MOH showed its commitment to the Project by organizing stakeholders’ consultations, and by managing all the arrangements for implementing RBF (Component 1), including the monitoring and evaluation aspects. (c) Justification of Rating for Overall Borrower Performance Rating: Moderately Satisfactory 89. Despite the fact that the DHS was not implemented and the 2013 payment to private sector providers is pending, the Project overcame a severe administrative burden and achieved and even surpassed four of the five PDOs. In addition, the health services package is currently being provided (using a loan from IADB) and it has secure funding until 2016. Based on the positive experience of HEPI, the MOH has decided to test new management agreements based on results with primary health care facilities.

6. Lessons Learned 90. Per capita payments and RBF are efficient policy tools to generate incentives for delivering health services to communities in rural areas. Furthermore, per capita payments and RBF can be useful not only for tackling old challenges like a maternal and

24

child health service coverage, but also for new problems such as proper treatment of diabetes and hypertension. However, when establishing the per capita cost, estimates should be updated and take into consideration the variation of regional costs to avoid problems like in the case of the HEPI Project where the costs substantially increased affecting the implementation of other activities. In addition, it is necessary to secure the proper controls and to comply with national administrative laws and regulations. However, the verification process should be carefully designed to avoid complex processes that threaten the achievement of results, as occurred in the HEPI Project. Constant evaluation of the burden of new procedures is generating in the public sector, its relevance, and utility for achieving results is required. Roles and responsibilities also should be clearly defined. 91. In addition, when implementing RBF mechanisms, the time period between the delivery of the health package and the verification and payment should be short, otherwise it could compromise the financial situation of providers and consequently jeopardize the delivery of services. In addition, if the time between service provision and payment is too long, it affects the perception of the payment actually being based on results. To shorten this period of time and avoid the duplication of steps in the RBF process, the roles of the different organizations involved in the implementation of the RBF scheme should be clearly defined and procedures must be made simple. In addition, when preparing indicators, there should be a balance between the technical and administrative objectives.

92. Private sector providers can be partners in the delivery of health services to the poor in rural areas, but this requires the MOH to perform a strong supervision role. In the HEPI Project, private sector providers showed they had the flexibility to rapidly adapt to adverse geographical conditions and overcome traditional public sector problems such as shortage of medicines or equipment. In addition, they had the financial capacity to continue delivering services in the rural areas despite the delays in payments. However, the participation of the private sector requires the MOH to be capable of performing a strong supervision role, particularly at the regional levels, to secure contract compliance and quality of the health services delivered. A capacity usually weak at regional levels.

93. Integrated and automated information systems can facilitate per capita payments, RBF and sector management. The verification process used information systems that were not well integrated and required an extensive manual review. Such manual reviews were inefficient, carried high transaction costs, and could produce reporting errors. Information about the Project was not available in a timely manner for decision making. Information systems could be integrated and automated to increase efficiencies and produce prompt information for sector management and results monitoring. In addition, these information systems should be as much independent and transparent as possible, and use the country indicators definitions.

94. Coordination among lending institutions is key to reduce the burden on public offices and avoid potential financial gaps and service provision. The HEPI Project was in line and financially supported the EEC strategy that started in the late nineties in Panama.

25

Since the beginning, the ECC strategy was financed by several international financial institutions, mainly the IADB and the World Bank. The management of different financial sources (projects) generates a burden on the implementing agency, and it also sporadically generates financial gaps that could affect the provision of services. The main international financial institutions could dialogue and discuss with the Government a long term plan for financing together key health goals, programs and interventions.

7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners (a) Borrower/implementing agencies

See Annex 7.

(b) Cofinanciers

(c) Other partners and stakeholders (e.g. NGOs/private sector/civil society)

26

Annex 1. Project Costs and Financing (a) Project Cost by Component (in USD Million equivalent)

Components Appraisal Estimate

(USD millions)

Actual/Latest Estimate (USD

millions)

Percentage of Appraisal

Percentage of Final Loan

Component 1: Ensuring access to primary health care services by the poor in rural areas

25,795,400.00 35,659,221.69 64.48 89.14

Component 2: Increase responsiveness of primary health services networks to vulnerable populations in rural areas

6,205,400.00 328,850.29 16.10 0.82

Component 3: Strengthening MOH’s institutional capacity to exercise stewardship in the health sector

6,987,950.00 4,008,785.81 18.78 10.04

Total Baseline Cost 40,000,000.00 40,000,000.00 100.00 100.00 Total Project Costs 0.00 0.00

Total Financing Required 40,000,000.00 40,000,000.00

(b) Financing

Source of Funds Type of

Cofinancing

Appraisal Estimate

(USD millions)

Actual/Latest Estimate

(USD millions)

Percentage of Appraisal

Borrower 18.73 16.30 87.02 International Bank for Reconstruction and Development

40.00 40.00 100.00

27

Annex 2. Outputs by Component

Component Planned outputs at Appraisal Actual outputs/outcomes at ICR

Component 1: Ensuring access to primary health care services by the poor in rural areas.

1. Increase access by targeted underserved communities to basic health services known to improve mother and child health care.

On average, 180,000 beneficiaries were enrolled into the PSPV and health services were provided to them.

Component 2: Increase responsiveness of primary health services networks to vulnerable populations in rural areas

2.1. Strengthening health regions’ management capacity to develop and implement primary health care interventions in rural areas.

A survey and mapping of human resources, equipment and infrastructure was completed by health regions. An accreditation plan was successfully complied by health regions.

2.2. Improving the delivery of health services to vulnerable populations living in remote rural areas.

Training of health personnel for intercultural health services provision.

Component 3: Strengthening MOH’s institutional capacity to exercise stewardship in the health sector

3.1. Improving the MOH’s institutional capacity for strategic analysis and planning to improve performance in the health sector.

National Health Strategy was elaborated. Terms of Reference for DHS were prepared.

3.2. Update health regulations and revise enforcement mechanisms.

Primary Health Care Level Essential Drug List is available at health centers.

3.3. Strengthening management and monitoring, and information systems of national programs to improve access to health services by the poor.

Health information system was developed and implemented for automated monitoring and assessing achievement of results of the primary health care providers.

28

Annex 3. Economic and Financial Analysis

1. This annex presents the cost-benefit analysis of the Health Equity and Performance Improvement Project, based upon the costs and measurable economic benefits of the Project. The cost-benefit analysis suggests that project benefits exceeded project costs in term of present value. Package of basic health services: the average unit cost

2. Since 2003, the MOH has developed a national program to increase access to health services in rural areas utilizing innovative disbursement mechanisms that create incentives for providers to achieve better results.

3. A basic health services package was established and delivered to rural populations through mobile teams from the MOH (known as Organizaciones Internas) and the private sector (known as Organizaciones Externas). Payments to mobile teams were based upon achievement of agreed sanitary goals and verified through concurrent external and social audits.

4. In 2002 and 2003, the MOH, with the support of a consulting firm and IADB’s PMTISS, estimated the costing of capitation payments to ensure access to a basic package of primary health care services. This estimate considered indicators such as rates of use of the services by the beneficiary population, cost differences across the regions, difficulties in accessing the services, and the epidemiological profile of the population covered. The average unit cost of the package was estimated to be US$ 27.3 per capita (this was the per capita used in the original PAD assessments).

5. However, during the implementation phase of the Project, the per capita was recalculated and estimated at US$ 40.88 per capita for the years 2009-2011 and US$ 49.00 per capita for the years 2012-2014. Economic Analysis

6. Two types of benefits for the target population of the project were estimated for the cost-benefit analysis: i) avoided deaths during the period of implementation of the Project, ii) lives with disability avoided as a result of the Project. The criteria adopted to evaluate the economic efficiency of the Project included the Net Present Value (NPV) and the Internal Rate of Return (IRR). In addition, a sensitivity analysis was included to consider different hypotheses on the effectiveness of the Project. Assumptions made for the economic analysis include:

7. Productive life for the poor was considered to be from 20 to 60 years-old.

Benefits of the lives saved were calculated using the salary of US$188.74 monthly which was the average salary for rural areas reported by the Ministry of Finance for the year 2011. A discount rate of 10 percent, which represents the

29

investment opportunity cost of project resources, was used to discount total benefits generated by the Project.

8. The Project benefits were derived from the economic value of the lives saved and

disabilities avoided. For the calculation of the Project benefit we focus on the future economic value of deaths and disabilities avoided.

9. To obtain the present value of total future benefits generated by the project, total

present value of deaths avoided and disabilities reduced were calculated using a discount rate of 10 percent. A summary of these results is presented in the following table:

Table 1. Estimated present value of total future benefits (US$) Year Present Value of Benefit

from Deaths Avoided Present Value of

Benefit from Disabilities Reduced

Total Benefit Present Value

2009 13,537,432.70 101,530.75 13,638,963.44 2010 14,493,502.36 108,701.27 14,602,203.63 2011 18,097,750.98 135,733.13 18,233,484.11 2012 18,702,666.08 140,270.00 18,842,936.08 2013 15,530,659.25 116,479.94 15,647,139.19 2014 4,648,218.61 34,861.64 4,683,080.25 Total 85,010,229.97 637,576.72 85,647,806.69

10. All this into consideration, the project’s proposed interventions generate a net

economic benefit of US$85 million, in present value terms.

Table 2. Net present value on investment rate of return (US$) Year Present value of benefit Present calue of cost Present value net

benefit 2009 13,638,963.44 26,905,788.05 8,114,643.47 2010 14,602,203.63

5,524,319.97 8,658,452.47

2011 18,233,484.11 5,943,751.16 13,633,484.11 2012 18,842,936.08 4,600,000.00 11,608,637.36 2013 15,647,139.19

7,234,298.72 5,855,297.09

2014 4,683,080.25 9,791,842.1 4,683,080.25 NPV (discount rate 10%) 10,979,959.86

IRR 24.97%

30

11. The internal rate of return of the Project was calculated in 24.97% percent, which

exceeded the 10 percent discount rate. In other words, other alternatives would obtained 10 percent versus the 24 percent generated by investing in this health Project. Even under a 20 percent reduction scenario, the 16 percent IRR was higher than the discount rate.

12. Finally, to evaluate the sustainability of the Project, we evaluated its fiscal impact. As described in the table below, the implementation of the Project had a fiscal impact of maximum 0.55 percent of MOH budget. And in 2012 it was only 0.36 percent. These figures indicate that the Project is sustainable.

Table 3. Fiscal Impact (in millions of US$)

2009 2010 2011 2012 2013 2014Project cost per year 7.1 5.5 5.9 4.6 7.3 9.6MOH annual budget 866.2 979.7 1078.8 1263.1 1511.9 1856.1% of MOH annual budget 0.82 0.56 0.55 0.36 0.48 0.52

31

Annex 4. Bank Lending and Implementation Support/Supervision Processes (a) Task Team members

Names Title Unit Responsibility/

Specialty Lending Fernando Montenegro Torres Senior Economist (Health) GHNDR TTL Luis Orlando Perez Senior Public Health Spec. GHNDR Health Andrea C. Guedes Senior Operations Officer GEDDR Operations

Fabienne Mroczka Senior Financial Management Specialist

GGODR FM

Alexandre Borges de Oliveira Senior Procurement Specialist GGODR Procurement Ximena B. Traa-Valarezo Consultant GSURR Indigenous Natalia Moncada Senior Executive Assistant GHNDR Administrative Supervision/ICR Keith E. Hansen Vice President GPSVP Management Fernando Montenegro Torres Senior Economist (Health) GHNDR TTL Maria Eugenia Bonilla-Chacin Senior Economist GHNDR TTL Carmen Carpio Senior Operations Officer GHNDR TTL Renzo Efren Sotomayor Noel Young Professional GHNDR ICR Author Luis Orlando Perez Senior Public Health Specialist GHNDR Health Evelyn Rodríguez Consultant GHNDR Procurement

Fabienne Mroczka Senior Financial Management Specialist

GGODR FM

Dmitri Gourfinkel Financial Management Specialist GGODR FM Alexandre Borges de Oliveira Senior Procurement Specialist GGODR Procurement Evelyn Villatoro Senior Procurement Specialist GGODR Procurement

Patricia De la Fuente Hoyes Senior Financial Management Specialist

GGODR FM

Fernanda Balduino Finance Analyst WFALN Disbursements Solange A. Alliali Lead Operations Officer AFCRI Legal Natalia Moncada Senior Executive Assistant GSURR Administrative Sonia M. Levere Program Assistant GHDNR Administrative

32

(b) Staff Time and Cost

Stage of Project Cycle

Staff Time and Cost (Bank Budget Only)

No. of staff weeks USD Thousands (including travel and consultant costs)

Lending FY08 41 275.193 FY09 0 0.00

Total: 275.193 Supervision/ICR

FY09 18 74.869 FY10 18 84.031 FY11 11 65.760 FY12 20 95.093 FY13 22 95.512 FY14 20 136.460 FY15 14 77.434

Total: 164 904.352

33

Annex 5. Beneficiary Survey Results N/A

34

Annex 6. Stakeholder Workshop Report and Results N/A

35

Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR

Unofficial translation

Republic of Panama Ministry of Health

International Bank for Reconstruction and Development (IBRD)

RepHEALTH EQUITY AND PERFORMANCE IMPROVEMENT PROJECT

Loan Agreement No. 7587-0-PA

PMES)‐

Executive Results Report

Panama, March 2015

36

Glossary of Terms No. TERM ACRONYM

[Spanish acronym]

DEFINITION

1 Comprehensive Attention to Childhood in the Community

AIN-C It is a new strategy to look after child health and nutrition based on "self-care" for children in families, by monitoring and promoting growth at community level, and for pregnant women, by providing individualized consultancy, iron supplements and appropriate references from the community to basic attention units for pre-natal controls.

2 Social Audit AS It is a follow-up and assessment mechanism by means of which communities may assess the degree of satisfaction regarding the services delivered by the Basic Health Teams.

3 External Technical Audit ATE Review of the main aspects of CPP provision, including at least: Checking the integrity, truthfulness and consistency of the beneficiary

population registry and its consistency with the information system. Checking the scope of results of coverage indicators and monitoring and

assessment. Checking the use and management of fund transfers made to the Health

Regions for the provision of CPP by UBAs of the fixed and mobile network. Checking the integrity, truthfulness and consistency of the payments made by

the provision of CPP to UBAs of the fixed and mobile network, which correspond to the results achieved.

Checking the agreement with the provision of health services rendered with the CPP, the manuals, regulations, guides and protocols by the UBAs of the fixed and mobile network.

Certify the compliance with the management commitments and contribute to the continuous improvement of the operation of the coverage extension strategy and the coverage strengthening strategy of primary health attention.

Certify the compliance with the contractual terms by the providing OA of the CPP in the UBAs of the fixed and mobile network.

Certify any act, fact or indication or omission which may be conceived as a prohibited practice.

4 Prioritized Service Package

CPP Is the package of services (including promotion, prevention and treatment) granted to beneficiaries protected by the coverage extension strategy (mobile network) and the coverage strengthening strategy at the primary healthcare level (fixed network).

5 Beneficiary Community - Is the community receiving the CPP by the coverage extension strategy (mobile network) and by the coverage strengthening strategy of primary healthcare level (fixed network). It is selected by means of the focusing process, which is based on technical criteria approved by MINSA.

6 Central Community - Is the geographically accessible community from which the population of said community and the beneficiary population of a group of neighboring communities are taken care of, which cannot exceed 600 people altogether.

7 Peripheral Community - That community geographically near to a central community where the attention process to strategy beneficiaries is executed.

8 Visited Community - That community which appears registered with the attention, prevention or promotion record of an individual and/or in the Daily Activity Registry, as its origin community during the same round.

9 Average Unit Cost Per Capita

- It is the average unit amount of monetary resources to be used as payment base associated to each beneficiary.

10 Attention Days - Days dedicated to the direct attention of the population. 11 Transfer Days - Time destined to the EBS mobilization between the beneficiary communities or

between the UBA headquarters and the beneficiary communities. 12 Basic Health Team EBS Minimum team of health human resources destined to the CPP provision by the

MINSA. EBS may be Institutional or Extra-institutional, directly hired by the MINSA or through Extra-institutional Organizations, respectively. EBS are attached to the UBAs, which may be of the mobile or the fixed network.

13 Community Health Workers

ECS Members of the community trained by the MINSA so that they support the CPP provision and follow-up health interventions in the community.

14 Management Team EG Human resources included in the support unit for managing, monitoring and

37

No. TERM ACRONYM [Spanish acronym]

DEFINITION

assessing the UBA's CPP. 15 Circuit - Group of visits performed to one or several beneficiary communities which belong

to a population group. 16 Institutional Circuits GI Generic denomination granted to the CPP provision through institutional providers,

i.e., Regional Health Offices (DRS). 17 Population Group GP Group of communities which made up a beneficiary sanitary sector of the

strategies. It is the result of the process of grouping previously focused communities, which maintain accessibility characteristics and communication lines in common according to the demand flow. One UBA is assigned to each population group.

18 Nominalization of healthcare

- Process consisting on the finished knowledge of the whole beneficiary population of the coverage extension strategy (mobile network) and the coverage strengthening strategy (fixed network) for the continuous follow-up of each beneficiary and his/her health condition.

19 Extra-institutional Organization

OE Social and autonomous organizations legally fit to be hired by the MINSA, through tender, to provide the CPP.

20 Initial Beneficiary Registry

- Nominalized list of people resident in communities focused by the MINSA, which have been registered by the providing UBAs of the CPP and which is delivered by the MINSA at the beginning of each provision year.

21 Beneficiary Population Registry

PPB Nominalized list of the total people resident in communities focused by the MINSA and registered by the UBAs as CPP beneficiaries, prior to the certification of the registration process which corresponds to the ATE.

22 Certified Beneficiary Population Registry

PPBC Nominalized list of the total people resident in communities focused by the MINSA and registered by the UBAs as CPP beneficiaries, which integrity, reliability and consistency have been certified by the ATE.

23 Capitated Payment - Monetary expression of the amount destined to the CPP provision insuring for each individual incorporated as beneficiary, which corresponds to the calculation of the average unit cost per capital of the CPP provision.

24 General Population Taken Care of

- Number of people who have received one (1) care, prevention and/or promotion activity during one (1) attention round.

25 Population of Less Than 24 Months Taken Care of

- Number of children from 0 to 24 months who have received one (1) growth and development control by the EBS and at least one (1) growth monitoring by the trained supervisor during one (1) attention round.

26 Beneficiary Population - People who belong to beneficiary communities of coverage extension strategy (mobile network) and/or coverage strengthening strategy (fixed network) and which are defined and grouped in population groups by means of technical criteria established by the MINSA to be delivered to the CPP.

27 Protected Population - Number of people living in a community which has been visited or looked after during an attention round.

28 Attention Round - Period of time destined to the attention to all the beneficiary communities belonging to a population group by the Basic Attention Units, whether mobile, fixed or combined. The duration of a round will be at least of fifty (50) days and maximum sixty (60) days, and six (6) Rounds should be carried out / Two-month period of service provision per year.

29 Coverage Strengthening Information System

- Group of elements among which the beneficiary population, the UBAs, the CPP, the registry forms, the computing tools and the deliverable reports are included, which interact among them for processing data and generating information.

30 Basic Attention Unit UBA Generic denomination of the different types of strategy service suppliers: A- Mobile network: health visits with institutional or extra-institutional basic

teams in charge of providing the CPP to the beneficiary population. B- Fixed network: health centers, health sub-centers and health posts.

38

Project Information (Project Card)

Project Name Project for the Improvement of Equity and Health Performance (PMES, for its Spanish acronym)

Start Date Effective date: December 8, 2008

Completion Date Closing: December 31, 2014 Extension granted: 6 months upon request of the MEF: New closing date to June 30, 2015

Amount of the Project USD 40,000,000 World Bank USD 18,730,000 GOPN Total USD 58,730,000

Cooperation or Financing Agency

International Bank for Reconstruction and Development (IBRD) / World Bank

Beneficiaries

Rural population from 10 out of the 14 health regions of the country facing geographical barriers which make its access to the fixed network of MINSA health facilities difficult and which live in conditions of poverty or extreme poverty.

Vulnerable populations are chosen through a focusing process and then, they are divided into population groups and ascribed to Basic Attention Units, which are in charge of providing this population with the Prioritized Service Portfolio containing health promotion, prevention and attention activities.

General Purpose Reducing the inequity in the health sector by increasing the access to basic care services in poor rural areas and improving health results, which will benefit the more vulnerable groups of the population.

Project Components

o Component 1: Health Protection for Vulnerable Populations o Component 2: Strengthening of service network for Health Protection of

Vulnerable Populations o Component 3: Strengthening of MINSA's administrative capacity

Strategic Points of the Project

Health Protection for vulnerable populations (through PSPV) Strengthening of service networks for health protection of vulnerable

populations, including, among other things: - Strengthening of management capacity of Regional Health Offices in order to develop primary attention interventions in rural areas - Improvement of health service provision to vulnerable populations which live in rural areas with access barriers to health services

Strengthening of MINSA's administrative capacity, including, among other things: - Strengthening of institutional capacity for the analysis and strategic planning in order to improve the performance in the health sector- Strengthening of MINSA's capacity to regulate and control the sector - Strengthening of management, monitoring and information systems of national programs to improve the access of poor people to health services

39

Financial Execution (Budget Observance)

To December 31, 2014, the Financial Execution of the Project was 89.3%.

Executive Summary During the project, MINSA worked to (i) cause the providers and/or executing units to manage the operations under their responsibility; (ii) manage the information, monitoring, supervision, assessment and audit systems; and (iii) control the operations for the purposes of reaching the goals of the development indicators and the intermediate results of the Project, thus achieving sustainability of interventions. Once the period established in the loan agreement ended, the following chart shows the project Cycle, where the service provision periods and the project's financial programming are evidenced and where it is indicated that 100% of the agreed term for the implementation of the project has been consumed.

Project Cycle

With respect to the financial Execution, to December 31, 2014, 89.3% of the project's funds has been executed. With the already accepted addendum to June 30, 2015, 100% will be executed.

To date, 95% of technical processes have concluded. Thus, the main goals of the PMES can be summarized as follows:

40

(i) 99% of the certified beneficiary population registry was satisfactorily administered by the 10 Health Regions of the Project. (ii) 97.8 % of the population groups received basic health services on a continuous basis through the 8 providers. (iii) 100% of the Health Regions with primary attention service networks in rural areas took steps in order to strengthen the organization of service networks and the authorization of their facilities. Said actions included the following elements: nominalization of beneficiary population, programming of activities to follow-up the beneficiaries, application of authorization protocol and request of equipment needs, supplies and forms to improve the attention according to the approved POA, provision of prioritized service portfolio, development of information, monitoring, supervision and assessment systems to comply with the accountability process of the Management Agreement. (iv) 100% of identified UBAs through the focusing and registration process of the beneficiary population of the primary attention service networks in rural areas have passed to the Equity and Strengthening improvement program of health services (PMEFSS) in the form of mobile teams and other teams in the form of combined networks. (v) The central MINSA has the updated study of costs per capita of the provision of the prioritized service portfolio, which is effective as from the second four-month period of 2012. The cost structure of the study will make its application in the fixed networks easier and will maintain the mobile network updated. (vi) The central MINSA has the guide to control the prioritized sanitary rules. (vii) The EEC information, monitoring, supervision, assessment and audit systems managed by the MINSA are generating important findings (good practices and/or deviations) to strengthen the operations and reliability of the results. (viii) The accountability of the providers and DRS has been gradually incorporated, acknowledging the legitimacy of its actions towards the MINSA and the value created to the beneficiary population.

Background General Purpose Reducing the inequity in the health sector by increasing the access to basic care services in poor rural areas and improving health results, which will benefit the more vulnerable groups of the population.

Specific Purpose

41

Increasing the access of underserviced rural beneficiary communities to basic health services, known for helping the improvement of the health of children and mothers.

Supporting the development of the strategic planning and the regulation and

monitoring mechanism known for improving the health system performance.

Relevant Indicators per Component Component 1 Development Indicator:

Goal for the 5th year of the Project: 60% of pregnant women that, at the end of the 3rd quarter of pregnancy have completed at least 3 prenatal controls (at least 1 for every pregnancy quarter).

o Progress made on the 2nd semester 6th year of the Project: A coverage of 66% is registered.

Goal for the 5th year of the Project: 80% of children of less than one year old with full vaccination scheme for their age.

o Progress made on the 2nd semester 6th year of the Project: A coverage of 87% is registered.

Goal for the 5th year of the Project: 55% of women delivering children with the

assistance of trained personnel from MINSA. o Progress made on the 2nd semester 6th year of the Project: A coverage

of 95% is registered. Intermediate Result Indicators:

Goal for the 5th year of the Project: 60% of health regions which reach more than 8 from the 10 indicators of agreed results.

o Progress made on the 2nd semester 6th year of the Project: Results are good, but the goals were substantially increased in the last year.

Component 2 Development Indicator:

Goal for the 5th year of the Project: 55% of individuals from rural areas diagnosed with diabetes and hypertension in need of medication receiving prescribed drugs by ambulatory primary healthcare units according to MINSA protocols. o Progress made on the 2nd semester 6th year of the Project: A coverage of

66% for diabetic patients and 60% for patients with high blood pressure is registered.

Goal for the 5th year of the Project: 80% of regional drug supply in health

centers, according to the basic chart for the first attention level.

42

o Progress made on the 2nd semester 6th year of the Project: the national average supply is 75.39%.

Result Indicators: Goal for the 5th year of the Project: 90% of health regions which have

completed the diagnosis and mapping of existing human resources, equipment and infrastructure. o Progress made on the 2nd semester 6th year of the Project: 100% is

registered.

Goal for the 5th year of the Project: 80% of health regions which comply with the facilities authorization plan of first attention level for the population focused on component 1. o Progress made to date on the 2nd semester of the 6th year of the Project:

0% progress is registered. With the beginning of the EFC and the UBA selection of the fixed network under the capitated financing model, the compliance with the authorization plans is expected.

Component 3 Development Indicator

Goal for the 5th year of the Project: 1st report on completed progress monitoring. o Progress made on the 2nd semester 6th year of the Project: The terms of

reference were reviewed by MINSA technicians in coordination with an Interinstitutional Board made up of the ICGES and the INEC. The terms are ready and are subject to the approval of the new authorities and waiting for financing sources.

Intermediate Result Indicators

Goal for the 5th year of the Project: Drawing up of the base line on the access of poor population to primary attention centers using the information furnished by the demography and health survey.

o Progress made on the 2nd semester 6th year of the Project: The terms of reference were reviewed by MINSA technicians in coordination with an Interinstitutional Board made up of the ICGES and the INEC. The terms are ready and are subject to the approval of the new authorities and waiting for financing sources. The aspects related to micronutrients were also incorporated to this activity in the Survey.

43

Goal for the 5th year of the Project: 80% of health regions using monitoring and information systems for determining the achievement of results of primary attention service providers.

o Progress made on the 2nd semester 6th year of the Project: The 12 participating DRS use the monitoring and information system for determining the achievement of results of its providers, reaching 100%.

Financial Execution

Loan Resources o With respect to the financial execution to December 31, 2014, we can

mention that accumulated disbursements received from the IBRD financing amount to US$40,0000.00, which correspond to 100% of the amount agreed with the World Bank.

o To the presentation of this report, investments made with contribution received from the IBRD amounted to US$37,020,261.25. The balance to be disbursed from the total received by the IBRD is executed this year 2015, since all amounts are committed and the Program is in the process of paying them.

Local Compensation Resources

o With respect to compensation resources, the contribution received amount to US$16,300,252 of the total agreed budget of US$18.7 million. To date, US$15,397,297.96 of the received resources have been invested. To execute 100% of the local contribution of the loan, the MINSA requested the MEF for 2015 a budget of US$2,429,748, with which all PMES commitments would be cancelled, which was not approved. In the light of this situation, MINSA will request an extraordinary credit.

Combined and pari passu resources

o The total contribution received by the Project, from both financing sources, amount to US$56,300,262, which represent 95.9% of the total budget of US$58.7 million. Accumulated contribution of compensation resources, for US$16,300,252 correspond to 29% of the total received contribution. If we consider that the agreed pari passu is 68/32, we observe that the contribution received from the compensation do not comply with the pari passu. However, this situation is expected to be overcame for 2015 with the allocation of the necessary resources to cover the local contribution commitments of the Program.

Accumulated investment statement

o During 2014, US$8,874,325.98 of the Agreement have been executed, 6,924,578.54 of which correspond to IBRD contribution and 1,949.747.44 of local contribution. The accumulated execution is US$.52,417,559.21,

44

US$37,020,261.25 correspond to the IBRD and US$15,397,297.96 to local contribution.

Budget Execution

o The PMES received from the Ministry of Economy and Finances a budget allocation for 2014 of B/.3,325,000. Due to the budget insufficiency to make the payments product of the commitments for the Project execution, the Ministry of Health requested the MEF an extraordinary credit to comply with these commitments. During June 2014, the MEF granted transfer of items and extraordinary credits necessary to execute the external contribution of the Project and substantially attain facing the local contribution commitments. The final budget was approved for the amount of B/.11.52.460. To date, 100% of the granted amount has been compromised, including the extraordinary credits of both financing sources.

Lessons Learned Constant and close follow-up and monitoring is required with service providers

and managers to comply faithfully with the quality rules and standards, as well as the observance of delivery times of accountability reports, which are the trigger for the disbursement and payment flows.

That it is necessary to comply exactly with timing in order to have audited information (ATE) with added value so that it provides us in due time with key elements to take decisions.

Continuous technical assistance to administrative staff of health regions in order to guarantee integration.

The number of reports subject to assessment exceeded the review capacity of the hired consultants, whereby additional technical staff has to be hired in order to conclude the assessments in due time.

As a limited number of monitoring and supervision activities was carried out, service providers performed many times incorrect processes or not within the provisions set forth in the Operating Regulations.

Critical Issues Decentralized payment process in Health Regions, which doubles the effective

payment time of commitments to providers. Lack of integration in the regional administrative structure for managing the

Strategy fund. The project assumed the hiring of national consultants to carry out the

monitoring, supervision and assessment functions of health service provision in the project. Since there were 47 population groups, each of which generated an initial report, a situation diagnosis, twelve monthly reports, six round reports, three four-monthly reports and an annual report, generating a total of 24 reports subject to assessment for each group, the load of activities of each consultant was focused almost exclusively on assessment duties. This resulted in weaknesses in the monitoring and supervision processes, since said

45

activities took second place when there were pending assessments because the beginning of the payment process to service providers depended directly on the assessment delivery date.

Recommendations Improvement opportunities:

Optimization of the registration IT system. Improve the payment process according to the reality of the rules of the

country and fully aware of the processes in order to avoid delays in the payment of suppliers and others.

Reframing of requested indicators and goals, not only on a quantitative basis but also in the creation thereof.

Use good practices of Health Regions through meetings and training among players who are part of the program management.

Standardize penalty days in findings of assessments and supervisions. The assessment process involves many manual steps, which delays the flow of

activities during the assessment. The information system used should be capable of issuing an assessment automatically. This activity may be developed by a less technical staff, thus allowing consultants to visit constantly the population groups assigned under their responsibility.

Review ATE and Social Audit processes and the Operating Regulations and adapt them in order to improve current processes.

46

Annex

DISTRIBUTION OF POPULATION GROUPS ASSIGNED TO PSPV – PMES

2009-2010 2011-2012 2013-2014

HEALTH REGION

TOTAL GI

GROUPS OE

GROUPS TOTAL

GI GROUPS

OE GROUPS

TOTAL GI

GROUPS TESTING GROUPS

OE GROUPS

TOTAL 47 47 0 47 21 26 163 21 116 26 BOCAS DEL

TORO 5 5 0 5 5 0 14 5 9 0

CHIRIQUÍ 7 7 0 7 0 7 38 0 31 7

COCLÉ 6 6 0 6 2 4 20 2 14 4

COLÓN 4 4 0 4 0 4 11 0 7 4

DARIÉN 4 4 0 4 4 0 11 4 7 0

HERRERA 2 2 0 2 2 0 10 2 8 0 LOS

SANTOS 2 2 0 2 2 0 12 2 10 0

PANAMÁ ESTE

2 2 0 2 2 0 6 2 4 0

PANAMÁ OESTE

6 6 0 6 2 4 15 2 9 4

VERAGUAS 9 9 0 9 2 7 26 2 17 7

MINSA, in the last semester when the service was provided (January - June 2014) by PSPV, maintained 163 population groups, 47 of which were trained with 5 Extra-institutional service providers and 8 Health Regions involved. This structure changed due to the fact the MINSA implemented a strategy to provide the service Portfolio with the same nominalization model in 116 sample population groups, i.e., not trained, in every Health Region. The table below shows the evolution of the group structure under the PMES.

47

GOALS OF ANNUAL AND FOUR-MONTHLY PERFORMANCE INDICATORS - PERIOD 2013-2014

No. Indicator Numerator Denominator Annual and Four-

monthly Goal Source

1

Percentage of pregnant women who, at the end of the 3rd pregnancy quarter, have completed at least 3

prenatal controls (at least 1 each quarter)

Number of pregnant women who, at the end of the 3rd pregnancy quarter, have

performed 3 prenatal controls (at least 1 each

quarter)

Total of pregnant women assessed in the

beneficiary population who, in the four-month

period, have reached the end of their third pregnancy quarter

100% HCPB Sheet

Prenatal control card

2 Percentage of pregnant women assessed before week 20 of pregnancy

Number of assessed pregnant women

Total of expected pregnant women in the beneficiary population

100% HCPB Sheet


3

Percentage of pregnant women with a second

doses or shot of TT or TD (appropriate vaccination according to standard)

Number of pregnant women with a second doses or shot

of TT or TD

Total of assessed pregnant women in the beneficiary population

100% HCPB Sheet


4 Percentage of births taken

care of by trained staff Number of births taken care

of by trained staff

Total of pregnant women assessed in the

beneficiary population who, in the four-month

period, have reached the end of their third pregnancy quarter

100% HCPB Sheet


5

Percentage of women of 20 years old or more, from

whom a sample of cervicovaginal cytology

was taken

Number of women of 20 years old and more from

whom a sample of cervicovaginal cytology was

taken in the twelve month period prior to the end of

the four-month period

Total of women of 20 years old and more in the

estimated beneficiary population

70% Clinical record, cytology control

card

6

Percentage of children under one year old with 4

or more growth and development controls

Number of children under one year old with 4 or more

growth and development controls

Total of children of the beneficiary population who have reached one

year old during the four-month period.

100%

Clinical record, growth and

development control card

7

Percentage of children under one year old with complete vaccination program for their age

Number of children who have reached one year old

during the four-month period with complete

vaccination program for their age

Total of children of the beneficiary population who have reached one

year old during the four-month period

100% Clinical record, vaccination card

48

GOALS OF ANNUAL AND FOUR-MONTHLY PERFORMANCE INDICATORS - PERIOD 2013-2014

No. Indicator Numerator Denominator Annual and Four-

monthly Goal Source

8

Percentage of children from 1 to 4 years old who have at least two growth

and development controls performed during the

assessed service period

Number of children who have reached 24, 36, 48 or 60 months old during the

four-month period of service provision and who have two or more registries with different dates in the growth and development

control cards during the last 11 months

Total of children who have reached 24, 36, 48 or 60 months old during the four-month period of service provision in the beneficiary population

100%

Clinical record, growth and

development control card

9

Percentage of children from 1 to 4 years old with

complete vaccination program for their age

Number of children who have reached 24, 36, 48 or

60 months old with complete vaccination program for their age

Total of children who have reached 24, 36, 48 or 60 months old in the beneficiary population

95% Clinical record, vaccination card

10 Percentage of assessed

respiratory symptomatic persons

Number of assessed respiratory symptomatic

persons

Expected respiratory symptomatic persons in beneficiary population

1%

Registry and follow-up card, clinical record and laboratory

registration book

11

Percentage of assessed diabetic patients, who have

received treatment, as established by MINSA

standards

Number of assessed diabetic patients, who have received treatment, as established by

MINSA standards

Number of assessed diabetic patients

65% Clinical record,

registry and follow-up card

12

Percentage of patients with high blood pressure, who

have received treatment, as established by MINSA

standards

Number of assessed patients with high blood pressure,

who have received treatment, as established by

MINSA standards

Number of assessed patients with high blood

pressure 65%

Clinical record, registry and

follow-up card

49

RESULTS OF ANNUAL SOCIAL AUDIT MEETINGS

REGION OF

HEALTH

POPULATION

GROUP

PERIOD 2011 - 2012 2012 - 2013

NOTE (ASSESSMENT)

PAYMENT SITUATION

OF 5%

NOTE (ASSESSMENT)

PAYMENT SITUATION

OF 5% Bocas Del Toro

BT - 1 3,85 60% Not applied (GI) Not paid BT - 2 4.68 100% Not applied (GI) Not paid BT - 3 2,90 Not paid Not applied (GI) Not paid BT - 4 5,00 100% Not applied (GI) Not paid BT - 5 3.38 60% Not applied (GI) Not paid

Coclé CC - 1 4,4 100% 4,5 100% CC - 2 4,6 100% 4,3 (AP) Pending

(09/14) CC - 3 4,7 100% 4,6 (AP) Pending

(09/14) CC - 4 4.8 100% 4.8 100% CC - 5 4,6 100% 4,1 100% CC - 6 4,5 100% 4,3 100%

Colón CL - 1 4,50 100% 4,3 100% CL - 2 4,50 100% 4,9 100% CL - 3 4,74 100% 4,6 100% CL - 4 4,62 100% 4,1 100%

Chiriquí CH - 1 4,72 100% 4,9 100% CH - 2 4,40 100% 4.53 100% CH - 3 4.67 100% 4.78 100% CH - 4 4.68 100% 4.86 100% CH - 5 4.85 100% 4.84 100% CH - 6 4.75 100% 4.08 100% CH - 7 4.82 100% 4.9 100%

Darién DA - 4 3,42 60% 3,5 (AP) Pending (09/14)

DA - 5 4.26 100% 4,4 (AP) Pending (09/14)

DA - 6 3.29 60% 1,6 (AP) Pending (09/14)

DA - 7 3.67 60% 4,6 (AP) Pending (09/14)

Herrera HE - 1 4.50 100% 4,3 (AP) Pending (09/14)

HE - 2 4.70 100% 4,5 (AP) Pending (09/14)

Los Santos LS - 1 4.50 100% Pending (09/14) Pending (09/14)

LS - 2 4.80 100% Pending (09/14) Pending (09/14)

Panamá Este

PE - 2 4.37 100% 3,9 (AP) Pending (09/14)

PE - 3 4.3 100% 4,5 (AP) Pending (09/14)

Panamá Oeste

PO - 1 4.34 100% 4.3 100% PO - 2 4.42 100% 4.8 100% PO - 3 3.84 60% 4.5 100% PO - 4 3.92 60% 4.5 100%

50

REGION

OF HEALTH

POPULATION

GROUP

PERIOD 2011 - 2012 2012 - 2013

NOTE (ASSESSMENT)

PAYMENT SITUATION

OF 5%

NOTE (ASSESSMENT)

PAYMENT SITUATION

OF 5% PO - 5 4.71 100% 4,5 (AP) Pending

(09/14) PO - 6 4.74 100% 4,7 (AP) Pending

(09/14) Veraguas VE - 1 4.83 100% 4.80 100%

VE - 2 4.29 100% 4.65 100% VE - 3 4.83 100% 4.9 100% VE - 4 4.77 100% 4,97 (AP) Pending

(09/14) VE - 5 4.95 100% 4.9 100% VE - 6 4.97 100% 4.98 100% VE - 7 4.94 100% 4,48 (AP) Pending

(09/14) VE - 8 4.95 100% 5.0 100% VE - 9 4.94 100% 4.9 100%

AP - Social Audit applied in central community or care center nearest to the social auditor house (Not applied for payment). GI - Failure to comply with minimum requirement of five comprehensive care visits.

51

Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders

N/A

52

Annex 9. List of Supporting Documents

July 2008. Project Appraisal Document (PAD) for a Panama Health Equity and Performance Improvement Project (Report N 42866- PA). Washington, DC

July 2008. Legal Agreement for the Panama Health Equity and Performance Improvement Project

Aide Memoires Mid-term Review Report Back to Office Reports Implementation Status Reports (ISRs) Restructuring Papers Borrower’s Evaluation Reports Project Progress reports as well as key Technical Assistance reports

Annex 10. MAP IBRD 33462R