document of the world bankdocuments.worldbank.org/curated/en/602881467777827597/pdf/icr3676... ·...

Document of

The World Bank

Report No: ICR00003676

IMPLEMENTATION COMPLETION AND RESULTS REPORT (IDA-50200)

ON A

CREDIT

IN THE AMOUNT OF SDR11.7 MILLION

(US$18.2 MILLION EQUIVALENT)

TO THE

REPUBLIC OF HONDURAS

FOR A

IMPROVING PUBLIC SECTOR PERFORMANCE PROJECT

June 28, 2016

Governance Global Practice Latin America and the Caribbean Region

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

ii

CURRENCY EQUIVALENTS (Exchange Rate Effective December 31, 2015)

Currency Unit = Honduran Lempira (HNL) HNL 1.00 = US$0.045 US$1.00 = HNL 22.13

FISCAL YEAR

January 1 – December 31

ABBREVIATIONS AND ACRONYMS CPF Country Partnership Framework CPS Country Partnership Strategy DPC Development Policy Credit FM Financial Management HRM Human Resource Management ICR Implementation Completion and Results Report IGR Institutional Governance Review IPSP Improving Public Sector Performance ISR Implementation Status and Results Report IT Information Technology M&E Monitoring and Evaluation MTEF Medium Term Expenditure Framework ONCAE National Procurement Office (Oficina Nacional de Contrataciones y

Adquisiciones del Estado) PAD Project Appraisal Document PCU Project Coordination Unit PDO Project Development Objective PEFA Public Expenditure and Financial Accountability PFM Public Financial Management PRSTAC Poverty Reduction Strategy Technical Assistance Credit SEFIN Ministry of Finance (Secretaría de Finanzas) SIAFI Integrated Financial Management System (Sistema Integrado de

Administración de Finanzas) SIAFI-GES New Integrated Financial Management System (New SIAFI) UAP Project Coordination Unit (Unidad de Administración de Proyectos) UATP Presidential Technical Support Unit (Unidad de Apoyo Técnico

Presidencial) UDEM Modernization Unit (Unidad de Modernización)

Senior Global Practice Director: Deborah L. Wetzel

Practice Manager: Arturo Herrera

Project Team Leader: Diego Dorado

ICR Team Leader: Svetlana I. Proskurovska/May Olalia

ICR Main Author: Marcelo Buitron

iii

HONDURAS Improving Public Sector Performance Project

CONTENTS

Data Sheet A. Basic Information B. Key Dates C. Ratings Summary D. Sector and Theme Codes E. Bank Staff F. Results Framework Analysis G. Ratings of Project Performance in ISRs H. Restructuring I. Disbursement Graph

1. Project Context, Development Objectives and Design ............................................................1

2. Key Factors Affecting Implementation and Outcomes ...........................................................4

3. Assessment of Outcomes .......................................................................................................12

4. Assessment of Risk to Development Outcome ......................................................................16

5. Assessment of Bank and Borrower Performance ..................................................................17

6. Lessons Learned .....................................................................................................................20

7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners ........................22

Annex 1. Project Costs and Financing .......................................................................................23

Annex 2. Outputs by Component ...............................................................................................24

Annex 3. Economic and Financial Analysis ..............................................................................29

Annex 4. Bank Lending and Implementation Support/Supervision Processes ..........................30

Annex 5. Beneficiary Survey Results ........................................................................................32

Annex 6. Stakeholder Workshop Report and Results ................................................................33

Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR ..................................34

Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders ....................................35

Annex 9. List of Supporting Documents ...................................................................................36

iv

A. Basic Information

Country: Honduras Project Name: Improving Public Sector Performance

Project ID: P110050 L/C/TF Number(s): IDA-50200

ICR Date: 06/28/2016 ICR Type: Core ICR

Lending Instrument: TAL Borrower: REPUBLIC OF HONDURAS

Original Total Commitment:

SDR 11.70M Disbursed Amount: SDR 3.45M

Revised Amount: SDR 3.79M

Environmental Category: C

Implementing Agencies: Finance Secretariat

Cofinanciers and Other External Partners: B. Key Dates

Process Date Process Original Date Revised / Actual

Date(s)

Concept Review: 02/23/2009 Effectiveness: 02/05/2012 12/15/2011

Appraisal: 10/13/2011 Restructuring(s): 12/15/2015 12/15/2015

Approval: 12/06/2011 Mid-term Review: 05/05/2014 06/16/2014

Closing: 12/31/2015 12/31/2015 C. Ratings Summary

C.1 Performance Rating by ICR

Outcomes: Unsatisfactory

Risk to Development Outcome: Negligible

Bank Performance: Unsatisfactory

Borrower Performance: Unsatisfactory

C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings

Quality at Entry: Moderately Unsatisfactory

Government: Unsatisfactory

Quality of Supervision: Unsatisfactory Implementing Agency/Agencies:

Unsatisfactory

Overall Bank Performance:

Unsatisfactory Overall Borrower Performance:

Unsatisfactory

v

C.3 Quality at Entry and Implementation Performance IndicatorsImplementation

Performance Indicators

QAG Assessments (if any)

Rating

Potential Problem Project at any time (Yes/No):

No Quality at Entry (QEA):

None

Problem Project at any time (Yes/No):

Yes Quality of Supervision (QSA):

None

DO rating before Closing/Inactive status:

Unsatisfactory

D. Sector and Theme Codes

Original Actual

Sector Code (as % of total Bank financing)

Central government administration 100 100

Theme Code (as % of total Bank financing)

Administrative and civil service reform 22 22

Managing for development results 6 6

Public expenditure, financial management and procurement 42 42

e-Government 30 30 E. Bank Staff

Positions At ICR At Approval

Vice President: Jorge Familiar Calderon Pamela Cox

Country Director: J. Humberto Lopez Carlos Felipe Jaramillo

Practice Manager/Manager: Arturo Herrera Gutierrez Arturo Herrera Gutierrez

Project Team Leader: Diego R. Dorado Hernandez Carolina Rendon

ICR Team Leader: May Cabilas Olalia/Svetlana Proskurovska

ICR Primary Author: Marcelo Buitron F. Results Framework Analysis

Project Development Objectives (from Project Appraisal Document)

The objective of the project is to strengthen the management of public finances and to establish a more efficient, effective, and transparent public procurement system through: (i) upgrading the public financial management system; (ii) upgrading the e-procurement platform; (iii) enhancing the internal control systems over personnel expenditures; and (iv) building capacity of the central administration.

vi

Revised Project Development Objectives (as approved by original approving authority)

(a) PDO Indicator(s)

Indicator Baseline Value

Original Target Values (from

approval documents)

Formally Revised

Target Values

Actual Value Achieved at

Completion or Target Years

Indicator 1: PEFA indicator PI-4, ‘Stock and monitoring of expenditure payment arrears’ - to obtain a rating of ‘A’ by the end of the project

Value (quantitative or qualitative)

B in the PEFA Report of 2011

A B+

Date achieved 11/01/2011 12/31/2015 03/31/2013

Comments (incl. % achievement)

Not achieved. The most recent national-level PEFA Report on Honduras was issued by the Millennium Challenge Corporation in March 2013. This report assigned B+ for PI-4 and B for PI-4 (i), ‘stock of expenditure payment arrears and any recent change in stock’; the same level it assigned in its PEFA Report of 2011 for both dimensions. Beyond the 2013 report, there is no available PEFA or PEFA-like internal evaluation at project closing to determine the current position. Project activities meant to contribute to this result were initiated only in 2014.

Indicator 2: Publication of bidding opportunities (including documents) and contract award information - to reach a level of 80% compliance.


Less than 10% of bidding documents are published

80% of bidding opportunities (including documents) and contract award information

Not possible to verify, but under target. Preliminary measures reveal that approximately 30% of a sample of contracts implemented between 2012 and 2014 are registered both at the Gaceta and HonduCompras.

Date achieved 11/01/2011 12/31/2015 12/31/2015 Comments (incl. % achievement)

Not achieved. Even though publishing documents is mandatory under Honduran law, contract award information is not published/registered automatically

Indicator 3: Improved control over payroll management in all central government institutions - through the implementation of payroll management audits’ recommendations


Weak controls over payroll processing in central administration institutions

Payroll management audit's action plans are fully implemented in all central government institutions

Action plans drafted but not implemented when project closed.

Date achieved 11/01/2011 12/31/2015 12/31/2015

vii


Not achieved. Payroll audits and functional reviews were done in select institutions. Action plans were drafted but not finalized nor implemented.

(b) Intermediate Outcome Indicator(s)

Indicator Baseline Value

Original Target Values (from

approval documents)

Formally Revised Target

Values

Actual Value Achieved at

Completion or Target Years

Indicator 1: Institutional coverage of the updated SIAFI.


Current framework complies with international best practice but needs to be updated. Covers 100% of central administration

100% of central government and 70% of decentralized government are using updated SIAFI

Only budget formulation module was implemented in 100% of central government and 70% of decentralized agencies

Date achieved 11/01/2011 12/31/2015 12/31/2015


Partially achieved. Only the budget formulation module was developed because the government decided to create a new SIAFI (SIAFI-GES) as opposed to upgrading the existing one without changing the underlying conceptual model as was originally negotiated and approved.

Indicator 2: Open Budget Index.


Score of 11 out of 100 in Open Budget Index

Improvement on Open Budget Index indicator (at least 50 out of 100). User satisfaction has improved as per user survey

Country scored 53 out of 100 in 2012 but dropped to 43 in 2015

Date achieved 11/01/2011 12/31/2015 12/31/2015


Achieved but not sustained. The 2015 score is a significant improvement from the baseline but fell by 10 points from its 2012 position and is below the end target. Additionally, there is no evidence that the customer survey was done during the life of the project.

Indicator 3: Additional functionalities to the SIAFI system implemented.


Some functionalities have not been implemented; others have been implemented partially (travel, HR, administration of goods)

All additional functionalities are in place and operating

No additional functionalities developed

Date achieved 11/01/2011 12/31/2015 12/31/2015


Not achieved. Only the budget formulation module was developed because the government decided to create a new SIAFI (SIAFI-GES) as opposed to upgrading the existing one without changing the underlying conceptual model, as was originally negotiated and approved.

Indicator 4: Accreditation of SIAFI users in central government institutions.

viii


No accreditation in place

All SIAFI users in financial management units of central government institutions are accredited

No accreditation process took place


Not achieved. Training activities on the budget formulation module at the conceptual level took place. This training covered over 1000 employees from all central government and 70% of decentralized agencies.

Indicator 5: New e-procurement platform implemented and at least 25% of procurement transactions carried out online.


No transactional e-procurement platform in place

At least 25% of procurement transactions carried out online

Only the bidding documents of the new platform were completed


Not achieved. The procurement agency (ONCAE) has no institutional capacity in place to support the development and operation of the new platform.

Indicator 6: Number of framework agreements implemented for common use goods. Value (quantitative or qualitative)

No framework agreements in place

6 framework agreements

6


Achieved. These 6 framework agreements covered over 5,000 products and reportedly generated savings of HNL 27 million (US$1.215 million) during 2015.

Indicator 7: Institutional strengthening of ONCAE.


Limited institutional capacity of ONCAE to fulfill mandate

Procurement units in all central government institutions are accredited/have received training in operation of new e-platform.

No accreditation took place


Not achieved. As new platform was never implemented, no accreditation took place.

Indicator 8: Enhanced control over personnel expenditure in identified high-risk areas.


Weak control over personnel expenditure in high risk areas

Implementation of corrective measures following findings of payroll audit for contracted personnel in the Ministry of

No corrective actions implemented

ix

Health and in the Ministry of Public Works are completed, leading to 3% savings in contracted personnel expenditures in both institutions on an annual basis


Partially achieved. Payroll audits were conducted in key entities: Education, Health, Infrastructure, and Security. Functional reviews were also contracted for these institutions plus SEFIN and Civil Service Directorate.

Indicator 9: Improved professionalization of public service in selected institutions.


Appointments to professional positions in key institutions are highly politicized. The Direccion Ejecutiva de Ingresos (Department of Revenues and Customs) has completed a revision of career and pay structures.

Activities supporting professionalization in one more institution is completed.

No professionalization activities were implemented.


Not achieved. This indicator relied on having an HRM strategy document elaborated by the second year, adopting its recommendations in SEFIN in the third year, and implementing similar recommendations in another institution.

Indicator 10: Number of sector cabinets using the indicators and goals established in the M&E system and basing their decisions on information provided by the system.


No M&E system in place

6 sector cabinets using the indicators and goals established in the M&E system; an increase in user satisfaction

Developed and implemented 4 sector cabinet strategies (i.e., Economic Development, Social Development, Productive Infrastructure, and Governance and Decentralization.)


Partially achieved. User satisfaction survey not implemented.

Indicator 11: Successful transition of project management.


No transition strategy Successful transition

Following the change in administration in early 2014, a Project Steering Committee was established to

x

help improve implementation. The Director of UAP-SEFIN assumed the role of the Project Coordinator and all fiduciary staff were maintained.


Achieved. While no transition document was ever drafted, a transition to the new administration took place.

G. Ratings of Project Performance in ISRs

No. Date ISR Archived

DO IP Actual Disbursements

(USD millions) 1 03/25/2012 Satisfactory Satisfactory 1.50 2 11/14/2012 Moderately Unsatisfactory Moderately Unsatisfactory 1.57 3 07/06/2013 Moderately Unsatisfactory Moderately Satisfactory 1.75 4 02/26/2014 Moderately Unsatisfactory Moderately Unsatisfactory 2.30 5 07/21/2014 Moderately Unsatisfactory Moderately Unsatisfactory 2.70 6 12/14/2014 Moderately Satisfactory Moderately Satisfactory 2.89 7 04/30/2015 Moderately Satisfactory Moderately Satisfactory 3.37 8 10/27/2015 Unsatisfactory Moderately Unsatisfactory 4.75 9 01/05/2016 Unsatisfactory Unsatisfactory 5.09

H. Restructuring (if any)

Restructuring Date(s)

Board Approved PDO

Change

ISR Ratings at Restructuring

Amount Disbursed at

Restructuring in USD millions

Reason for Restructuring & Key Changes Made

DO IP

12/15/2015 N U MU 4.75 Partial cancellation of SDR 7.9 million prior to closing

I. Disbursement Profile

1

1. Project Context, Development Objectives and Design

1.1 Context at Appraisal

1. Country Context. At appraisal, Honduras was one of the poorest and most unequal countries in Latin America. It had the worst social indicators in the Latin America and the Caribbean Region, which were exacerbated by the economic contraction in 2009, rising share of the population living in poverty (44.5 percent) and extreme poverty (20.1 percent), and high inequality (Gini at 0.57 in 2007). Crime and violence imposed high costs on the country, reaching an estimated 10 percent of GDP. Citizens perceived violent crimes as major threats to their welfare and the failure to address this has been undermining the legitimacy of the Government and the rule of law. Corruption, as measured by the Corruption Perception Index, was very strong resulting in Honduras’ ranking of 134 among the 178 countries evaluated in 2010.

2. Sector and institutional context. The 2009 Institutional Governance Review (IGR) Strengthening Performance Accountability in Honduras (Report No.53517-HN) documented a strong impact of vested interests on the country’s political and economic system and underscored (a) patronage and clientelism practices in public service management, (b) limited transparency in policy decision making, and (c) weak organization of civil society groups that represented core interests and rights of the citizens of Honduras. These governance challenges contributed to forming a society in which trust in the Government was at its lowest level. The IGR underscored the lack of incentives for political leaders to implement reforms and improve public services, and recommended that development assistance be targeted at the strengthening of transparency and accountability of governance processes, as well as at developing institutional capacity of the state.

3. To address weaknesses in the country’s accountability framework, the Government approved a Transparency and Anticorruption Plan for 2011–2014, aimed at supporting the achievement of key policy goals set in the National Plan (Plan de Nación, Decree 286-2009). Nevertheless, key challenges remained that are at the core of the ineffective, inefficient, and non-transparent public sector governance in the areas of management of public finances, public procurement, links between planning, budgeting, and accountability, public sector employment, and controls over public personnel expenditure. Institutional capacities, at appraisal, were deemed very weak in public procurement, planning, and human resource management (HRM). Moreover, the Honduras’ public financial management (PFM) platform (Sistema Integrado de Administration de Finanzas, SIAFI) needed technical upgrades to allow for greater access, reduced access costs, adequate security levels, to enable linkages with other information systems, and to complete tracking of expenditures and obligations. The HonduCompras, (the national procurement platform) was not a transactional platform, which did not allow for advances in improving the country’s procurement system. The institutional framework to formulate, monitor, and evaluate public sector programs and policies were outdated. HRM was the weakest area in public administration. The wage bill was increasing, primarily due to pay increases to teachers, lack of effective controls for payroll management, and creation of new positions. The Government took measures to contain the education sector wage bill but these efforts needed to be extended across the administration.

4. Government priorities. At appraisal, the Country Vision (Vision de Pais) 2010–2038, National Plan (Plan de Nacion) 2010–2022, and Government Plan (Plan de Gobierno) all shared a goal of creating a modern, transparent, responsible, efficient, and competitive state. In this regard,

2

the Improving Public Sector Performance (IPSP) Project was positioned to support the Government’s objectives of (a) implementing mechanisms for monitoring and evaluating public sector performance, (b) transparency in procurement, and (c) access to information for civil participation.

5. Rationale for World Bank Involvement. The World Bank had a long-term engagement with Honduras in supporting public sector management and more specifically PFM. Back in 2004, SIAFI was launched with World Bank support through the Economic and Financial Management Project (EFMTAC, P060785). Support continued with the follow-on project, Poverty Reduction Strategy Technical Assistance Credit (PRSTAC, P083951) which was co-financed with funds from the Swedish International Development Cooperation Authority, the U.K. Department for International Development, and the Government of Honduras. This project was also part of a broader donor coordination effort for public sector reforms in the country and this time with the Inter-American Development Bank, the German Development Bank (Kreditanstalt für Wiederaufbau), and the World Bank.

6. The Country Partnership Strategy (CPS, Report No.63370-HN) for the period of FY2012–2014 was designed to support the Honduras Country Vision and National Plan formulated by the new Lobo administration.1 The IPSP Project was one of the main instruments identified to support the goal of modernizing state institutions under the Strategic Objective 3 of the CPS (Enhancing Good Governance). In particular, it was the key instrument expected to deliver improved accountability in public expenditures (CPS Results Area 3.1).

1.2 Original Project Development Objectives (PDOs) and Key Indicators

7. The PDO was to strengthen the management of public finances and to establish a more efficient, effective, and transparent public procurement system through: (i) upgrading the public financial management system; (ii) upgrading the e-procurement platform; (iii) enhancing the internal control systems over personnel expenditures; and (iv) building the capacity of the central administration.

8. Progress toward achievement of PDO was to be measured using the following PDO-level indicators:

(a) PEFA indicator PI-4: ‘Stock and monitoring of expenditure payment arrears’2 - to obtain an ‘A’ rating by the end of the project.

(b) Publication of bidding opportunities (including documents) and contract award information - to reach a level of 80 percent compliance.

1 This was later extended through FY2015 to allow for the preparation of a new Country Partnership Framework with the change in administration in 2014. 2 PEFA Indicator 4 (P1-4). This indicator measures the extent to which there is a stock of arrears, and the extent to which the systemic problem is being brought under control and addressed.

3

(c) Improved control over payroll management in all central government institutions - through the implementation of payroll management audits’ recommendations.

1.3 Revised PDO (as approved by original approving authority) and Key Indicators, and reasons/justification

9. No changes were made to the PDO or PDO indicators during implementation.

1.4 Main Beneficiaries

10. The main beneficiary of the project was the Government and its staff, who stood to benefit from (a) more efficient business processes and improved information technology (IT) systems that would enable them to better perform their tasks, and (b) from increased knowledge and skills achieved through training. The private sector and citizens of Honduras were identified as secondary beneficiaries, who would benefit from enhanced transparency, better access to information, including that on the use of public resources, and enhanced possibilities of the private sector to participate in the public procurement process.

1.5 Original Components

11. Component 1: Strengthening and Consolidating Financial Management Systems (US$7.9 million). The objective of this component is to upgrade the technological platform of the SIAFI; strengthen its conceptual and functional framework to connect to the budget module, the public investment management system, and the procurement module (HonduCompras) to provide a more efficient processing and effectiveness to FM information; and increase the capacity of SIAFI operators. It was also expected to contribute to better cash management thus allowing the Government to reduce the stock of expenditure payment arrears. This component was to be implemented by the Modernization Unit (Unidad de Modernización, UDEM) in the Ministry of Finance (Secretaría de Finanzas, SEFIN).

12. Component 2: Strengthening the Public Procurement System (US$4.7 million). The objective of this component was to support the Government’s initiative to create a regulatory body with specific goals and objectives to strengthen the overall governance of the procurement system and to improve transparency and compliance. Activities included the development of a module to link the procurement system to SIAFI and standard procurement plan templates; preparation of an institutional strategic plan for the National Procurement Office (Oficina Nacional de Contrataciones y Adquisiciones del Estado, ONCAE); creation of a statistical unit to monitor and evaluate performance of the procurement system; creation of a unit specialized in the development implementation of framework agreements; development of standardized procurement materials; and design and implementation of a new e-procurement platform (acquisition of software and hardware, installation and configuration, and training of system users). This component was to be implemented by ONCAE in SEFIN.

13. Component 3: Improving Public Sector Human Resource Management (US$3.7 million). The objective of this component was to improve HRM in the public sector by enhancing controls on overall personnel expenditures and improving the attraction and retention of qualified personnel to increase professionalization of the public service in selected institutions. Activities were to include: the implementation of process-based payroll management audits in selected

4

ministries and the development of a political economy and communications strategy to communicate the results; human resource diagnosis to support efforts to attract and retain qualified personnel; and technical assistance related to the certification of positions, skill gap assessment, external validation of recruitment or promotion processes; and update of job profiles. Implementation of this component was split between the Project Coordination Unit (Unidad de Administración de Proyectos, UAP) in SEFIN and Presidential Technical Assistant Unit (Unidad de Apoyo Técnico Presidencial, UATP) in the Secretariat of the Presidency.

14. Component 4: Strengthening Government Monitoring and Evaluation Capacity (US$0.8 million). The objective of this component was to support the implementation and institutionalization of a results-based monitoring and evaluation (M&E) system for selected sector cabinets. Activities under this component were to include the conceptual and functional design of the results-based M&E system that involved the design and implementation of a new technological platform for an M&E system; development of indicators and targets for Government ministries; design and implementation of an incentive system to promote use of performance information; the definition of the institutional arrangements among ministries for purposes of operating the M&E system; and strengthening the technical managerial and organizational capacities of the Ministry of Presidency (Secretaria de Estado en el Despacho de Finanzas) to implement the M&E system. This component was to be implemented by the UATP in the Secretariat of the Presidency.

15. Component 5: Support to Transition and Project Coordination (original US$1.1 million). The objective of this component is twofold: to support (a) the administration of the project including all fiduciary aspects and regular reporting of overall project advances and status of indicators; and (b) design and implement a transition strategy. The UAP-SEFIN was responsible for project coordination while the UATP is responsible for the transition strategy.

1.6 Revised Components

16. No revisions to project components were made during implementation.

1.7 Other significant changes

17. A restructuring for a partial cancellation of more than 60 percent of the original IDA credit proceeds in the amount SDR 7.9 million was processed on December 15, 2015. This partial cancellation was meant to channel these unspent funds to financing other IDA operations in the country. This also led to the reduction of the proceeds allocated to each component.

2. Key Factors Affecting Implementation and Outcomes

2.1 Project Preparation, Design, and Quality at Entry

18. Soundness of background analysis. Recommendations of the 2009 IGR were considered and applied. The project built on the following lessons learned from the implementation of the PRSTAC Project: Lesson 1 - the operation should be focused on a few key stages in the policymaking process and on performance; Lesson 2 - project design and outcomes should be based on realistic expectations, informed by the country context and be commensurate with the implementation timeline; and Lesson 3 - the implementation arrangements should reflect the capacity constraints encountered during the PRSTAC Project.

5

19. Lesson 1 was reflected in the design when identifying the discrete content areas. It was more focused on core public management areas, such as the PFM system, public procurement capacity, personnel expenditure controls, and sector strategies and indicators to monitor public sector performance. Technical assessments of the problems and the current situation of each core area were made to inform the design and choice of activities to be supported. For example, in the area of PFM, studies were conducted, which concluded that: (a) the functional design of SIAFI was viable; (b) the technology architecture required improvements to enable a stable operation; and (c) the technological platform was outdated, was no longer covered by a warranty, and needed to be replaced by new technology solutions. Hence, Component 1 was positioned to support the technological upgrade of SIAFI and to improve its conceptual and functional framework to provide more efficient processing and effective access to financial information.

20. However, it failed to apply lessons 2 and 3 adequately. With respect to lesson 2, it failed to have a realistic timeline for project implementation. The selection of activities could have been given more critical consideration from the political economy analysis and implementation capacity angles. On the other hand, with respect to lesson 3, while it improved the implementation arrangements when compared to the PRSTAC arrangements, the arrangements proved to be inadequate to advance project implementation as discussed in more detail below.

21. It should also be noted that the ousting of President Zelaya in June 2009 interrupted the preparation of the originally proposed State Modernization Project until the time that the World Bank reengaged with the newly elected government of President Lobo in 2010. A new Project Concept Note for the project—renamed the Improving Public Sector Performance Project —was developed. The project was prepared in parallel with the CPS for 2012–2014 (Report No. 63370-HN) and the First Programmatic Reducing Vulnerabilities for Growth and Development Policy Credit (P127331), thus allowing for a strong alignment of the project with the World Bank’s medium-term strategic goals in Honduras and development policy support. All three documents (CPS, IPSP, and the Development Policy Credit, [DPC]) were presented to the Board as a single package and approved in December 2011. Moreover, the IPSP was to provide technical assistance to the series of DPCs in the area of HRM.

22. Assessment of the project design. The project was designed to address critical shortcomings noted in core public sector management areas with the overarching goal of improving public sector performance through enhanced efficiency, transparency, and accountability in the areas of PFM, procurement, HRM, and enhancing M&E systems. It focused on addressing very specific challenges and activities that had the buy-in of key stakeholders and avoided duplication of initiatives already being supported by other donors. Its main interventions were to focus on improving processes and upgrading the functional and technological frameworks of current IT systems. In choosing the activities, the design also took into consideration the high likelihood that the majority of these activities could be implemented within the first two years of project implementation given the expected Government transition in early 2014. Project design was reflected in the PDO, results framework, the project components, and selected project activities as well as in the project timeline. The design was well-intentioned, but proved to be too optimistic given the evolution of Government commitment after effectiveness, capacity constraints, and the implementation experience discussed in the next section. On hindsight, the project could still have been further simplified, including the formulation of the PDO to eliminate

6

reference to the required outputs to allow for flexibility of restructuring without having to go to the Board.

23. Moreover, the fact that the project financed the creation and/or revamping of IT systems, and given the difficult political context and weak capacities, the project length (four years) was very short based on what literature suggests. Evidence from World Bank-financed integrated FM system projects suggest that these last on average 7.9 years, with the majority of them falling in a five- to ten-year range.3 The length of the project was also particularly problematic as the country’s limited fiscal space and procurement processes augured long bidding processes that would have been hard to achieve on time. Furthermore, reforms of this nature usually require significant change management campaigns which except for the communications on the results of the payroll audits was conspicuously absent in the other components.

24. Implementation arrangements were significantly enhanced compared to the previous operation by creating a unit for project coordination that was informed by a consultative process. However, the lack of clear leadership for the overall project remained unaddressed given the multiple agencies involved.

25. Adequacy of Government’s commitment. The design of the project was a product of a close coordination with the Secretariat of the Presidency, which through a specially appointed presidential designated official articulated project priority areas that responded to identified development challenges. On the other hand, the engagement with SEFIN at preparation was limited to the definition of the scope of needs related to the technological upgrade of SIAFI and developing interfaces, despite the fact that implementation of most critical technical components4 would be in SEFIN’s purview. There was no single sponsor and champion for the overall project nor a clear understanding of their accountabilities at the implementation stage.

26. Assessment of risks. From the concept stage, the implementation of the project has always been rated high risk because of the volatile political context, expected change in political priorities as a result of the change in administration, implementation slowdown during the electoral period, capacity constraint given the limited technical capacity and fiduciary staff relative to the expected complex procurement of IT packages, high country procurement risk and the involvement of multiple agencies whose priorities may not always be aligned and could potentially be worsened by a potential reshuffle in Government. Recognizing these, mitigation measures were introduced from the preparation phase such as putting in place an operations manual and procurement plans, procurement documents initiated at preparation, fiduciary training including handholding from World Bank fiduciary staff based in Honduras, the hiring of a project coordinator and fiduciary staff whose sole focus is the project to be complemented from the World Bank side with the provision of close implementation and technical support. In addition, to address stakeholder risks and the potential Government reshuffle resulting from change in administration, a transition strategy was to be put in place. Moreover, to address the risk of delivery monitoring and sustainability, achievement of results indicators was to be closely monitored by having a separate

3 For more information see “Financial Management Information Systems, 25 years of World Bank Experience on What Works and Doesn’t”. Dener, Watkins, and Dorotinsky (2011) 4 Apart from component 1, other critical technical components under SEFIN’s responsibility were Component 2 – Public Procurement System, Component 3 – Human Resource Management except subcomponent 3.2.

7

section on them in the Aide Memoires and annual operational audits on the results indicators were to be deployed. Key substantial and high-risk areas materialized during implementation that led to a significant start-up delay and compromised project completion moving forward. In fact, even before the change in administration, political priorities/commitment and stakeholder buy-in to key project activities changed as demonstrated by SEFIN’s initial opposition to the agreed approach to upgrading SIAFI and the non-advancement of HRM reforms under Component 3 during the first two years of the project. Implementation capacities for Component 2 were immediately compromised with the almost en masse resignation of ONCAE staff when the agency was re-mapped to SEFIN and salaries, therefore, needed to be aligned with those of SEFIN staff, and even when this was addressed,5 ONCAE still continued to lack capacity to implement the upgrade of the e-procurement platform. Project implementation did slow down during the electoral period and was manifested by politically sensitive reforms such as those for HRM not being pursued and the SIAFI upgrade being put in limbo. The risk of delivery monitoring and sustainability was a substantial risk that also materialized, which compromised the ability of the project to make course corrections during implementation.

2.2 Implementation

27. The shift in commitment and political will of the Government and its executing agencies while operating in a context of the electoral period critically compromised project implementation from start-up. Following effectiveness, only the UATP demonstrated its continued commitment to implement activities within its purview under Component 4 which succeeded in enabling three sector cabinets (out of a target of six) to develop and implement sectoral strategies with indicators and M&E systems during the first two years. Significant changes in government commitment at start-up for Components 1, 2, and 3 that were highly critical to the achievement of the PDO and against an already tight implementation schedule led to significant slippages during the first two years and eventually to non-completion of expected outputs and consequently to non-achievement of desired outcomes at exit.

28. Under Component 1, the approach to upgrade SIAFI as appraised was challenged by SEFIN soon after effectiveness. It insisted on abandoning SIAFI, as a technologically non-viable system, and proposed that SIAFI should be replaced by a specific off-the-shelf solution based on an SAP operational system to be purchased on a sole-source basis. The World Bank objected to this approach from a technical point of view, as this was not the technical solution that best responded to the challenges of the current system and the current needs of the country. The World Bank also did not provide a ‘no objection’ to acquire this platform on a sole-source basis as World Bank policy mandates international competitive bidding for contracting the supply and installation of multi-million dollar information technology systems. In 2012, there were two changes in SEFIN leadership. Both agreed to proceed with the approach as appraised by the World Bank. However, while the UDEM was refining and clarifying their requirements for an upgraded SIAFI, SEFIN’s leadership was more focused on the upcoming election and no clear decisions were taken to advance under this component.

5 Initially through financing of consultants under the Project.

8

29. Under Component 2, SEFIN’s commitment to the reforms related to ONCAE was unclear. After ONCAE’s move under SEFIN from the Secretariat of the Presidency, the salaries of the ONCAE staff were revised downward to align with those of SEFIN. As a consequence, almost all ONCAE staff resigned and the World Bank had to exceptionally approve financing of consultants to fill in the gap in the fall of 2012 that continued through August 2013. Even with this exceptional arrangement, the ONCAE team, supported with consultants, was only able to advance in the area of framework agreements: (a) implementation of four out of seven framework arrangements, (b) establish the organization unit on framework agreements; (c) the enactment of the Electronic Procurement Act regulating the use of framework agreements; and (d) the preparation of standard procurement documents. In the end, ONCAE was unable to establish a new e-procurement platform—the reform that was critical to the PDO—neither during the first two years nor after.

30. The implementation of Component 3, took a long time to gain political support and get off the ground. With the Government getting into the election period, HRM reforms were too politically charged to pursue. Moreover, the project was envisaged to accompany the First Programmatic Reducing Vulnerabilities for Growth Development Policy Credit (P127331) particularly with respect to providing the technical assistance for HRM reforms. It was also expected that through the programmatic DPC, commitment to pursuing the reforms would continue. However, due to slippages in triggers for the follow-on DPC and deteriorating fiscal performance, no follow-on DPL was pursued. As such, the IPSP Project lost a mitigating measure identified in the Project Appraisal Document (PAD) for ensuring continuity of support for the reforms it was supporting, particularly in improving HRM in the public sector.

31. The transition to the new administration had a positive impact on the project. As anticipated, a change in Government took place in early 2014 following the November 2013 election. However, contrary to expectations at preparation, where a change in Government could disrupt implementation, the transition to the new administration had significant beneficial influence to the project, in general. To a certain extent, the change in administration was a catalyst in clarifying and, more importantly, in re-confirming the Government commitment to the project. A Project Steering Committee was established by the Government to address implementation bottlenecks, the project coordinator was removed, and the director of UAP-SEFIN took over his role. With the Project Steering Committee, a more comprehensive and better monitoring of project implementation was put in place. However, there were still shortcomings and the significant delay that the project suffered could not be overcome.

32. Under the new Administration, the way forward for upgrading SIAFI to the new Integrated Financial Management System (SIAFI-GES) through a combination of in-house and external experts was articulated to be developed with a conceptual framework based on the further analysis undertaken on SIAFI and building on the Inter-American Development Bank-supported Integrated Municipal Management System (Sistema de Administración Municipal Integrado) platform. However, this approach resulted in longer implementation schedules, which could no longer be accommodated within the project lifetime. As a consequence, the World Bank, after reviewing the new approach on its technical merits, decided that the best way forward was to support the budget formulation module as it was the most pivotal one for SIAFI-GES and could be completed before the project closed. Eventually, UDEM with the assistance of individual consultants funded under the project developed, tested, and launched in production a new budget formulation module that was extended to all central government entities and used for preparation of the 2016 budget. Other

9

modules of SIAFI are still to be developed in the course of 2016 and onward, potentially with the Government’s own resources, estimated at a level of US$2 million.

33. With respect to ONCAE, it was remapped again to the Secretariat of the Presidency albeit this time under a newly created unit—Presidential Directorate of Transparency Reform and Modernization—that was keen on developing the role of ONCAE as a regulatory body but due to time constraint and delays in approval of ONCAE’s organizational structure, the project failed to advance in this area. Nevertheless, the new administration could not resolve the inherent limited technical capacity in ONCAE to enable it to deliver a new e-procurement platform.

34. Commitment to Component 3 was resurrected and gained sufficient attention. Implementation of the agreed activities started only in 2014 under the new administration with its strong commitment to control the public sector wage bill in efforts to manage the fiscal constraints. While this led to advances on completing payroll audits and a functional review that recommended action plans, the project was only able to complete these close to closing and consequently the recommended corrective actions have yet to be implemented.

35. The execution responsibilities for Component 4 were also moved from the UATP to the Presidential Directorate of Strategic Planning, Budget, and Public Investment. This allowed one additional sector cabinet to develop sectoral strategies and indicators.

36. The midterm review for the project was conducted within the first half of 2014 and essentially endorsed the new administration’s priorities.

37. The lack of technical capacity in ONCAE was key to the failure to upgrade the e-procurement platform and its functioning as a regulatory body. ONCAE did not have the capacity to develop the new e-procurement platform. The World Bank tried to assist ONCAE by arranging for a shared services agreement between ONCAE and UDEM where UDEM took over the responsibility for linking HonduCompras with SIAFI and for assisting in implementing the upgrade of HonduCompras as well as maintaining it thereafter. However, this arrangement did not work and so in the absence of qualified information technology personnel, ONCAE resorted to hiring a consultant for development of the bidding documents for the system solution. When the bidding documents were finally ready, it was no longer feasible to launch the procurement process for HonduCompras (as documented in the October 2015 Implementation Status and Results Report [ISR]) due to the lack of time before project closing. Presently, SEFIN administers the technological operation of the existing HonduCompras, although ONCAE is no longer part of SEFIN´s institutional structure.

38. Moreover, under the new administration, efforts to enhance the capacity of ONCAE in its role as a regulatory body were considered including taking on a role similar to an analytical think tank that will allow for an analysis of the procurement database and provide evidence-based inputs to policy makers on improving procurement efficiency, particularly in a context of severe fiscal constraint. As such, a new organizational structure for ONCAE was prepared, but approved too late to support its implementation under the project.

39. The lack of leadership and high-level oversight for the overall project contributed to its failure. While a PCU was established to address the challenge of working with multiple

10

agencies, this was inadequate. The PCU did not have the influence to pressure the executing agencies to implement the project activities. This was a shortcoming at the design stage and it manifested itself during implementation. It was only upon the assumption of the new administration that a Project Steering Committee was established with SEFIN’s IT adviser as the head. With the Project Steering Committee, a higher-level oversight was put in place. Nevertheless, delays on follow-up action for pending items discussed by the Project Steering Committee persisted. There was a clear need for leadership and accountability, for the overall project to ensure that it was progressing satisfactorily.

40. There were missed opportunities for restructuring. Persistent perception of high corruption in Honduras and lack of institutional capacity created a very difficult implementation environment in which proposed reforms cut against the embedded interests, which led to delays and incomplete implementation of Components 1, 2, and 3. As a consequence, implementation delays would be expected that would naturally lead to consideration for project adjustments/restructuring. During the lifetime of the project, restructuring was mentioned as a possibility, especially upon the assumption of the new administration albeit late in the process, including contemplating a possible extension of the closing date after the midterm review. However, these were not pursued. In the midterm review report, the Bank concluded that changing the components was not necessary and instead contemplated project extension, which was not requested by the Government in the end. No reflection was made on the actual significant delay in achieving any of the results indicators, which in itself should have justified a restructuring to recalibrate the project and allow it to have a more realistic set of outcomes against what could be feasibly completed at exit. Instead, only a partial cancellation was processed at the request of the Government to re-channel funds to other operations.

2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization

41. When it comes to the design of M&E arrangements, the project from the onset had targets set to be monitored by the executing agencies and consolidated by UAP-SEFIN. As most of the values consisted of achievements of activities—primarily output oriented, no complex statistical data were ever needed, just proof of achievement through internal documents and progress reports. The indicators chosen were linked to the objectives and the activities lead to a causal link to the PDO through the intermediate indicators. However, in hindsight, the PDO formulation was so specific that any slippage or change in the envisaged outputs listed in the PDO would automatically trigger a level 1 restructuring.

42. As discussed earlier, the risk associated with delivery monitoring and sustainability was rated substantial at the implementation stage. In this regard, three mitigation measures were identified in the PAD’s Operational Risk Assessment Framework: (a) conduct of yearly compliance audits, the requirement for which was a covenant in the Financing Agreement; (b) supervision Aide Memoires were to have a results indicator section; (c) close dialogue with the Government, including a midterm review allowing for adjustments to new priorities.

43. Under these circumstances, the M&E design for the most part could be considered substantial.

11

44. However, M&E implementation and utilization at best can be considered modest. M&E implementation only started in later 2013 after an M&E specialist was hired in the UAP-SEFIN. Thereafter, progress of results were reported and then captured in the World Bank’s ISRs with one notable exception. One PDO indicator (payment arrears) relied on the verification done by a Public Expenditure and Financial Accountability (PEFA) assessment. After the 2013 PEFA Report was issued by the Millennium Challenge Corporation, no follow-up PEFA or PEFA-like evaluations were conducted. Finally, the monitoring of the results indicators did not appear to have informed the Government to trigger required adjustments and attention to bottlenecks that affected implementation until the Project Steering Committee was created in April 2014 under the new administration. In the early years of implementation, decisions appear to have been more politically driven. Furthermore, the risk mitigation measures for the delivery, monitoring, and sustainability risk categories were not fully implemented and therefore failed to sufficiently inform decision making for project course correction. For example, results in the Aide Memoire were not consistently covered albeit reported in the ISRs since 2013. The conduct of annual operational audits to validate results was never done—a legal covenant that was consistently not complied with until the conduct of one single audit in 2015. Given these shortcomings, both the World Bank and the Government failed to have a regular validation of the project outcomes and, therefore, timely discussions on course corrections were not conducted.

45. Furthermore, a summary of the operational audit finally conducted in 2015 concluded that given the low disbursements and the poor achievements in the results framework, the project needed to adopt a new implementation strategy. This included creating coordination mechanisms between the PCU and implementing agencies as well as formalizing duties and responsibilities. It also suggested revamping the results framework to better capture the objectives and goals of the components. The report also suggested updating the operational manual. It also encouraged that the Aide Memoires include annexes with detailed status of agreements and the status of actions agreed during previous missions. Had this been done annually, the operational audits as envisaged could have helped in improving course correction in the project trajectory and proactivity.

2.4 Safeguard and Fiduciary Compliance

46. Safeguards. The project is a category C project with no safeguards issues. It must be noted that when the project was approved by the Board, OP 4.01 - Environmental Assessment was triggered. This was because a Disaster Recovery Center was going to be refurbished. This entailed minor rehabilitation work and the team developed an Environmental Management Framework. This activity never took place.

47. Financial Management. In general, the project’s FM performance has been rated either Moderately Satisfactory or Satisfactory and it exited with a rating of Satisfactory. Based on the FM supervision, carried out during the project life, it was concluded that (a) FM provided reasonable assurance that the credit proceeds were used for the intended purposes; (b) the FM-related arrangements allowed an appropriate level of transparency that facilitated oversight and control; and (c) the legal FM-related covenants were met, including timely submission of interim financial reports and annual financial audits, with the exception of submission of annual operational audit reports during the first two years of project implementation.

12

48. During implementation, some additional FM-related issues identified and followed-up during the project implementation were (a) low level of budget execution and insufficient budget allocations, given the country-level fiscal constraints; (b) slow disbursement was partially affected by the implementation of additional controls, instituted by SEFIN, related to budget execution and payments; and (c) limited accuracy of financial information presented in the project interim financial reports and annual financial audits, which eventually improved before project closing.

49. Procurement. For the first two years of the project, procurement performance was rated Moderately Unsatisfactory due to delays in implementation. This turned around in mid-2014 with the project exiting with a Satisfactory rating. Throughout the life of the project, no major issues arose, although the risk rating has been considered high from the beginning. The due diligence and proactivity exercised by the Bank, prevented the project from engaging in direct contracting for implementation of a new SIAFI. Also, close monitoring of proposed changes to the procurement plans in the early stages of the project helped to instill adequate attention on the part of the Government to managing procurement in line with the guidelines. For the life of the project, the UAP was well versed in the World Bank’s policies and submitted necessary documents on time.

2.5 Post-completion Operation/Next Phase

50. There are currently no planned follow-up World Bank-supported operations to the IPSP Project. The Government committed to continue working on the remaining modules of SIAFI-GES and is keen on implementing the recommendations of the human resource action plan but funding for these activities has yet to be identified.

3. Assessment of Outcomes

3.1. Relevance of Objectives, Design, and Implementation

51. As argued earlier, the project objectives remained consistent with the Government´s priorities and the World Bank´s CPS FY12–14, during project implementation. The design of the project attained modest relevance during its lifetime. Soon after effectiveness, it was clear that even though the objectives were very much aligned with the needs of the country, the implementation arrangements were not as robust as expected and the activities set forth in most of the components were not viable as originally envisaged.

52. With respect to the Government’s own strategy, ‘Plan for a Better Life’, that is in place for 2014–18, the IPSP Project remains relevant and its objectives fit within the plan’s pillar of enhancing transparency and modernization of the state.

53. The FY16–FY20 Country Partnership Framework (CPF, Report No. 98367 HN) acknowledged the limited results of public sector reform under the prior CPS and set forth areas of engagement that are linked to institutional strengthening. Under the CPF, the importance of building institutions and increasing institutional capacity continues to be recognized and are now seen as cross-cutting themes across the three pillars of the CPF. Nevertheless, there appears to be less appetite for a similar discrete operation that supports governance systems in the upstream. In this regard and as discussed earlier, the IPSP Project’s modest achievements informed Prior Action 4 of the First Fiscal Sustainability and Enhanced Competitiveness Development Policy Financing (P155920) to Honduras that was approved in December 2015.

13

54. In this regard, while relevance of project objective remains substantial given its alignment with national development plans and the current CPF, the relevance of design and implementation is modest at best considering it does not appear to be a preferred instrument of engagement at this point.

3.2 Achievement of Project Development Objectives

55. The objective of strengthening PFM and establishing a more efficient, effective, and transparent public procurement system was not achieved because of the failure to complete the critical inputs of (a) upgrading the PFM system; (b) upgrading the e-procurement platform; (c) enhancing the internal control systems over personnel expenditures; and (d) building capacity of the central administration. Regarding the upgrade of the PFM system, the efficacy is modest. No significant upgrade materialized, a new system was proposed, and only the budget formulation module was implemented, although it had better features than the previous SIAFI as it is linked to the Medium Term Expenditure Framework (MTEF). With this better feature, the Government is supposed to have a better budget control mechanism with a comprehensive view of the commitments made over time while avoiding the creation of new commitments without enough funding.

56. With respect to the other parts of the PDO, efficacy is negligible. The upgrade to the e-procurement platform was never implemented although some efficiencies have been reported but not due to a new procurement platform as discussed in the Efficiency section. Regarding the control over personnel expenditures, these never materialized, albeit functional reviews and action plans were drafted. Finally, the capacity of the central administration was somewhat strengthened by various workshops and trainings that were financed under the credit, but have little impact to the overall PDO and objectives, therefore the efficacy rating is negligible.

57. Moreover, external indicators on the Government’s performance have not really shown much improvement. Compared to its 2010 standing, its Corruption Perception Index remains below the bottom 30 percent of countries ranked by Transparency International in 2015 at 128 compared to 168 countries.6 Furthermore, Honduras’s aggregate indicator ranking for Government Effectiveness under the Worldwide Governance Indicator has been worsening from 36.5 in 2011 to 20.7 in 2014.7 Hence, efficacy can only be rated negligible. At closing, the project cannot claim to have strengthened management of public finances nor efficiency, effectiveness, and transparency in the public procurement system.

58. With respect to transparency, the Government’s Open Budget Index score significantly rose from its baseline value of 11 in 2010 to 43 in 2015. However, the 2015 score is 10 points lower than its previous score in 2012 of 53. The drop from 2012 was accounted for by the lack of comprehensive budget information made available to the public during the formulation process. To address this shortcoming, SEFIN implemented a business intelligence tool to perform queries on SIAFI through SEFIN’s web page and is planning to enable a consultation module with pre-established reports that will be made public on the SIAFI-GES web page.

6 Transparency International, Corruptions Perception Index 2015. 7 Worldwide Governance Indicators, Country Data Report for Honduras, 1996–2014.

14

59. With respect to the three PDO indicators, for the most part, these were not achieved. PDO Indicator 1 sought to improve stock and monitoring of expenditure payment arrears (PI-4). Its target was to obtain an ‘A’ in the respective PEFA indicator. The impact of the project on this indicator cannot really be measured due to the delay in the implementation of SIAFI-GES, which as of completion only had the budget formulation module implemented. The module was operationalized in 2015 and its impact on PI-4 could therefore not be measured yet. Moreover, beyond the PEFA 2013 report prepared by Millennium Challenge Corporation, no other update is available. A PEFA-like assessment by the Government was considered but not implemented and none is expected in the near future. Nevertheless, while the budget formulation module would not help in day-to-day management of the expenditure payment arrears, its features with its link to the MTEF would allow the Government to plan and anticipate for future expenditure management arrears only if the information were used as the basis for the decision-making purposes. Further, with its link to the MTEF, the budget formulation module, would enable the Government to make decisions that are informed both by monetary and development results consideration for each Government project/activity.

60. With respect to the PDO Indicator 2 that aimed at publishing 80 percent of bidding opportunities (including documents) and contract award information, Honduras law mandates that all bidding documents must be published. The Government also reported that 100 percent of these have been published. However, a preliminary measure requested by the World Bank suggests that from 2012–2014 only 30 percent of contracts were registered both in the Gaceta and HonduCompras.

61. Finally, PDO Indicator 3 sought to improve control over payroll management in all central government institutions. This was going to be achieved by having payroll management audits, followed up by the implementation of action plans in all central government institutions. The payroll audits were conducted but because this was done toward the end of the project, the actions plans could not be implemented. Nevertheless, and not due to the project, the latest CPF mentions how the Government was successful at reducing expenditures through greater fiscal consolidation and more discipline regarding the wage bill and procurement, but did note that these are areas where work is still needed.

62. Annex 2 provides more details on similar limited progress in intermediate outcome indicators and status of completion of expected outputs.

3.3 Efficiency

63. As this was a technical assistance project devoted to institutional strengthening, computations of rate of return, cost effectiveness, and other parameters are difficult to quantify. Nevertheless, it was expected to have a positive fiscal impact and generate economic gains that were to come from a more efficient use of public finances through improved public procurement processes, proper budget execution, and enhanced controls of the public sector wage bill. In this regard, a few indicators of efficiencies can be identified.

64. The Government reported that as a result of the six framework agreements implemented it has saved HNL 27 million (US$1.215 million or almost double the investment under this Component 2 but negligible compared to the size of the 2015 national budget of HNL 185 billion)

15

in 2015 alone covering 5,000 products for 70 different government institutions. These are nominal savings and the methodology used to calculate the savings has not been verified by the World Bank.

65. Likewise, by renting a space in a commercial data center instead of establishing its own physical Disaster Recovery Center for SIAFI, the Government was able to gain a data center to mitigate for business continuity disruptions at a lower cost. Original allocation was for US$2 million to establish the physical center; instead, they spent US$200,000 in consultancy services (about 10 percent of the estimated cost).

66. Nevertheless, overall efficiency of the project is rated negligible given the elapsed time and modest achievements in outputs. Moreover, the benefits from the limited outputs of the project could only be realized if the Government uses/applies them in their decision-making processes moving forward.

3.4 Justification of Overall Outcome Rating Rating: Unsatisfactory

67. The project’s objective continued to have substantial relevance while its design and implementation relevance were modest. It efficacy and efficiency were negligible. There were major shortcomings in the operation’s achievement of objectives. Primarily, the PDO was not achieved. None of the PDO indicators were achieved. This is also the case with most intermediate indicators. By the time the project closed, it disbursed only 30 percent of the original amount allocated with modest achievements. Accordingly, the project’s overall outcome rating is Unsatisfactory.

3.5 Overarching Themes, Other Outcomes and Impacts

(a) Poverty Impacts, Gender Aspects, and Social Development

68. None, according to the design.

(b) Institutional Change/Strengthening

69. Institutional strengthening was at the heart of this project. Improving public sector performance was extremely important for the IPSP Project but the results obtained were very limited as few activities were implemented and the trajectory of the proposed reforms is yet to be seen. Several reports and analyses were financed by the project but it is yet to be seen whether some of these recommendations will be taken.

70. Nevertheless, several training activities have been deployed under the project to support the implementation and sustainability of the outputs produced. To support the operationalization of the budget formulation module of SIAFI-GES, UDEM provided training to over 1000 staff of 88 central government institutions and 70 percent of decentralized agencies on the use of the module. ONCAE also trained over 600 public employees from 76 central institutions on the use of framework agreements. Likewise, ONCAE provided training to over 200 government employees on the use of the new procurement documents, which were expected to be adopted after the project exited. Staff of the four sectoral cabinets (that is, Economic Development, Social

16

Development, Productive Infrastructure, and Governance and Decentralization) were also trained on the M&E systems that were developed under the project.

(c) Other Unintended Outcomes and Impacts (positive or negative)

71. Despite the project’s implementation shortcomings, it was instrumental in informing and providing for some of the prior actions of the two DPCs that were processed under the new administration. In 2014, the Honduras Fiscal Sustainability and Enhanced Social Protection DPC (P151803) was approved. With the objective of supporting the Government of Honduras to (a) strengthen fiscal management and FM; (b) strengthen the management of the power sector; and (c) improve the targeting of social protection programs, its prior actions included those implemented by the IPSP Project: enactment of and issuance of the enabling regulation for the Electronic Procurement Law that regulated the use of framework agreements, shared purchases, and inverse auctions as well as the signing of an agreement between SEFIN and the Presidential Directorate of Transparency Reform and Modernization for a shared service agreement to administer HonduCompras.

72. On the other hand, in December 2015, the Honduras First Fiscal Sustainability and Enhanced Competitiveness Development Policy Financing (P155920) was approved. With the objective of supporting the Government’s efforts in (a) strengthening institutional arrangements to support fiscal sustainability and (b) enhancing the regulatory framework to promote competitiveness, its prior actions also included the creation of the budgeting module in SIAFI-GES that specifies budget ceilings are consistent with the medium-term macroeconomic and fiscal framework which was an activity financed by the IPSP Project. Likewise, this DPC also adopted results indicators that were implemented under the IPSP Project: 88 of the central government agencies using the budget formulation module and that at least 50 percent of civil servants are covered by an institutional functional review.

3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops

73. No beneficiary surveys or stakeholder workshops were undertaken for the preparation of this Implementation Completion and Results Report (ICR).

4. Assessment of Risk to Development Outcome Rating: Negligible

74. With the project outcomes not achieved, the risk to development outcome is Negligible. Given the difficult political and economic context, the ‘stop and go’ nature of development efforts in the country,8 weak capacities, and many of the activities that were to be financed by this operation not being implemented during the life of the project. However, with respect to the outputs produced, the risk of their sustainability is high. It is hard to envision how those activities that have been implemented and action plans developed will become a reality now that the financing operation has come to an end and given the fiscal constraints. This is, particularly, a concern for those that require significant funding, including the remaining modules of SIAFI-GES

8 As characterized in the CPF FY16-20.

17

and its attendant disaster recovery center, the implementation of the human resource action plans, and the development of a new e-procurement system.

5. Assessment of Bank and Borrower Performance

5.1 Bank Performance

(a) Bank Performance in Ensuring Quality at Entry Rating: Moderately Unsatisfactory

75. The overall World Bank Performance in ensuring quality at entry is Moderately Unsatisfactory. The team leveraged past operations and analytical work to design a project that was considered more robust than the prior intervention. The areas of focus were limited and very relevant at the time of approval, albeit perhaps too ambitious given the country context for implementation.

76. Moreover, the project preparation could have been better informed by technical reviewers. Only two peer reviewers provided feedback at the concept stage, Quality Enhancement Review, and the decision meeting (the same ones on those three occasions). These two peer reviewers had solid and extensive experience and understanding of the country context. Given the broad technical areas covered by the project (PFM, procurement, HRM, and M&E), certain technical components received no feedback at all during the various decision points and could perhaps explain the lack of a technical analysis in the Appraisal Summary section of the PAD. The lack of a change management strategy for the type of reforms supported by the project is an example that could have been better vetted by technical reviewers. The inclusion of a PEFA indicator to verify a PDO indicator is another example of something that should have been fully vetted at the Quality Enhancement Review or the decision meeting, particularly regarding its financing. It is also the case that there is no written evidence of any qualms regarding the length of the project or the challenges that would entail implementing the various activities given the tight fiscal space and the burdensome procurement processes.

77. Four years did not provide enough time to implement all the reforms proposed and, as mentioned, earlier literature suggests these kinds of projects take more time to implement. Moreover, the CPF FY16–20 also did recognize that enhancing good governance requires not only sustained commitment for the reforms on the part of the counterparts but that these go beyond the four-year term of Honduran administrations, indirectly suggesting that the four-year implementation period was insufficient.

78. The World Bank failed to establish a clear leadership and accountability arrangement for the project’s implementation. When it comes to the implementation arrangements, the team incorporated lessons from the prior operation, and for example created a PCU. However, the lack of a steering committee or other entity capable of spearheading and taking full ownership of the project had a significant impact on the overall implementation.

79. The proposed M&E arrangements were adequate and the indicators set forth were measurable. On the other hand, there was significant underestimation of risks, particularly with respect to SEFIN’s approach to SIAFI and ONCAE’s almost en masse resignation following their

18

remapping to SEFIN. As a consequence, the project was not ready for implementation upon effectiveness.

80. Similarly, the risk assessment of high of implementation is proper, although this could have been complemented with political economy analysis to better inform the design of the project, such as further simplifying and limiting the choice of activities. Overall, World Bank inputs and processes had shortcomings.

(b) Quality of Supervision Rating: Unsatisfactory

81. Supervision was a challenge for reasons earlier discussed. Primarily, the World Bank adopted lessons learned from these types of operations that require intensive supervision. The World Bank has to be credited for the valiant effort expended in getting the project to take off amidst the significant shift in Government commitment immediately after effectiveness and the persistent weak capacities for implementation. Constant virtual contact and physical visits had been established early on, including having a local focal point among the country staff to handhold counterparts in the process of implementation. Having the complementary development policy operation also had its advantages in continuing the dialogue for reforms at the beginning.

82. The World Bank throughout the life of the project consistently ensured that the client complied with its fiduciary responsibilities and ensured the technical soundness of the activities that were implemented. The World Bank was also always concerned about institutional strengthening and sustainability as this was most evident in the case of ONCAE where the World Bank persisted in gaining Government buy-in for the operation of ONCAE as a regulatory body and for it to be well staffed technically for the new e-procurement platform.

83. However, regarding the adequacy of supervision inputs and processes, the implementation support plan of the PAD and early ISRs revealed that the frontloading of complex terms of references which the team expected to have ready early on during project implementation did not happen as indicated by the delayed implementation and the Moderately Unsatisfactory rating of the procurement performance in the first two years of implementation.

84. With the nature of key activities requiring the establishment of new IT systems, restructuring should have been considered to adjust activities and recalibrate results after the first year, given the constant slippage in initiation of key activities and weak implementation capacity on the ground. There should have been a constant effort as well to revisit the project timeline and assess the project’s continuing viability and development impact. The moment it had been determined that the project cannot be delivered as designed and given significant slippages in results, a restructuring should have been considered. Perhaps, utilizing the mitigation measures proposed in managing the risk of delivery, monitoring, and sustainability could have helped call the attention for adjustments promptly. In this regard, while the focus has been on getting the project off the ground, there appeared to be a lack of attention as to where it was with respect to outcome targets that should have triggered serious reflection on how far behind the project was in relation to its PDO and the options that were available to make corrections—such as restructuring and perhaps even consideration of applying World Bank remedies, which were not employed in this case.

19

85. That said, the political economy context cannot be ignored. Given, the ‘stop and go’ nature of development reforms in Honduras, perhaps the value proposition of the World Bank pursuing a restructuring was not the way to go.9 Yet, it cannot be ignored that the World Bank does have the options for improving the project’s performance and prospect of achieving its outcomes by at least recalibrating the results indicators to align them with the activities that can only be completed by closing and, if necessary, also apply World Bank remedies.

86. The midterm review occurred when only a year and a half was left before project closing. Although this was only six months after the half-life of the project, it is a significant delay considering the total length of the project. By the time the midterm review took place, the team realized that an extension could be warranted, but given the continued very slow and poor implementation and still weak capacity for implementing the project, this approach was not pursued.

87. Regarding the transition strategy that was supposed to be produced under Component 5 and as a mitigation measure for the risk posed by the change in administration, there is no evidence of a document produced. The value added of the transition strategy was viewed primarily as a way to avoid disruptions in implementation and failed to use it as a tool to reflect on the overall status of the project, identify the bottlenecks of the operation, and engage the Government to create a greater sense of ownership.

88. In relation to the candor of quality and performance reporting, the team consistently filed ISRs on time and the indicators captured are for the most part accurate. However, the World Bank was overly optimistic in upgrading the ISR in two instances,10 after the new Government took over in 2014 to reflect the renewed commitment to implement Component 3 and clarifying plans for Component 1. This upgrade was premature and not based on results achieved.

(c) Justification of Rating for Overall Bank Performance Rating: Unsatisfactory

89. Given the significant shortcomings in the quality control during project preparation as evidenced by the moderately unsatisfactory quality of design of the project and the low proactivity exhibited during supervision, the Bank’s overall performance is unsatisfactory.

5.2 Borrower Performance

(a) Government Performance Rating: Unsatisfactory

90. Government commitment and political will to implement the project was uneven for the life of the project, negatively affecting the achievement of the PDOs. For example, activities under Components 1 and 3 were essentially halted during the first two years. Immediately, after

9 This is further complicated by the specificity of the PDO that detailed the critical inputs required to achieve the desired outcomes and as a consequence would require a level 1 restructuring if the critical inputs were to be revised or recalibrated as was what happened during implementation. 10 ISRs 6 and 7.

20

effectiveness, the Government opposed the appraised design for the upgrading of SIAFI, which set back the implementation and did not allow for a full upgrade by closing. In the case of Component 3, the timing for pursuing politically sensitive HRM reforms was not opportune for a Government that is going into an election period.

91. Moreover, regarding the readiness for implementation, capacity as well as the appointment of staff was unsatisfactory. For example, the shifting of ONCAE to SEFIN was not adequately managed and this stalled Component 2 implementation from the beginning. The timely resolution of implementation issues is also unsatisfactory as evidenced by the time it took to get some components to begin implementing activities. Even with the establishment of the Project Steering Committee under the new administration, follow-up on required actions remained an issue. Most critical was the lack of leadership or a project champion within the Government for the overall project before and after the Project Steering Committee was established by the new administration that did not allow the project to progress satisfactorily.

92. Under these circumstances, the Government performance is rated Unsatisfactory.

(b) Implementing Agency or Agencies Performance Rating: Unsatisfactory

93. The project had a PCU (UAP-SEFIN) and three executing agencies: UDEM, ONCAE, and UATP. The PCU fulfilled its fiduciary role, but had some weaknesses on the M&E side; and was responsive to the needs of internal and external clients, but did not have the authority to ensure executing units were implementing their respective activities. For its part, UDEM initially challenged the technical approach for the SIAFI upgrade that led to significant delays and limited accomplishments under Component 1. ONCAE, throughout the life of the project, lacked capacity to implement the elements of Component 2 that were critical to the PDO—that is, the upgrade of the e-procurement platform. Finally, the UATP was unable to implement most of the HRM activities under Component 3 at the beginning, due to the lack of political support, and it was only in the last two years of the project that some activities finally started. With respect to Component 4, the UATP was able to advance the development and implementation of M&E systems in four sectoral cabinets (Economic Development, Social Development, Productive Infrastructure, and Governance and Decentralization) but was unable to complete the full target of six.

(c) Justification of Rating for Overall Borrower Performance Rating: Unsatisfactory

94. The overall borrower performance is Unsatisfactory. The lack of political commitment from the Government and lack of leadership for the overall project resulted in many difficulties faced during implementation. It affected the achievement of the PDO and the related intermediate indicators. Likewise, executing agencies’ lack of buy-in (SIAFI), weak capacities (ONCAE), and inadequate political backing (HRM) did not augur well for a satisfactory project implementation of the executing agencies for their components.

6. Lessons Learned

95. There are several main lessons that can be drawn from the experience of this operation. The first has to do with the adequacy and length of the instrument. A traditional investment

21

operation that involves the acquisition of systems, even with strong Government commitment, could be stymied by burdensome procurement processes, weak capacities, and limited fiscal space. As such, this project was not the best way to move forward for Honduras. Either the length of the project should have been longer from the onset or other instruments should have been considered. Related to this is that at the time of design, other World Bank instruments, such as the Program-for-Results instrument, were not fully developed. Exploring its use in this kind of project might be worthwhile although given country systems issues and persistent fiscal constraints this may not be immediately feasible for Honduras. New public sector reform projects are starting to focus on performance so the use of development linked indicators or similar tools might be worth exploring.

96. The second lesson is that the project scope could have been further narrowed given the country context and weak capacities. For example, the project could have just covered Component 1 like a stand-alone Integrated Financial Management Information System project or only one key IT system could have been pursued instead of two. Given a client that is capacity constrained, with fluid reform commitments, with multiple agencies involved, the project is a complex undertaking for them. Even having the first DPC as, supposedly, a mitigating measure to ensure HRM reform commitment did not work; in the end, the programmatic DPC (P151283) became a stand-alone operation due to issues in meeting triggers for the second operation. Further, while the IPSP Project was complemented by three DPCs, it would appear not to have benefited the project in advocating for continuing the reform implementation after the DPCs disbursed. The use of programmatic DPCs to help implement activities and push public sector reform forward should go beyond prior actions, but more specifically triggers should be considered. Moreover, if tied closely with DPCs, it may be better to engage the Government in technical assistance activities and reward them through a DPC when a critical prior action has been taken.

97. The third lesson has to do with the implementation arrangements. The team readily adopted the lessons learned from the previous project and created a PCU and, at least on paper, tried to house most of the components under SEFIN and bind those outside agencies to implementing the activities through Memorandums of Understanding. The PCU, according to design, lacked the authority and capacity to enforce the implementation arrangements or do technical work. SEFIN had difficulties taking the helm of the project at a policy level and making the other executing agencies follow the lead. The creation of a steering committee or another decision-making entity capable of leading the implementation of the project would have been helpful and created greater ownership among executing agencies, as is the case in other investment projects.

98. Improvements in the quality control and peer review system is the fourth important lesson. Having at least three peer reviewers and ensuring that all the components are vetted technically is of utmost importance. Ensuring change management strategies and communication campaigns are in place to accompany these reforms is critical for this type of project.

99. Fifth, the project’s implementation experience also reinforces the findings in the Latin America and the Caribbean Region with respect to integrated FM information system projects that

22

highlighted political environment and committed leadership as among the keys to success along with close supervision.11

100. Sixth, with respect to the PDO formulation, this could have been simplified to just focus on the desired outcomes without specifying the activities/outputs that would need to be pursued12 which could have facilitated processing of a project restructuring that is processed as a level 2 restructuring. As it is, given the specificity, any change in the critical inputs would have required a level 1 restructuring.

7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners

(a) Borrower/implementing agencies

101. No comments were received from the Borrower.

(b) Cofinanciers

102. Not applicable.

(c) Other partners and stakeholders

(e.g. NGOs/private sector/civil society)

103. No comments.

11 “Financial Management Information System: 25 years of World Bank Experience on What works and What Doesn’t.” Dener, Watkins, Dorotinsky. 2011. 12 That is deleting the secondary clause of the PDO “through: (i) upgrading the public financial management system; (ii) upgrading the e-procurement platform; (iii) enhancing the internal control systems over personnel expenditures; and (iv) building the capacity of the central administration.”

23

Annex 1. Project Costs and Financing

(a) Project Cost by Component (in US$ Million equivalent)

Components Appraisal Estimate

(US$, millions) Actual/Latest Estimate

(US$, millions) Percentage of

Appraisal Strengthening and Consolidating Financial Management Systems

7.9 1.62 20.51

Strengthening the Public Procurement System

4.7 0.48 10.21

Improving Public Sector Human Resource Management

3.7 1.85 50.00

Strengthening Government Monitoring and Evaluation Capacity

0.8 0.23 28.75

Project Coordination 1.1 0.63 57.27 Total Baseline Cost 18.20 4.81 26.42

Physical Contingencies 0.00 0.00 0.00 Price Contingencies 0.00 0.00 0.00

Total Project Costs 0.00 0.00 Front-end fee Project Preparation Facility

0.00 0.00 0.00

Front-end fee IBRD 0.00 0.00 0.00 Total Financing Required 18.20 4.81 26.42

(b) Financing

Source of Funds Type of

Cofinancing

Appraisal Estimate

(US$, millions)

Actual/Latest Estimate

(US$, millions)

Percentage of Appraisal

Borrower 0.00 0.00 0.00 International Development Association (IDA)

18.20 4.81 26.42

24

Annex 2. Outputs by Component

Component 1: Strengthening and Consolidating Financial Management Systems

Intermediate Indicator Target Value at Exit Status at Closing Institutional coverage of the updated SIAFI: 100% of central government and 70% of decentralized institutions operate with the updated SIAFI.

Partially Achieved. Only budget formulation module has been completed and implemented in all central government and 70% of decentralized institutions.

Open Budget Index: Improvement on Open Budget Indicator Index (at least 50 out of 100)

Achieved but not sustained. Although the country scored 53 out of 100 in 2012, it dropped to 43 in 2015.

All additional functionalities to the SIAFI are implemented

Not achieved. Only the budget formulation module has been completed and implemented.

Accreditation of all SIAFI users in central government institutions

Not achieved. Nevertheless, with the budget formulation module operational, training at its conceptual level has been deployed covering over 1,000 employees from all central government institutions and 70% of decentralized agencies.

Subcomponent 1.1: Technological Upgrade of SIAFI

(a) Planned Output: Development and implementation of a new technological framework to enhance usability and access to SIAFI, including the design and development of applications and functionalities of SIAFI. Delivered Output: Only the budget formulation module was completed and operationalized.

(b) Planned Output: Design and implementation of information and communication technology security measures for the SIAFI system and carrying out of audits and certifications to ensure compliance with information and communication technology security measures. Delivered Output: This output was not delivered.

(c) Planned Output: Creation of a disaster recovery center. Delivered Output: Instead of a physical establishment, a colocation solution of renting space in a commercial data center was pursued for US$200,000 (10 percent of the original estimated cost of the center).

Subcomponent 1.2: Strengthening of the Conceptual and Functional Framework of SIAFI

(a) Planned Output: Design and implementation of a procurement module for SIAFI to be connected to HonduCompras. Delivered Output: As only budget formulation module was developed, the procurement module has not been designed yet.

(b) Planned Output: Design of a connection system between SIAFI’s budget module and the public investment system. Delivered Output: This activity was not implemented.

25

Subcomponent 1.3: Capacity-building Strategy

(a) Planned Output: Development of an accreditation program for SIAFI operators, including development of training materials and training curricula. Delivered Output: SIAFI upgrade was not fully developed at completion; as such no accreditation program has been conducted. Nevertheless, training was conducted for about 1,000 employees on the budget formulation module.

(b) Planned Output: Carrying out of pilot training and accreditation programs. Delivered Output: SIAFI upgrade was not fully developed at completion; as such no accreditation program has been conducted.

Component 2: Strengthening the Public Procurement System

Intermediate Indicator Target Value at Exit Status at Closing New e-procurement platform implemented and at least 25% of procurement transactions carried out online

Not Achieved. Only bidding documents for the new platform have been completed.

6 framework agreements implemented for common use goods

Achieved.

Institutional strengthening of ONCAE as measured at close by procurement units in all central government institutions that have received training in operation of new e-platform.

Not achieved. ONCAE does not have capacity to support the development of the new e-platform.

Subcomponent 2.1: Strengthening Compliance and Transparency

(a) Planned Output: Development of a module connecting HonduCompras with SIAFI. Delivered Output: As implementation of the new HonduCompras has not started, this output was not delivered.

(b) Planned Output: Development of the standard template for procurement plans. Delivered Output: Standard templates produced in May 2012.

Subcomponent 2.2: Institutional Strengthening of ONCAE

(a) Planned Output: Strategic and organizational plan to ensure the sustainable functioning of ONCAE. Delivered Output: Consultants were hired to document business processes. The civil directorate was approached twice regarding interest to create an organizational plan and no response was received. Understaffing was a constant issue for ONCAE during the lifetime of the project. The ONCAE organizational plan was approved in end-2015 but its implementation could no longer be supported by the project.

(b) Planned Output: Developing methodologies and training to support the creation of a statistical unit tasked with monitoring and evaluating the performance of the public procurement system. Delivered Output: Not implemented.

26

(c) Planned Output: Activity would finance the development of a unit specialized in the development and implementation of framework agreements. Delivered Output: Six framework agreements were signed and a framework agreement unit was established.

(d) Planned Output: Creation of standardized procurement materials such as bidding documents, procurement plans, and manuals. Delivered Output: Standardized documents were created with project support and completed in May 2013 but not deployed for application during project implementation.

Subcomponent 2.3: Design and Implementation of a New e-Procurement Platform

(a) Planned Output: Design of a new e-procurement platform. Delivered Output: Bidding documents of platform ready but due to low capacity of ONCAE process has halted.

(b) Planned Output: Development and implementation of new platform. Delivered Output: No outputs delivered.

(c) Planned Outputs: Hardware for the new platform. Delivered Output: No outputs delivered.

Component 3: Improving Public Sector Human Resources Management

Intermediate Indicator Target Value at Exit Status at Closing Enhanced control over personnel expenditure through implementation of corrective measures following findings of transactions-based payroll audit for contracted personnel at Ministry of Health and Ministry of Public Works leading to at least 3% savings in contracted expenditures on an annual basis.

Partially Achieved. Payroll audits in 5 (Education, Health, Infrastructure, and Security Secretariats, and Tax Administration Agency) institutions completed and functional reviews also took place. Recommended corrective actions remain to be implemented.

Improved professionalization of public service in SEFIN and one more institution completed.

Not Achieved. Only completed payroll audits in 5 institutions mentioned above and institutional functional review of 6 institutions (Education, Health, Infrastructure, and Security Secretariats, SEFIN, and Civil Service Directorate). The reports include an action plan for strengthening HRM but they are yet to be adopted/implemented.

Subcomponent 3.1: Enhancing Controls over Personnel Expenditures

(a) Planned Output: Implementation of process-based payroll audits for selected central administration institutions. Delivered Output: Payroll audits took place in five key institutions, as mentioned in the Component 3 table, covering over 80 percent of the central government payroll.

(b) Planned Output: Transaction-based payroll audits in those areas with considerable increases in expenditures and weak controls.

27

Delivered Output: In addition to the five payroll audits aforementioned, functional reviews took place in these institutions, SEFIN, and the Civil Service Directorate.

(c) Planned Output: A political economy and communications strategy. Delivered Output: None.

Subcomponent 3.2: Improving Attraction and Retention of Key Personnel and Supporting the Professionalization of Key Positions in Selected Institutions

(a) Planned Output: Diagnostic of SEFIN’s career path, including a pay structure. Delivered Output: Functional review of SEFIN as well as Civil Service, Health, Education, Infrastructure, and Security Secretariats—covering over 50 percent of central government payroll.

(b) Planned Output: Certification of positions, skills gap assessment, and external validation of recruitment or promotion processes. Delivered Output: Functional review of six secretariats conducted at end of project hence no implementation of recommendations that would allow for these planned outputs could be supported by the project.

Component 4: Strengthening Government Monitoring and Evaluation Capacity

Intermediate Indicator Target Value at Exit Status at Closing 6 sector cabinets using the indicators and goals established in the M&E system. An increase in user satisfaction.

Partially Achieved. Implementation of the developed sectoral strategies and indicators was completed in 4 out of the 7 sector cabinets. The indicators and tools developed were successfully integrated in new budgeting module of SIAFI and used in the 2016 budget formulation.

Subcomponent 4.1: Conceptual and Functional Design of the Results-based M&E System

(a) Planned Output: Redesign the M&E platform. This would include the necessary interfaces and linkages to other systems such as SIAFI. Delivered Output: Sectoral strategies and indicators developed for four sector cabinets and indicators and tools integrated into new budgeting module of SIAFI.

(b) Planned Output: Definition of indicators and targets for the goals included in selected sector cabinets. Delivered Output: Sectoral strategies and indicators developed for four sector cabinets (that is, Economic Development, Social Development, Productive Infrastructure, and Governance and Decentralization).

(c) Planned Output: Design and implementation of incentives to promote production and use of performance information. Delivered Output: Consultancy for program-based budget in 2015 as well as in the Government’s strategic plan of 2014–2018.

Subcomponent 4.2: Institutionalizing a Results-based M&E System

28

(a) Planned Output: Technical assistance to assess and redefine institutional arrangements between key actors of the system. Delivered Output: Consultancy for program-based budgeting.

(b) Planned Output: Strengthening the technical, managerial, and organizational capacities of the UATP to implement the M&E system, manage results information, and provide strategic advice for decision making in the sector cabinets. Delivered Output: No output delivered.

Subcomponent 4.3: Elaboration and Dissemination of Methodologies for the Institutionalization of M&E Functions and Systems

(a) Planned Output: Manuals to standardize and disseminate M&E methodologies. Delivered Output: None

Component 5: Project Coordination

Intermediate Indicator Target Value at Exit Status at Closing Successful transition of project management elaborated and implemented

Achieved. Following the change in administration in early 2014, a Project Steering Committee was established to help improve implementation. The Director of UAP-SEFIN assumed the role of the project coordinator and all fiduciary staff were maintained.

Subcomponent 5.1: Project Coordination

(a) Planned Output: Hiring of a project coordinator to oversee implementation of the project, and other support staff, that is, procurement and FM specialists; annual external audits and final audit of the project; regular compliance audit for the project’s results indicators; and other consultancies contributing to the achievement of the development objective. Delivered Output: PCU with project coordinator and all other necessary fiduciary staff. Financial audits completed as agreed as well as reporting on overall status of projects and indicators. Annual operational audit not conducted through project life; only one was prepared at the end of the project.

Subcomponent 5.2: Transition Strategy

(a) Planned Output: A transition strategy to prevent any negative impact (such as delays, loss of institutional memory, and so on) in the project implementation due to unforeseen change of priorities, and strategies of the incoming administration or a high turnover of counterparts. Activities would include the systematization of information as well as its socialization to key stakeholders (through seminars, workshops, and other dissemination activities). Activities would also include capacity-building activities for incoming personnel assigned to the project. Delivered Output: Consultant hired to facilitate transition strategy in February 2014.

29

Annex 3. Economic and Financial Analysis

1. As this was a technical assistance project devoted to institutional strengthening, computations of rate of return, cost effectiveness, and others are difficult to quantify. Nevertheless, it was expected to have a positive fiscal impact and generate economic gains that were to come from more efficient use of public finances through improved public procurement processes, proper budget execution, and enhanced controls of the public sector wage bill. In this regard, a few indicators of efficiencies can be identified.

2. The government reported that, as a result of the six framework agreements implemented, it has saved HNL 27 million (US$1.215 million or almost double the investment under this Component 2) in 2015 alone covering 5,000 products for 70 different government institutions. These are nominal savings and the methodology used to calculate the savings has not been verified by the World Bank.

3. Likewise, by renting a space in a commercial data center instead of establishing its own physical Disaster Recovery Center for SIAFI, the Government was able to gain a data center that would mitigate business continuity disruptions at a lower cost. Original allocation was for US$2 million to establish the physical center; instead, they spent US$200,000 in consultancy services (about 10 percent of estimated cost).

4. Nevertheless, overall efficiency of the project is rated negligible given the elapsed time and modest achievements. Moreover, the benefits from the limited outputs of the project could only be realized if the Government uses/applies them in their decision-making processes moving forward.

30

Annex 4. Bank Lending and Implementation Support/Supervision Processes

(a) Task Team Members

Names Title Unit Responsibility/Special

ty Lending

Ana Bellver Senior Public Sector Specialist GG018 TTL at first Concept Note

Gisela Durand Consultant GGO16 Operations consultant Henry Forero Senior Public Sector Specialist GGO16 IT specialist Valerie Hermann Perez Program Assistant EXC Administrative support

Mariano Lafuente Public Sector Mgmt. Specialist LCSPS -

HIS HRM specialist

Andres MacGaul Senior Procurement Specialist GGO03 Procurement Carolina Rendon Senior Public Sector Specialist GGO16 TTL Jose Rezk Senior Financial Management Specialist GGO22 Financial management David Santos Consultant GGO16 Joao Nuno Vian Lanceiro da Veiga Ma

Practice Manager GGO04 Procurement specialist

Fanny Weiner Public Sector Management Specialist GGO16 Operational/ technical support

Supervision/ICR Daniel Alvarez Senior Public Sector Specialist GGO16 Caretaker TTL Marcelo Buitron Extended Term Consultant GGO16 ICR main author Diego Dorado Senior Public Specialist GGO16 TTL at closing Gisela Durand Consultant GGO16 Operations consultant Leonel Estrada Procurement Specialist GGO04 Procurement Dmitri Gourfinkel Financial Management Specialist GGO22 Financial management Pablo Andres Guzman Research Analyst GGO16 Operational support Andres MacGaul Senior Procurement Specialist GGO03 May Olalia Senior Public Sector Specialist GGO16 Final TTL of ICR Karina Ramirez Research Analyst GGO16 Operational support Carolina Rendon Senior Public Sector Specialist TTL at effectiveness Jose Rezk Senior Financial Management Specialist GGO22 Financial management Svetlana Proskurovska Senior Public Sector Specialist GGO15 Initial TTL of ICR David Santos Public Sector Specialist GGO16 IT specialist

(b) Staff Time and Cost

Stage of Project Cycle Staff Time and Cost (Bank Budget Only)

No. of Staff Weeks US$, Thousands (Including

Travel and Consultant Costs) Lending FY08 4.70 58.34 FY09 19.83 159.77 FY10 4.45 25.58 FY11 18.81 98.43 FY12 21.10 95.53

Total: 68.89 437.66* Supervision/ICR FY12 7.93 51.21

31

FY13 22.26 96.04 FY14 29.61 137.51 FY15 38.23 161.05 FY16 22.18 136.79

Total: 120.21 582.60*

*Difference is due to rounding.

32

Annex 5. Beneficiary Survey Results

No beneficiary survey was done.

33

Annex 6. Stakeholder Workshop Report and Results

No workshop took place.

34

Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR

No comments were received from the Borrower. Since December 2015, the Bank has requested for the Borrower’s version of events and have repeatedly asked for this. The Borrower advised that they will provide it to the Bank. To date, no inputs nor comments on the ICR were received from the Borrower.

35

Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders

No comments provided.

36

Annex 9. List of Supporting Documents

None.