world bank document...2017/03/15 · sidunea automated system for customs data platform (sistema...

Document of

The World Bank

Report No: ICR00003947

IMPLEMENTATION COMPLETION AND RESULTS REPORT

(IBRD-78120, TF-95841)

ON A

LOAN

IN THE AMOUNT OF US$20 MILLION

TO THE

REPUBLIC OF EL SALVADOR

FOR A

FISCAL MANAGEMENT AND PUBLIC SECTOR PERFORMANCE TECHNICAL

ASSISTANCE PROJECT

March 10, 2017

Governance Global Practice

Latin America and the Caribbean Region

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

CURRENCY EQUIVALENTS

The official currency unit in El Salvador is the U.S. Dollar

FISCAL YEAR

January 1 – December 31

ABBREVIATIONS AND ACRONYMS

CPS Country Partnership Strategy

DGA Customs Agency (Dirección General de Aduanas)

DGII Internal Revenue Agency (Dirección General de Impuestos Internos)

DGP General Budget Office (Dirección General de Presupuesto)

DGT Treasury Office (Dirección General de Tesorería)

DINAFI National Financial Administration and Innovation Office (Dirección Nacional

de Administración Financiera e Innovación)

GDP Gross Domestic Product

GIZ German Agency for International Cooperation (Deutsche Gesellschaft für

Internationale Zusammenarbeit)

IADB Inter-American Development Bank

ICR Implementation Completion and Results Report

ICT Information and Communication Technology

IMF International Monetary Fund

ISR Implementation Status and Results Report

IT Information Technology

LDSW Locally Developed Software Solution

M&E Monitoring and Evaluation

MoF Ministry of Finance

MTEF Medium-term Expenditure Framework

OIR Office of Information and Response (Oficinas de Información y Respuesta)

PAD Project Appraisal Document

PDO Project Development Objective

PEFA Public Expenditure and Financial Assessment

PFM Public Financial Management

PIU Project Implementation Unit

SAFI Integrated Financial Management System (Sistema de Administración

Financiera Integrada)

SCPT Secretariat of Citizen Participation and Transparency

SIDUNEA Automated System for Customs Data Platform (Sistema Aduanero

Automatizado)

SIRH Human Resource Management System

TTL Task Team Leader

UNAC National Procurement Office (Unidad Normativa de Adquisiciones y

Contrataciones de la Administración Publica)

UNCTAD United Nations Conference on Trade and Development

USAID U.S. Agency for International Development

Senior Global Practice Director: Deborah Wetzel

Practice Manager: Arturo Herrera

Project Team Leader: Maria Guadalupe Toscano Nicolas

ICR Team Leader: Joanna Watkins

EL SALVADOR

Fiscal Management and Public Sector Performance Technical Assistance Project

CONTENTS

Data Sheet

A. Basic Information ........................................................................................................ i

B. Key Dates .................................................................................................................... i C. Ratings Summary ........................................................................................................ i D. Sector and Theme Codes ........................................................................................... ii E. Bank Staff ................................................................................................................... ii F. Results Framework Analysis ..................................................................................... iii

G. Ratings of Project Performance in ISRs .................................................................... v H. Restructuring (if any) ................................................................................................ vi

1. Project Context, Development Objectives and Design ............................................................... 1

2. Key Factors Affecting Implementation and Outcomes .............................................................. 4

3. Assessment of Outcomes .......................................................................................................... 12

4. Assessment of Risk to Development Outcome ......................................................................... 18

5. Assessment of Bank and Borrower Performance ..................................................................... 19

6. Lessons Learned........................................................................................................................ 21

7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners........................... 24

Annex 1. Project Costs and Financing .......................................................................................... 25

Annex 2. Outputs by Component.................................................................................................. 26

Annex 3. Economic and Financial Analysis ................................................................................. 30

Annex 4. Bank Lending and Implementation Support/Supervision Processes ............................. 32

Annex 5. Beneficiary Survey Results ........................................................................................... 34

Annex 6. Stakeholder Workshop Report and Results ................................................................... 35

Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR ..................................... 36

Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders ....................................... 37

Annex 9. List of Supporting Documents ...................................................................................... 38

Annex 10. Achievement of Objectives Analysis .......................................................................... 39

Annex 11. El Salvador PEFA Indicators 2009 versus 2013 ......................................................... 44

MAP .............................................................................................................................................. 45

i

A. Basic Information

Country: El Salvador Project Name:

Fiscal Management and

Public Sector

Performance Technical

Assistance Loan

Project ID: P095314 L/C/TF Number(s): IBRD-78120, TF-95841

ICR Date: 03/10/2017 ICR Type: Core ICR

Lending Instrument: Technical Assistance

Loan Borrower:

REPUBLIC OF EL

SALVADOR

Original Total

Commitment: US$20.00 million Disbursed Amount: US$15.36 million

Revised Amount: US$20.00 million

Environmental Category: C

Implementing Agencies:

Ministerio de Hacienda

Cofinanciers and Other External Partners:

B. Key Dates

Process Date Process Original Date Revised / Actual

Date(s)

Concept Review: 09/10/2009 Effectiveness: 05/24/2011 05/24/2011

Appraisal: 10/19/2009 Restructuring(s): — 10/08/2014

Approval: 11/24/2009 Midterm Review: 02/11/2013 02/11/2013

Closing: 12/31/2014 09/30/2016

C. Ratings Summary

C.1 Performance Rating by ICR

Outcomes: Unsatisfactory

Risk to Development Outcome: Moderate

Bank Performance: Moderately Unsatisfactory

Borrower Performance: Unsatisfactory

C.2 Detailed Ratings of Bank and Borrower Performance (by ICR)

Bank Ratings Borrower Ratings

Quality at Entry: Moderately

Unsatisfactory Government:

Moderately

Unsatisfactory

Quality of Supervision: Moderately

Unsatisfactory

Implementing

Agency/Agencies: Unsatisfactory

Overall Bank

Performance:

Moderately

Unsatisfactory Overall Borrower

Performance: Unsatisfactory

ii

C.3 Quality at Entry and Implementation Performance Indicators

Implementation

Performance Indicators

QAG Assessments (if

any) Rating

Potential Problem Project at

any time (Yes/No): No

Quality at Entry

(QEA): None

Problem Project at any time

(Yes/No): Yes

Quality of Supervision

(QSA): None

DO rating before

Closing/Inactive status: Unsatisfactory

D. Sector and Theme Codes

Original Actual

Major Sector/Sector

Public Administration

Public Administration - Industry and Trade 15 15

Public Administration - Financial Sector 60 60

Central Government (Central Agencies) 25 25

Major Theme/Theme/Sub Theme

Economic Policy

Fiscal Policy 15 15

Tax policy 15 15

Public Sector Management

Public Administration 45 45

Administrative and Civil Service Reform 10 10

Transparency, Accountability, and Good Governance 35 35

Public Finance Management 40 40

Domestic Revenue Administration 15 15

Public Expenditure Management 25 25

E. Bank Staff

Positions At ICR At Approval

Vice President: Jorge Familiar Calderon Pamela Cox

Country Director: J. Humberto Lopez Laura Frigenti

Practice Manager/Manager: Arturo Herrera Gutierrez Nicholas Paul Manning

Project Team Leader: Maria Guadalupe Toscano Nicolas Alberto Leyton

ICR Team Leader: Joanna Alexandra Watkins

ICR Primary Author: Joanna Alexandra Watkins

iii

F. Results Framework Analysis

Project Development Objectives (from Project Appraisal Document) To strengthen the institutional capacity of specific government processes and agencies to increase the

effectiveness and efficiency of revenue and expenditure management and enhance accountability and

transparency in the public sector.

Revised Project Development Objectives (as approved by original approving authority) Not applicable.

(a) PDO Indicator(s)

Indicator Baseline Value

Original Target

Values (from

approval

documents)

Formally Revised

Target Values

Actual Value

Achieved at

Completion or

Target Years

Indicator 1: Increased efficiency of customs through increased number of customs points

with nonintrusive control methods.

Value

(quantitative or

qualitative)

None of customs points

use nonintrusive control

methods, of a total of 10

(6 land, 2 air, 2 sea)

— 7 6

Date achieved 12/31/2009 — 09/30/2016 09/28/2016

Comments

(including %

achievement)

Partially achieved. This indicator was added during the 2014 restructuring.

Indicator 2: Stronger planning, budgeting, treasury, and accounting systems in place, in line with

international standards

Value

(quantitative or

qualitative)

SAFI does not allow for

program-based

budgeting and payments

are processed through a

quasi-Treasury Single

Account in a separate

system

—

The new SAFI II

system is fully

developed and

ready to be

launched. It

includes program-

based budgeting,

payment

processing through

the Treasury Single

Account in a

separate system

The MoF concluded

the development of

the budget

formulation and

budget execution

modules of SAFI

II. Program-based

budgeting is fully

embedded in SAFI II

as well as the MTEF.

Programs are also

results-based and

adopt the

logic framework

methodology.

Date achieved 12/31/2009 — 09/30/2016 09/28/2016

Comments

(including %

achievement)

Partially achieved (target value). This indicator was added during the 2014

restructuring.

iv

(b) Intermediate Outcome Indicator(s)

Indicator Baseline Value

Original Target

Values (from

approval

documents)

Formally

Revised Target

Values

Actual Value

Achieved at

Completion or

Target Years

Indicator 1: Hours to clear merchandise from the moment it arrives to customs terminal until it is

released

Value

(quantitative or

qualitative)

60 48 17 21

Date achieved 12/31/2009 12/31/2014 09/30/2016 09/28/2016

Comments

(including %

achievement)

Partially achieved. The targets for this indicator were revised during the 2014

restructuring.

Indicator 2: % increase of web-processed transactions in customs

Value

(quantitative or

qualitative)

89 5 98 98

Date achieved 12/31/2009 12/31/2014 09/30/2016 09/28/2016

Comments

(including %

achievement)

Achieved. The targets for this indicator were revised during the 2014 restructuring.

Indicator 3: Development of the new SAFI II module for budget execution will include program-

based budgeting

Value

(quantitative or

qualitative)

The current SAFI system

does not include proper

budget execution controls

and only allows for line

item budgeting.

SAFI is able to

automatically create

accounting reports

and financial

statistics according

to the International

Public Sector

Accounting

Standards and 2001

IMF Government

Financial Statistics

standards.

A new module

for budget

execution in

SAFI II that

includes

program-based

budgeting has

been developed.

The MoF concluded

the development of

the budget

formulation and

budget execution

modules of SAFI

II. Program-based

budgeting is fully

embedded in SAFI II.

Programs are also

results-based and

adopt the logic

framework

methodology.

Date achieved 12/31/2009 12/31/2014 09/30/2016 09/28/2016

Comments

(including %

achievement)

Achieved.

Indicator 4: Implementation of training and dissemination initiatives to strengthen Government

transparency, citizen participation, and access to public information

Value

(quantitative or

qualitative)

0 initiatives have been

implemented —

22 initiatives

have been

implemented.

50 initiatives

implemented

v

Institutional

capacity of the

Government is

strengthened

and knowledge

about right of

access to public

information and

Government

transparency

tools has

increased

among public

servants and

the general

public.

Date achieved 12/31/2010 — 09/30/2016 09/28/2016

Comments

(including %

achievement)

Achieved. This indicator was added during the 2014 restructuring.

Indicator 5: Increased capacity in budget planning and formulation through the new integrated

financial system

Value

(quantitative or

qualitative)

0 12 training sessions — 17 training sessions

Date achieved 12/31/2010 09/30/2016 — 09/28/2016

Comments

(including %

achievement)

Achieved. This indicator was added during the 2014 restructuring.

G. Ratings of Project Performance in ISRs

No. Date ISR

Archived DO IP

Actual Disbursements

(US$, millions)

1 06/04/2010 Moderately Satisfactory Moderately Satisfactory 0.00

2 12/01/2010 Moderately Unsatisfactory Moderately Unsatisfactory 0.00





7 09/03/2013 Satisfactory Moderately Satisfactory 3.11




11 02/17/2016 Unsatisfactory Unsatisfactory 8.86

12 07/28/2016 Unsatisfactory Unsatisfactory 12.10

vi

H. Restructuring (if any)

Restructuring

Date(s)

Board

Approved PDO

Change

ISR Ratings at

Restructuring

Amount

Disbursed at

Restructuring

in US$, millions

Reason for Restructuring & Key

Changes Made DO IP

2/16/2011 MU MU 0.00

Delays in effectiveness and

substantial changes to design of

components.

10/08/2014 MU MU 6.53

Extension of closing date.

I. Disbursement Profile

1

1. Project Context, Development Objectives and Design

1.1 Context at Appraisal

1. Country context. At appraisal in 2009, El Salvador was facing an economic slowdown precipitated by the 2008 global financial crisis. Declines in remittances, exports, foreign

investment, and tightened access to credit linked to the slowdown in the United States led to a

sudden drop in revenues, increasing the country’s fiscal deficit. In 2008, economic growth

slowed to 2.5 percent, while the fiscal deficit rose to 3.1 percent of the gross domestic product

(GDP) from 1.9 percent in 2007. The deterioration in macroeconomic and fiscal indicators posed

a tremendous challenge to the new Government. According to the 2008 Multiple Purpose

Household Survey, poverty had risen to 42.3 percent in 2008. Between 2008 and 2009,

unemployment increased from 5.9 percent to 7.3 percent. More than 30,000 workers had lost

their jobs, and combined with the sudden decrease in family remittances, there was a risk of

further exacerbating poverty levels and social problems.

2. In response to these challenges, the Government announced a comprehensive Anti-Crisis Plan to protect vulnerable populations affected by job losses or lower remittances through

actions targeting revenue and expenditures to create the needed space for priority spending.

These included austerity measures to reduce non-priority spending and interventions to tackle tax

evasion to contain further declines in public revenues. On the expenditure side, the Government

decided to embark on the upgrading of budgeting, financial management, procurement systems,

and improvements in public access to Government financial and nonfinancial information. The

proposed World Bank operation was designed to specifically respond to a core pillar of the

Government’s Anti-Crisis Plan addressing short- and medium-term challenges in public sector

development and fiscal management. This operation was part of a package of lending operations

designed to support El Salvador’s response to the crisis. A Development Policy Lending

operation and various sector lending operations accompanied this loan. At appraisal, this

operation was considered critical by the Ministry of Finance (MoF) to address weaknesses in its

management of revenue and expenditure.

3. The objectives of the operation were aligned to the Government’s strategic priorities and the World Bank’s Country Partnership Strategy (CPS).

1 The main objective of the FY2010–12

CPS for El Salvador was to support the Government’s focus on addressing poverty and

inequality through strengthening fundamentals for economic recovery by addressing macro and

institutional vulnerabilities; strengthening social service delivery; and increasing economic

opportunities, particularly for the poor. Under the first sub-objective (strengthening the

fundamentals for economic recovery), the CPS identified support to (a) the expansion of fiscal

space, (b) improved targeting of subsidies, (c) implementation of results-based budgeting, (d)

enhanced access to information, and (e) improved fiscal transparency. The proposed operation

sought to address critical short-term fiscal pressures through tax measures while developing

institutional capacities and systems in the areas of transparency, results-based budgeting, and

public financial management (PFM).

1 World Bank. 2009. El Salvador - Country Partnership Strategy 2010–2012. Report N. 50642-SV.

2

1.2 Original Project Development Objectives (PDO) and Key Indicators (as approved)

4. The PDO was to strengthen the institutional capacity of specific government processes and agencies to increase the effectiveness and efficiency of revenue and expenditure

management and enhance accountability and transparency in the public sector.

5. Progress toward achievement of the PDO was to be measured using the following PDO-level indicators:

(a) Tax collection as a percent of GDP increases at least 1 percent as a result of greater effectiveness and efficiency

(b) Stronger planning, budgeting, treasury, and accounting systems in place, in line with international standards

(c) The introduction of multi-annual projections and programmatic structure in the budget strengthens strategic planning and the implementation of performance-based

budgeting

(d) Savings generated in central government purchasing are at least 10 percent by comparing the same type of items (by adopting optimizing procurement policies)

1.3 Revised PDO (as approved by original approving authority) and Key Indicators, and

reasons/justification

6. The PDO remained unchanged throughout implementation; however, key PDO indicators were revised. During the first restructuring, three of the original four PDO indicators were

dropped and one new indicator was added (see table 1). This was done to retain the relevance of

project activities following delays in loan approval and advances in tax reforms and multi-annual

budgeting using alternative financing sources.

PDO indicators

Original List Revised List

Tax collection as a percent of GDP increases at

least 1% as a result of greater effectiveness and

efficiency

Stronger planning, budgeting, treasury, and

accounting systems in place, in line with


Stronger planning, budgeting, treasury, and

accounting systems in place, in line with


Increased efficiency of customs through increased

number of customs points with nonintrusive

control methods financed by the loan

The introduction of multi-annual projections and

programmatic structure in the budget strengthens

strategic planning and the implementation of

performance-based budgeting

Savings generated in central government

purchasing are at least 10% by comparing the

same type of items (by adopting optimizing

procurement policies)

3

1.4 Main Beneficiaries

7. The project’s main beneficiaries were the MoF and its respective offices and agencies: the National Financial Administration and Innovation Office (Dirección Nacional de

Administración Financiera e Innovación, DINAFI); the Customs Agency (Dirección General de

Aduanas, DGA); the Internal Revenue Agency (Dirección General de Impuestos Internos,

DGII); the General Budget Office (Dirección General de Presupuesto); the Treasury Office

(Dirección General de Tesorería, DGT); and the National Procurement Office (Unidad

Normativa de Adquisiciones y Contrataciones de la Administración Publica, UNAC). For the

transparency component, the target beneficiary was the Secretariat of Citizen Participation and

Transparency (SCPT) within the President’s Office. After the first restructuring, the DGII and

the UNAC were no longer beneficiaries of project activities. Secondary beneficiaries of the

project include the 114 institutions of the central public administration and citizens, who would

benefit from the reforms introduced.

1.5 Original Components

8. The project was structured around four main components:

9. Component 1: Strengthening tax collection agencies (US$7.8 million). The objective of this component was to analyze and implement options to integrate activities to strengthen

revenue agencies’ institutional capacity to control tax evasion and smuggling. Activities included

(a) strengthening coordination among the DGII, DGA, and DGT to improve information sharing

practices and increase common control procedures for better tax compliance; (b) strengthening

the DGII’s institutional capacity; and (c) supporting the DGA by strengthening its auditing and

control functions.

10. Component 2: Modernizing of public expenditures and financial management (US$7.5 million). This component sought to help the MoF modernize its budget and financial

management processes and systems through four main activities: (a) introduction of a multi-

annual and a performance-based budgetary framework; (b) implementation of a Treasury Single

Account and use of electronic payments; (c) strengthening and upgrading of the integrated

financial management information system (Sistema de Administración Financiera Integrada,

SAFI); and (d) modernization of the public procurement system.

11. Component 3: Enhancing and piloting information management and public sector transparency initiatives (US$3.2 million). The main purpose was to support complementary

initiatives to support planning and decision-making processes and transparency and access to

public information to strengthen existing accountability systems. This component was explicitly

designed to be flexible and adaptable to changing conditions.

12. Component 4: Project coordination and strengthening of the MoF (US$1.1 million). This component was designed to provide support to the MoF to adequately coordinate and

monitor project implementation through strengthening the capacities of DINAFI and the MoF’s

administrative units of financial management and procurement (UNAC).

4

1.6 Revised Components

13. In 2011, before project effectiveness, the team conducted a Level 2 restructuring to respond to changes in scope required by the National Assembly for their approval of the project.

All components were revised to modify specific activities that would be financed with funds

from other donors and the restructuring reoriented activities to focus on customs management,

PFM, and transparency activities. In particular, Component 1 (strengthening tax collection

agencies) was substantially reduced and reoriented to focus on customs. The restructuring

dropped activities related to the provision of software, equipment, and training for the DGII.

Within Component 2, which focused on modernizing public expenditure and financial

management, various subcomponents were revised, with the development of a multi-annual

budgeting approach dropped from this component, while activities related to the upgrading and

integration of associated financial information systems (SAFI II) were significantly expanded.

The update of procurement policies to increase efficiency was removed and a new subcomponent

was added to support the modernization of public investment and debt management systems.

Overall, this component saw the largest increase in reallocated funds (from US$7.5 million to

US$13 million). Activities on piloting information management under Component 3 were also

reduced as alternative funding became available. Component 4, which focused on project

coordination, was also reduced. While the four components of the project remained the same, the

restructuring substantially altered the subcomponents and reoriented activities to focus on

custom management, SAFI II, and transparency initiatives.

1.7 Other significant changes

14. Other changes related to scope and scale resulted from the interplay of two factors. First, the 18-month delay in effectiveness meant that the Government effectively had three years to

implement all activities, rather than the originally intended five years. In 2014, the team

conducted a second Level 2 restructuring of the project to extend the closing date by 21 months.

The justification for this change was to allow for sufficient time to develop and deliver the core

of the new financial management information system known as SAFI II, which experienced a

number of delays in implementation and was rated as Moderately Unsatisfactory at the time.

Second, the project activities were affected by the actions of the Government to fund the fiscal

and PFM reform program with other sources, including grants from bilateral donors. The fiscal

reform partly financed by this project was being carried out by a donor group (the U.S. Agency

for International Development (USAID), German Agency for International Cooperation

(Deutsche Gesellschaft für Internationale Zusammenarbeit, GIZ), European Union, and the

Inter-American Development Bank (IADB) and required significant coordination efforts.

Throughout project implementation, other donors contributed financial and technical expertise to

the core activities under the project’s components, necessitating frequent adjustments in the

activities to be financed by loan proceeds, albeit without additional formal restructuring of

components and subcomponents.

2. Key Factors Affecting Implementation and Outcomes

2.1 Project Preparation, Design, and Quality at Entry

15. The Implementation Completion and Results Report (ICR) finds the quality at entry to be

5

Moderately Unsatisfactory because of the shortcomings in the project design, which included an

ambitiously large number of technically complex reforms underpinned by weak diagnostics.

Background Analysis

16. The project design took into account lessons identified in the program and strategic assessments under the CPS, as well as the project-specific lessons highlighted in the ICR of the

recently closed El Salvador Public Sector Modernization Technical Assistance Loan Project.2

The team specifically noted two lessons from the ICR of relevance to the design of the operation.

The first was the creation and maintenance of well-staffed monitoring and evaluation (M&E)

units, given that the other implementing agency staff are overwhelmed with ‘getting things

done’. The Project Appraisal Document (PAD) described the need for effective coordination and

included a component for project coordination that would cover these functions. A second lesson

that the team incorporated was assigning project activities to appropriate agencies. The ICR for

the El Salvador Public Sector Modernization Technical Assistance Loan Project advised that the

project “components should be located within agencies that consider them to be a strategic

priority, otherwise progress will be compromised.” The design of Component 3 represents a

practical operationalization of this recommendation because activities were selected based on

explicit demands from implementing agencies and units.

17. However, a few lessons from the previous ICR were not fully considered in the PAD and underlying diagnostics were weak in some of the reform areas. From the previous project’s ICR,

technical assistance for change management and monitoring were highly recommendable in

supporting the Government’s implementation of reforms. While the PAD included resistance to

change as one of the key risks, none of the project activities contemplated change management

strategies nor was there an assessment of the capacity and engagement of senior and technical

staff in pursuing the PFM reform program. Rather, change management was interpreted as

training for staff. In addition, the previous El Salvador Public Sector Modernization Technical

Assistance Loan Project had also experienced problems with the Government combining various

fund sources with bilateral grants from other donors and had gone through four closing date

extensions. Finally, the depth of background analysis in some of the proposed reform areas was

thin. For example, the PAD contained no assessment of the readiness of El Salvador to adopt

International Public Sector Accounting Standards.

18. Assessment of the project design. The project was designed to address a number of key issues of relevance to authorities in strengthening both revenue and expenditure management

functions. The team drew on recent diagnostic assessments to identify priority actions, most

notably, the findings of the 2009 Public Expenditure and Financial Assessment (PEFA). The

PEFA results underscored the need to improve the strategic orientation of the budget through

introduction of a medium-term focus and a programmatic classification. However, the design

lacked a thorough assessment of the enabling environment for upgrading the financial

management information system (SAFI) and embarking on a complex integration with other

systems (human resources, debt, public investment). Designing and implementing a

comprehensive financial management information system project is a difficult process and must

2 El Salvador Public Sector Modernization Technical Assistance Project (1997–2007) (P007164) of US$25 million,

which was rated Satisfactory.

6

be carefully sequenced with other PFM reforms (such as establishing a Treasury Single Account).

How the proposed reforms would be sequenced under the Government’s PFM reform strategy

was notably absent in the PAD. The project length was also very short, based on what the

literature suggests for these types of reforms. According to a World Bank study analyzing 25

years of experience in implementing financial management information system projects,

financial management information system projects take a minimum of six to seven years to

complete and often require multiple operations for full integration and rollout of the system

across the government.3 The design of the project included major PFM reforms that were

sequenced in parallel with automation, including improvements to the budget classification,

Treasury Single Account operations, and cash management functions.

19. Adequacy of Government commitment. The project was designed in close collaboration with senior leadership of the MoF, and the Minister of Finance assumed an

important leadership role in advocating for the project in the Congress. The project anticipated

the need to establish a Coordination Committee under the direction of the Vice Minister of

Finance with all line Directors as members of the committee. DINAFI was to serve as the

Technical Secretariat for the Committee, and the Vice Minister of Finance was to play a

championship role for the overall project. However, there was uneven involvement of the

beneficiary units across the MoF in the design of the activities in Component 2, leading to some

confusion of accountabilities and coordination once implementation began.

20. Assessment of risks. This operation was considered to pose Substantial risk due to risks at the country and operational levels. The highest risk identified was the risk related to the

approval of the loan in Congress because the new administration did not have the required

majority to approve World Bank loans in the National Assembly. To mitigate this risk, the

Government and the World Bank engaged in a consensus-building process, including the

involvement of the opposition in defining priorities and providing the opportunity to comment on

project design. In addition, the loan was presented with the CPS and a package of loans to be

approved by Congress, which represented a carefully selected program focused on priorities

where there was broad support. Operationally, specific risks identified included limited capacity

of the Government (the new administration had assumed power after 20 years of opposition rule

and had limited experience in working in the Government), the lack of financial management

and procurement experience of the implementing agency, and resistance to change. The proposed

mitigation measures included setting realistic time frames and achievable targets in the Results

Framework, a flexible approach for selecting activities to be financed through Component 3,

hiring of specialized personnel for financial management and procurement, and an analysis of

relevant political and economic factors that may undermine progress.

21. A number of these risks materialized during implementation. Once the package went to Congress, this loan was the last of the World Bank projects to be approved, leading to the 18-

month delay in effectiveness. The operational risks proved to be high, rather than substantial,

during the implementation, leading to deserted procurement processes and resistance to reform

by beneficiaries. The risk of inadequate coordination between donors supporting the PFM reform

was not initially identified in the PAD as a risk. Given the lessons learned from the previous

3 Dener, Cem, Joanna Watkins, and William Dorotinsky. “Financial Management Information Systems: 25 years of

World Bank Experience on What Works and What Doesn’t.” a World Bank Study, 2011.

7

project, this should have been contemplated in the risk assessment for this project. It was only

during the 2011 restructuring, that an additional risk was identified regarding the implementation

of complementary activities (previously under the scope of the loan) with alternative financing

sources. This risk related to the timely implementation of these complementary activities and

was identified as Moderate. The World Bank should have been more explicit about the risks of

donor coordination at design and ensured that the activities financed by World Bank proceeds

were not subject to other donor interventions.

2.2 Implementation

22. During implementation, several factors negatively affected progress. With an 18-month delay in effectiveness, implementation was off to a rocky start. It took more than a year to launch

project activities, explained mainly by the number of internal and external actors to coordinate;

the scope and technical complexity of the reforms (covering customs, PFM reforms and system

integration, and transparency); and the technical capacity of the Project Implementation Unit

(PIU). In the Implementation Status and Results Reports (ISRs), the rating for overall

implementation progress oscillated between Moderately Satisfactory, Moderately Unsatisfactory,

and Unsatisfactory over the course of implementation. The factors that negatively affected

progress include donor coordination, weak implementation agency and PIU capacity to execute

project activities, and overall governance of the fiscal and PFM reform program.

23. Donor coordination. The PFM reform program—partly financed by the loan (Component 2)—was carried out by a donor group (USAID, GIZ, European Union, World Bank,

IADB), which required significant coordination efforts to ensure that each reform, as well as the

integrated financial management information system (SAFI II), were compatible and functional.

At project effectiveness, it was agreed that all donors involved, except for the World Bank,

would contribute grants and/or non-reimbursable assistance to various elements of the reform

program. The MoF preferred the use of non-reimbursable funds for process reforms and wanted

to use loan proceeds mainly for equipment and software development of the new SAFI II. In this

context, each donor was guided by distinct objectives, incentives, and deadlines without unified

objectives under the reform agenda defined by the Government. Aligning these incentives

proved challenging, and the Government’s focus on exhausting grant funds led to delays in loan

disbursements (it took 11 months for the first loan disbursement after effectiveness). Donor

meetings were regularly organized to address issues, but interactions were mainly at a

political/general level, even though the reform was highly technical. Notably absent from the

dialogue were technical discussions on the sequencing of reform interventions and alignment of

methodologies and objectives. As noted in ISR 8, donor interests were not necessarily aligned

given their own internal commitments to prioritize their programs. In the development of an

integrated financial management information system, loan activities substantially depended on a

close sequencing and alignment with USAID interventions, which were developing the treasury

and accounting modules of SAFI II, accounting reforms, and partially funding the contract of the

first project manager. Despite attempts to work together, coordination remained tumultuous

throughout implementation. The MoF’s coordination of donor activities and funds was also very

limited. Though DINAFI was the central unit responsible for coordinating donor interventions

each Directorate managed its own donor projects in an independent and isolated way. At one

point, the ministry had roughly 55 projects with distinct sources of financing.

8

24. The role of the implementing agency (DINAFI and PIU). Over the course of the project, the head of DINAFI, the unit responsible for overall project coordination and monitoring,

changed three times and key staff in the PIU left. These changes in management negatively

affected the execution of project activities. The PIU had not managed a World Bank project

before and required time and training to understand World Bank policies and complex

procurement procedures. With the exception of the Financial Management Specialist, hired with

loan proceeds, the PIU was mainly formed by a small number of MoF employees (ranging from

four to six persons over the course of the project) who had full-time jobs in addition to their PIU

functions. This lack of full-time staff to attend to the operational, technical, and donor

coordination aspects of the project contributed to delays in implementation and elevated the costs

of supervision and assistance by the World Bank team. As noted in the ISRs, the Government

had committed to strengthen the procurement function by hiring a Procurement Specialist, but

this was not done until near project closing. The Government’s justification was to not distort the

internal job market (salaries would be higher for a person financed through project funds). The

project management ratings oscillated between Moderately Satisfactory, Moderately

Unsatisfactory, Unsatisfactory, and was rated Highly Unsatisfactory in the last two ISRs.

25. Governance of the overall PFM reform. While the senior management of the MoF continued to view the project as strategically important throughout implementation, a number of

critical decisions related to the integration of treasury, budget, and accounting processes were

decentralized to lower levels (to Directors and, sometimes, external consultants), jeopardizing

the integration of modules under SAFI II. At the start of the project, a Coordination Committee

was established, headed by the Vice Minister, with the task of resolving issues of coordination

between the different modules with all relevant Directors present. However, the committee was

not fully operational until the last year of the project. While the original intention was to meet on

a bimonthly basis, in practice, meetings were held only a few times every year. The frequency

increased in the last year, but it was not sufficient for a project that needed quick and definitive

solutions at key points in time. Finally, as documented in the Aide Memoires, the Minister took a

strategic decision to allow the information technology (IT) development of SAFI II along two

parallel tracks—one financed by the World Bank and led by DINAFI and another financed by

USAID and led by a contractor. Unfortunately, the different teams used distinct methodologies

to build the system and had different perspectives on the functional aspects of the system.

Implementation of Component 1 (Customs)

26. This component required a number of large and complex procurement processes for equipment and software to improve customs control systems. A major procurement was that of

the commercial nonintrusive scanners (básculas camioneras), which failed four times before it

was eventually deserted. The reasons behind these repeated failures were mistakes in the

technical specifications, inadequate market assessments, and difficulties in adhering to World

Bank procurement procedures. To avoid another procurement failure with the scanners, at one

point, DINAFI and the Customs Directorate jointly reviewed the bidding documents and carried

out an on-site visit to the locations in which the weighing equipment would be placed. As a result

of the visit, it was found that one of the locations was not suitable for placing a scale because of

weight restrictions. The World Bank team attempted many times to support the Government in

the technical definitions of the goods and services and in trainings on World Bank procurement

procedures. Despite these procurement failures, the Government moved forward with the

9

nonintrusive customs control strategy by acquiring scanners on a concessionary basis.

27. The other major activity of this component was the upgrading of the customs software system. A contract was issued with United Nations Conference on Trade and Development

(UNCTAD) to purchase their Automated System for Customs Data platform (SIDUNEA, for its

Spanish acronym). This platform automates and integrates customs business processes (for

example, transit, regimes, and control of deposits and guarantees). This contract suffered a

number of setbacks because of delays in start-up and major differences between the Government

and UNCTAD with regard to the customization of the software. There were problems with the

interoperability between existing customs systems (for example, the transit system) and

SIDUNEA. The Government filed a complaint with UNCTAD for the development of the

system, citing problems with the low level of attention provided by the technical staff of

UNCTAD to the project, the parametrization of the software, and other technical aspects to make

it web accessible. Though the system was close to finalization by the time of the project’s

closing, unresolved contractual issues remain between UNCTAD and the Government.

Implementation of Component 2 (PFM)

28. This component suffered several setbacks during implementation because of resistance to the reform by different divisions within the MoF, major donor coordination problems, weak

capacity of the coordinating unit, and lack of a Project Director for the design and

implementation of SAFI II.

29. Once the project became effective, it was clear that the initial design of the PFM reform program—specifically, the upgrading and integration of the financial management information

system—was not feasible. The PFM reform strategy was revised, leading to a gradual

implementation and sequencing of reforms. Initially, the design of SAFI II was to include a

considerable number of modules, including public investment and human resources. In view of

the accumulated delays and the challenges posed by interagency coordination and ensuring that

all modules (often financed by different donors) were compatible with one another, the

Government decided at the end of 2013 that it was necessary to pursue a gradual implementation

approach and, as such, reduce the scope of SAFI II for the first stage. Recognizing the need for

sequencing of the PFM reform, the Government modified its reform approach into three phases.

The first phase of SAFI II was reduced to the ‘financial core’ budget, treasury, and accounting

modules.

30. In selecting the software solution, the Government decided to pursue a locally developed software (LDSW) solution rather than purchase a commercial, off-the-shelf solution.

4 The

Government tried to procure a firm to implement the LDSW system. However, this procurement

failed twice because of a lack of firms willing to bid on the project. The contingency plan was to

contract 20 local developers because there was not enough time before project closing to conduct

another bidding process. Given the size of the project and the lack of technical leadership by the

Government, donors consistently called for the Government to put in place a Project Manager

with solid technical background for SAFI II. Eventually, a Project Manager was selected, but

4 LDSW is usually less expensive and the client owns the rights to the software. Necessary modifications to the

software can be done more easily. However, it requires substantial leadership and coordination to implement the

decisions needed for LDSW.

10

within three months, the manager resigned. In September 2015, the World Bank financed a new

Project Manager but the reception by some Government Directors was complicated. Directors

had already experienced distinct approaches from various consultants and it was hard to have any

agreement between World Bank consultants, USAID, and the Directors. The two main donors

involved in SAFI II (the World Bank and USAID) had conceptual differences and utilized

distinct methodological approaches. Within the MoF, coordination among the various divisions

posed a serious challenge, and there was no clear hierarchy for resolving challenges. The

situation deteriorated to the point that the Project Manager and main technical consultants

resigned. At that point, DINAFI continued software development with the methodologies and

approaches defined by the last Project Manager and technical consultants and finalized the

development of the budget formulation, budget execution, and management of human resources

structures modules with local consultants. The World Bank provided significant technical

support and used project supervision missions to convene stakeholders to discuss pending issues,

achievements, and next steps. Aside from this technical assistance, loan proceeds were directed

to upgrading the hardware of the MoF. Detailed Aide Memoires provide evidence of the World

Bank team’s advice to the Government on system implementation.

31. As originally conceived, DINAFI did not play a major leadership role in ensuring that the overall PFM reform strategy was well articulated. DINAFI started to drift from the strategy of

developing an integrated financial management information system (SAFI II) by financing

modules with other funding sources, thereby limiting the strategic orientation across

components/subcomponents. DINAFI was thrifty with loan funds, mindful that they could

finance activities using grants from donors. Organizationally, DINAFI was also limited in its

ability to coordinate and resolve conflicts between Directors (necessary for the integration of

processes for SAFI II). Hierarchically, DINAFI is on the same level with the other directions in

the ministry. Legally, DINAFI’s responsibility for implementing the new system was by way of

delegation from the minister through an Executive Order. This was important because in the

Organic Budget Law (Ley Orgánica de Administración Financiera del Estado), DINAFI was

responsible for integration and not for functional modifications.5 As a result, the Directors of

Budget, Treasury, and Accounting did not think they had to comply with DINAFI’s requests.

Implementation of Component 3 (Transparency)

32. Despite some initial delay in the start of activities, once the project became effective, implementation proceeded smoothly. The Law on Access to Public Information became effective

in 2011 and dissemination activities commenced under the project. Initial delays under this

component were because of a lack of communication between the PIU and the SCPT and delays

in planning its activities in accordance with the Operations Manual. The World Bank served as a

facilitator between the PIU and the SCPT to ease communication and speed up implementation,

which then proceeded relatively smoothly.

2.3 Monitoring and Evaluation (M&E) Design, Implementation, and Utilization

33. The project’s M&E framework is considered to be modest. The original design of the operation’s M&E framework exhibited weaknesses in the logic connecting project inputs with

5 See Articles 9 and 24.

11

outputs and outcomes. While some of the original shortcomings were overcome during the first

restructuring with the dropping of some indicators and revisions to targets, weaknesses in the

logical framework and the formulation of indicators proved challenging in the evaluation of the

operation’s impact.

34. M&E design. The Results Framework included in annex 3 of the PAD specified four project outcome indicators and 20 intermediate outcome indicators which monitored a mixture of

higher-level outcomes and outputs of subcomponent activities. The original Result Framework

included a number of PEFA indicators to measure performance in PFM practices and used the

2009 PEFA to establish the baseline. However, in some cases, the design of the Results

Framework was not amenable to changes in the activities of subcomponents, because a number

of the intermediate indicators were directly linked to outputs such as the procurement of software

or number of trainings conducted (for example, The Human Resource Management System

[Sistema Integral de Recursos Humanos - SIRH] is updated and interoperable with SAFI). Some

of the indicators were also not well defined. For example, one of the PDO indicators was,

“stronger planning, budgeting, treasury, and accounting systems in place, in line with

international standards.” It was not clear how this would be measured. This was only partially

corrected with the changes introduced in the Results Framework in the first restructuring where

many indicators were dropped or revised. Such changes are indicative of elements of both a

corrective and adaptive restructuring to fix design flaws in the original indicators and improve

the relevance of the activities and associated Results Framework, in light of the delays to the

effectiveness in the project and changes in loan-financed activities.

35. M&E implementation. DINAFI coordinated the monitoring of project activities throughout implementation; however, the PIU within DINAFI was small and a person was never

hired to take on the M&E functions. Though the PAD did note the need for careful consideration

of M&E arrangements (as a lesson learned from the ICR of the previous project), this was not

translated into practice. Data collection relied heavily on meetings and reports by the

counterparts involved in each of the subcomponents, which on occasion, led to delays in the

provision of timely information. The lesson from the ICR of the previous operation on M&E was

not translated into practice during implementation. Information on key project outputs and

indicator progress was regularly collected as part of the World Bank’s supervision efforts. In the

ISRs, the ratings for M&E ranged from Moderately Satisfactory to Satisfactory, with the

exception of ISR 12 when it was rated Moderately Unsatisfactory, noting “the PIU has not

satisfactorily monitored the project. Production of project performance information is not

adequately systematized and takes significant time to be integrated and sent to the Bank.”

36. M&E utilization. As originally conceived in the PAD, the MoF was to establish a Coordination Committee under the direction of the Vice Minister of Finance responsible for

strategic planning and project monitoring. DINAFI was to serve as the Technical Secretariat of

this committee, with specialized personnel to be hired for this purpose. Participating agencies

were to be present on the committee to ensure all activities were monitored timely and

substantively. However, in practice, this Coordination Committee mainly focused on activities

related to PFM reform and did not take on a substantive monitoring role across all components.

12

2.4 Safeguard and Fiduciary Compliance

37. Safeguards. No safeguard policies were triggered by the project.

38. Financial management. Financial management performance is considered to have been Satisfactory. Some issues arose at the start of the project with the need to hire a dedicated

Financial Management Specialist in the PIU. Once this issue was resolved, financial

management performance was rated Satisfactory in the remaining ISRs (4–13).

39. Procurement. Procurement performance is considered to have been Moderately Unsatisfactory. In all of the ISRs, procurement performance was rated Moderately Satisfactory;

however, the PIU did not hire a dedicated Procurement Specialist as was initially agreed with the

World Bank until the end of the project. There were recurrent delays and failures in the largest

procurements of the project, particularly in equipment and software in the customs and PFM

components. Procurement plans were revised substantially during implementation and a large

number of complex procurement activities were undertaken—15 International Competitive

Bidding packages (US$6.9 million in total, almost half of the disbursements). There were 18

cancelled procurement packages. Staff capacity in the implementing agencies responsible for the

technical specifications was also very weak. This situation could have been avoided by careful

planning, bringing experienced IT experts to the team, and learning more from similar activities

using available World Bank resources.

2.5 Post-completion Operation/Next Phase

40. There are no planned World Bank supported follow-up operations. The Government is committed to continue working on the remaining modules of SAFI II and will continue to

receive the support of other donors, such as USAID and IADB, on parts of the PFM reform

agenda. SAFI II will need to be rolled out to 118 institutions, and the latest decision of the MoF

is that it will continue the rollout using internal technical expertise. Elections for Congress are

scheduled for 2018 and presidential elections are scheduled for 2019. Though uncertainties

naturally attend such changes in administration, it is unlikely that the activities associated with

the purchasing of hardware will be undermined by the change in administration. On the other

hand, there are some risks that a transition in administration will affect the transparency agenda

and specifically, the main beneficiary agency, the SCPT. Secretariats are established by decrees

and they could disappear in 2019 with the elections. The SCPT is aware of this and has

established a far-reaching network of public servants in each of the institutions they work with to

improve the sustainability of their interventions.

3. Assessment of Outcomes

3.1 Relevance of Objectives, Design and Implementation

41. The PDO continues to be relevant to both Government priorities and to the World Bank’s strategic priorities and, as such, relevance is rated Substantial. The World Bank’s Country

Partnership Framework covering 2016 to 2019 sets out a strategic objective of promoting the

efficiency of public spending under Pillar 2 to foster sustainability and resilience. Moreover, the

13

Government’s Five-Year Development Plan (2014–19)6 sets out an agenda on fiscal policy and

PFM with the following lines of action: increase tax collection, increase and improve the

execution and quality of public investments, improve the efficiency of social programs and

subsidies, and generate primary savings so that finances are sustainable in the medium and long

term. The 2016 Article IV mission of the International Monetary Fund (IMF) concluded that El

Salvador continues to suffer from significantly lower growth than neighboring countries with

GDP growth expected to be 2.4 percent in 2017. According to IMF estimates, the fiscal deficit is

expected to widen further over the medium term without austerity measures, and public debt is

projected to rise to above 70 percent of GDP by 2021. Wage pressures, higher interest rates,

additional security costs, and public investment projects are all expected to add to demands on

fiscal resources.

42. The relevance of the project’s design to the objective is considered Modest. While the project was restructured and substantially adjusted over time to respond to changes in

Government priorities, the PDO was not adjusted nor was the design of the components. The

design of the subcomponents of the project was not fully aligned with the stated objective and

this disconnect was not addressed through a Level 1 restructuring. The team faced the constraint

that the project would have to go back to the National Assembly for approval if there were any

changes in objective (which would require a Level 1 restructuring). The subcomponents

corresponding to the revenue management objective were considerably reduced at effectiveness

and reoriented to customs management. The component on PFM was also significantly altered

during implementation when the World Bank stopped financing the development of the core

modules of SAFI II. The PDO was not adjusted to align with the change in these activities. The

Results Framework was also not well suited to measure the project objectives. A lack of internal

logic connecting the project’s inputs, outputs, and outcomes make the project’s contributions to

the development objective difficult to evaluate.

3.2 Achievement of Project Development Objectives

43. Before addressing the achievement of the PDO, it is important to clarify how the efficacy of the project is rated. In line with the ICR guidelines, the overall project performance is a

weighted average of the three primary objectives as described in the PDO. Each objective is

rated equally. To assess each objective, PDO and intermediate outcome indicators, as well

supplementary evidence, are used to arrive at an overall assessment. Annex 10 provides a

detailed analysis of this evidence base, organized by objective.

44. From the PDO, one can identify three main objectives of the project: (a) increase the effectiveness and efficiency of revenue management, (b) increase the effectiveness and

efficiency of expenditure management, and (c) enhance the accountability and transparency in

the public sector.

45. To analyze the first objective, one should consider whether the project strengthened the institutional capacity to increase the effectiveness and efficiency of revenue management. At

design, the project components were focused on strengthening the links between tax and customs

with the aim of improving revenue management. However, at effectiveness, the revenue

6 El Salvador Productivo, Educado y Seguro. Plan Quinquenal de Desarrollo (2014–19).

14

management component was substantially adjusted to focus solely on the customs aspects of

keeping out undesirable goods, trade facilitation, and back-office improvements to customs

systems. The role of customs management is generally to keep undesirable goods out while

collecting revenue and taxes on goods that are allowed in, as well as facilitate trade.7 The PDO

indicator on increasing tax revenue was dropped. The objective of revenue management

remained the same, but the project effectively de-emphasized revenue improvement activities.

46. The PDO and intermediate indicators focus on the effectiveness and efficiency of customs management. The PDO indicator was partially achieved—the target was to have seven

customs points with nonintrusive control methods, and at project closing, six were achieved.

However, none of the six customs points’ nonintrusive methods were actually funded through the

loan (because of procurement problems), but the strategy funded through the project did

contribute to the end result. Before the project, the strategy and mechanisms in customs points

did not exist. The intermediate indicator on web-processed transactions was achieved. The

intermediate indicator related to reducing the hours to clear merchandise was partially achieved,

reduced from 60 to 21 hours (the target was 17 hours).

47. While the indicators were partially achieved, it is not clear that there were any attributable improvements in revenue management as a result of project activities. From this

assessment, it appears that the project had a negligible contribution to increasing the

effectiveness and efficiency of revenue management.

48. To analyze the second objective, one should consider whether the project strengthened the institutional capacity of the MoF to increase the effectiveness and efficiency of expenditure

management. The PDO indicator was “stronger planning, budgeting, treasury, and accounting

systems in place, in line with international standards.” Given the breadth of areas that this

indicator covers, the target value was defined as the SAFI II system to be fully developed and

ready to be launched and to include program-based budgeting and payment processing through

the Treasury Single Account. According to the ISRs, this indicator was partially achieved

(because the budget formulation and execution modules funded by a combination of World Bank

and USAID funds of SAFI II were completed by project closing), though a more nuanced

assessment of this indicator would likely have relied on PEFA indicators, which appeared in the

original Results Framework. El Salvador’s two most recent PEFAs are from 2009 (at the time of

project design) and 2013. A comparison of the relevant PEFA indicators between 2009 and 2013

provides no evidence of progress on the relevant indicators (see annex 11).

49. With regard to stronger budget planning, program-based budgeting, and the medium-term expenditure framework (MTEF), these reforms are embedded in the design of the new SAFI II

modules. Treasury reforms, which were initially conceptually guided by the World Bank and

operationalized by USAID, did achieve some results. Payment processing for Central

Government payments (excluding payroll) is now being done through the Treasury Single

Account. Full implementation of the Treasury Single Account (across all institutions and for the

payroll) is expected once SAFI II is fully operational. Even with partial payments through the

Treasury Single Account, the Government has seen a reduction in the time to pay private

contractors. Before the reform, it took two weeks (15 days) for funds to arrive to providers, now

7 Gerard McLinden et al, “Border Management Modernization,” World Bank (2011).

15

it takes one day or less. This has helped resolve serious issues in paying health and education

providers. The Treasury now also has real-time information on its cash balances. All of the debts

are registered and finance officials can conduct a daily analysis of the operations, which is

critical given the liquidity crisis facing the country. Nonetheless, project performance can only

be partially attributed to these results, given the predominant role of USAID in this area.

50. Both intermediate indicators were also achieved. The SAFI II modules—budget formulation, budget execution, and the Management of Human Resources Structures—were

developed by project closing, but they are not operational yet. The Government expects to put

the modules in operation in the 2017/18 budget process. Training sessions on how to use the

modules were conducted for the Government staff. At closing, SAFI II was not operational.

Modules that were the responsibility of the World Bank have been developed, and other modules

being developed by USAID are still in development.

51. Other achievements not captured in the Results Framework were the rollout of results-based budgeting and the substantial upgrading of the MoF’s IT hardware. The legal framework

has been modified to allow institutions in the central public administration to prepare their

budgets following a programmatic classification. As of September 2016, 79 out of 84 public

sector institutions (excluding 30 hospitals) have a programmatic budget structure, which is ready

to be applied in the preparation of the 2018 budget through the SAFI II budget formulation

module. Approximately 1,820 Government officials of the public administration have been

trained on the results-based budgeting techniques. As a result of the project-financed investments

in upgrading hardware, the MoF’s information and communication technology (ICT) capabilities

have improved. About 90 percent of the hardware of the MoF has been updated through the

purchasing of new computers, servers, video conferencing equipment, critical network

infrastructure, and a data warehouse. This infrastructure has contributed to improvements in the

processing, storage, accessibility, and security of data which in turn has contributed to an

increase in the number of online citizen/business services provided by the MoF (see section on

unintended outcomes in section 3.5 for further details).

52. While there is no evidence of improvement in the efficiency and effectiveness of expenditure management as measured by comparing relevant PEFA indicators between 2009 and

2013, the project was able to introduce some of the inputs necessary for eventually realizing

these changes (new systems, processes, hardware, and so on). From this assessment, it appears

that the project had a negligible contribution to increasing the effectiveness and efficiency of

expenditure management.

53. The third objective, as stated in the PDO, was to enhance the accountability and transparency in the public sector. No PDO indicator was established for the transparency

component, but the related intermediate indicator was achieved and surpassed, with 50 training

and dissemination initiatives conducted to strengthen Government transparency. However, this

indicator does not provide much evidence of the impact of the range of activities financed under

this component. From supplemental evidence, it appears that this component had a substantial

impact. With regard to transparency initiatives, the project supported improvement and

expansion of the InfoÚtil Portal (originally developed with funding from the IADB in 2012),

which enables citizens to access useful day-to-day public information (public procedures

and services) for health, education, economy, and finances. This portal allows citizens to obtain

16

updated information and submit suggestions, opinions, or complains about public management.

The portal has received more than 5.3 million visits in the last three years. The project also

financed a user satisfaction survey of the portal.8 The survey results suggest that users were

generally satisfied with content, design, and accessibility of the portal (see annex 10 for details

on the results of the survey). The project also supported the development of a Public Jobs

Vacancy Portal (portal empleos publicos) that was launched in December 2015, which

consolidates vacancies from 45 public institutions. The project also contributed to the design of

the proposal for open data policy and the creation of the Gobierno Abierto Portal which

integrates information, requests for information, complaints, and rankings of the performance of

public institutions. Evidence of the impact of these reforms are reflected in improvement to El

Salvador’s Open Government score. From 2014 to 2016, Open Government improved from 0.37

to 0.51.9

54. With regard to accountability initiatives, this component supported the national and international dissemination of El Salvador’s Law on Access to Public Information through

television campaigns, brochures, and trainings in schools and public institutions. Eighty-three

Offices of Information and Response (OIRs) were created within the Central Government. As of

November 2013, the OIRs had received 24,021 requests for information and responded to 90

percent of the requests.10

Higher-level evidence of the impact of these reforms reflected in

improvement to El Salvador’s voice and accountability scores. From 2012 to 2015, the score

improved from 46.5 to 50.7.11

For a comprehensive analysis of the results of each objective, see

annex 10. For the list of outputs by component, see annex 2.

3.3 Efficiency

55. Project efficiency is considered to be negligible. Given the nature of the project, no formal economic cost-benefit analysis was performed at the time of design. While results, such

as increases in the web-processing of various transactions and reductions in hours to clear

customs merchandise are likely to have contributed to savings from enhanced efficiency, it is

difficult to quantify such benefits ex post given the lack of concrete data.

56. The PAD’s Economic and Financial Analysis listed a number of potential financial and economic impacts mainly related to efficiencies generated from improvements in revenue

collection, facilitation of payments to public sector suppliers, and reductions in idle balances in

commercial banks. To the extent possible, these impacts have been updated based on the results

of the project and are captured in annex 3. Some of the achievements, such as systems

development and capacity building, affect back-office government processes, and while it is

expected that such improvements will end up contributing to improved services for citizens and

businesses, such gains are difficult to measure.

57. Nevertheless, aspects of project implementation likely reduced the efficiency of some activities. In line with ICR guidelines, beyond traditional measures of efficiency, the review

should take into account aspects of design and implementation that reduced efficiency. As

8 InfoÚtil, “Evaluación Ciudadana del Portal Infoútil,” 2014.

9 World Justice Project.

10 Rendición de Cuentas Fondos de Cooperación 2009–14.

11 Worldwide Governance Indicators.

http://www.gobiernoabierto.gob.sv/

17

discussed in the implementation section, the restructuring and extension of the project as a result

of procurement issues and the complexity of reform reduced the overall efficiency in a number

of the subcomponents.

3.4 Justification of Overall Outcome Rating

Rating: Unsatisfactory

58. The overall outcome rating is considered to be Unsatisfactory (see table 2), reflecting major shortcomings in the operation’s achievement of its objectives. While the objectives of the

project are still relevant, the design of the project was not fully consistent with the objectives.

The efficacy of project interventions was negligible in terms of revenue and expenditure

management but substantial in improving transparency of the public sector. The efficiency of the

project was negligible.

Dimension Rating

Relevance

Relevance of Objectives Substantial

Relevance of Design Modest

Efficacy (equal weight across three objectives)

Objective 1 (revenue management)

Objective 2 (expenditure management)

Objective 3 (transparency)

Negligible

Negligible

Substantial

Efficiency Negligible

Overall Unsatisfactory

3.5 Overarching Themes, Other Outcomes and Impacts

(a) Poverty Impacts, Gender Aspects, and Social Development

59. Through the dissemination and implementation of the Access to Public Information Law (signed into law in 2012) and transparency initiatives, the operation has had an impact on the

accessibility of public information for citizens. While the project was not designed to have a

direct impact on poverty, gender, or social development, access to information and transactional

services for citizens through online portals has improved. For example, the InfoÚtil Portal

was redesigned enabling citizens to access useful day-to-day public information (public

procedures and services) for a range of health, education, economic, and financial services.

(b) Institutional Change/Strengthening

60. This project has been an important catalyst in the institutional strengthening of both the MoF and the SCPT. From a long-term institutional development perspective, the activities begun

during the first loan and continued in the second loan have strengthened the MoF’s ability to

perform critical functions related to cash management, program-based budgeting, medium-term

budgeting, and customs management. Through this project, the World Bank has maintained a

continuous policy dialogue on PFM and other important areas for the country’s development

with the authorities over the last decade.

61. The institutional evolution of the SCPT also suggests that the transparency and

18

accountability agenda has taken hold in the administration. In 2009, the Secretariat was a Sub-

Secretariat of Transparency and Anti-corruption (Subsecretaría de Transpaencia y

Anticorrupción) under the Secretary for Strategic Matters of the Presidency. In 2014, it was

converted into the Secretary of Citizen Participation and Transparency (SCPT), as citizen

participation was added to its functions. The number of staff increased from 45 in 2014 to 60 in

2016. The SCPT has also built a network of public officials across institutions to promote and

incorporate tools related to transparency, citizen participation, and accountability.

(c) Other Unintended Outcomes and Impacts (positive or negative)

62. As a result of project-financed investments in upgrading hardware, the MoF’s ICT capabilities have improved. Approximately 90 percent of the hardware of the MoF has been

updated through the purchasing of new computers, servers, video conferencing equipment,

critical network infrastructure, and a data warehouse. This infrastructure has contributed to

improvements in the processing, storage, accessibility, and security of data that has contributed

to an increase in the number of online citizen/business services provided by the MoF. The

upgraded infrastructure contributed directly to improving the capacity of the MoF to make the

following citizen services available online: (a) tax refund consultations; (b) presentation of

information and declarations and payment of taxes online; (c) request forms and consultations

for tax solvency; and (d) requests for cancellation, receipt, or modifications of one’s Tax

Identification Number (Número de Identificación Tributaria) for naturalized citizens. These

services represent about 20 percent of total number of services available on the MoF’s online

portal. Along with the infrastructure, the project also contributed to improvements in ICT

knowledge of the MoF staff through trainings.

3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops

63. No beneficiary survey or stakeholder workshops were conducted.

4. Assessment of Risk to Development Outcome

Rating: Moderate

64. The progress made under the project is likely to be sustained for a number of reasons, though full implementation of SAFI II will require a concerted effort by the MoF leadership.

First, the legal framework provided by laws and decrees across a number of areas (transparency,

treasury, PFM), makes the sustainability of project outcomes more likely. Second, the use of

transparency and open data tools by the ministries and agencies has created a demand for support

from the SCPT. The tools developed through this project have been incorporated into the

agencies’ current procedures and are unlikely to be reversed. The moderate, rather than low, risk

rating for these reforms is because of potential difficulties in the full implementation of SAFI II.

The budget preparation, budget execution, and human resource modules have all been developed

and are expected to be operational in 2017/18. Nonetheless, critical interconnection issues

remain between the Treasury, accounting, and the budget divisions of the MoF to ensure that all

modules are integrated within SAFI II. Once these issues are resolved, the software modification

should be relatively straightforward given improvement to the IT capacity of DINAFI staff. It

appears likely that SAFI II will be implemented in the next 1–3 years given continued USAID

support to the reform and the Government’s expressed interest in ensuring the new system is up

19

and running.

5. Assessment of Bank and Borrower Performance

5.1 Bank Performance

(a) Bank Performance in Ensuring Quality at Entry Rating: Moderately Unsatisfactory

65. During project preparation, the World Bank team drew upon its previous experience in implementing the El Salvador Public Sector Modernization Technical Assistance Loan Project

from 1997–07 (P007164), an understanding of the political economy of El Salvador, and PFM

and tax diagnostic assessments to inform the design of reforms. The overall risk of the operation

was rated Substantial and the risk of delays in the approval of the loan in Congress was the most

significant risk identified. Other risks identified included institutional implementation capacity,

procurement, and resistance to change. While these risks were acknowledged in the PAD, the

calibration of activities and components to these risks could have been better reflected in the

design of the reforms by reducing the scope of activities. The proposed mitigation measures were

also not sufficient for reducing the high costs of coordination inherent to the design of the project.

66. Moreover, from the complex design of the project—covering tax, PFM, financial management information system, and transparency reforms—it is not clear if similar projects

from other contexts were reviewed to anticipate implementation problems. The integrated

financial management information system reform, in particular, was overly ambitious from the

start. It appears that the project did not take into account the lessons of other financial

management information system projects, which on average take seven years to complete, and

must be appropriately sequenced (starting with core budget and treasury modules, and eventually

moving to integration of additional modules such as human resources or public investment

management PIM). The large and complex procurement packages were also not given enough

attention at the design phase. It appears that the original bar was set too high for a number of the

PFM reforms. For example, the application of International Public Sector Accounting Standards

and Government Financial Statistics 2001. There was limited description of the weaknesses of

the current arrangements in the PAD or attention to how to sequence the introduction of the

standards.

67. Finally, while the PAD acknowledged that the new administration, which had assumed power after 20 years of opposition rule, might lack the political and technical experience needed

to implement the project, the design of the project was not adequately tailored to potential

problems in capacity to execute reforms. Additional support and supervision were seen as

mitigating measures rather than reducing reform expectations or coordination costs across

components.

(b) Quality of Supervision Rating: Moderately Unsatisfactory

68. Over the course of the project, the World Bank team was somewhat inconsistent in the quality of its supervision. At times, it was proactive, organizing several missions and meetings

20

with the Government and donors to align incentives and priorities and strengthen the

implementation capacity of the PIU. At other times, it was noticeably absent in defining the

reform trajectory, using restructurings to adjust the PDO or better al