document of the world bank report no: icr00003553 … · 2016. 7. 15. · document of the world...

Document of The World Bank

Report No: ICR00003553

IMPLEMENTATION COMPLETION AND RESULTS REPORT (IBRD-81290)

ON A

LOAN

IN THE AMOUNT OF US$ 15.0 MILLION EQUIVALENT

TO THE

REPUBLIC OF ARMENIA

FOR A

MUNICIPAL WATER PROJECT

December 15, 2015

Water Global Practice South Caucasus Country Unit Europe and Central Asia Region

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

ii

CURRENCY EQUIVALENTS

(Exchange Rate Effective 12/1/2015)

Currency Unit = AMD US$ 1.00 = [482.81]

FISCAL YEAR

ABBREVIATIONS AND ACRONYMS

ADB Asian Development Bank AMD Armenian Dram AWSC Armenia Water and Sanitation Company CAPEX Capital Expenditure Program CAS Country Assistance Strategy CMU Contract Monitoring Unit CPS Country Partnership Strategy DPO Development Policy Operation EBRD European Bank for Reconstruction and Development ECA Europe and Central Asia ERR Economic Rate of Return EMC Enhanced Management Contract FIRR Financial Internal Rate of Return FM Financial Management FSU Former Soviet Union FY Fiscal Year GoA Government of Armenia IDA International Development Association IFRS International Financial Reporting Standards ISR Implementation Supervision Report KfW Kreditanstlat fur Wiederaufbau KW Kilowatt l/c/d Liters per capita per day MDP Municipal Development Project MWP Municipal Water Project MWWP Municipal Water and Wastewater Project NPV Net Present Value NRW Non-Revenue Water O&M Operation and Maintenance PAD Project Appraisal Document PDO Project Development Objective PIE Project Implementing Entity PIU Project Implementation Unit PPP Public-Private Partnership

iii

PPIAF Public-Private Infrastructure Advisory Facility PSP Private Sector Participation PSRC Public Services Regulatory Committee SCWE State Committee for Water Efficiency WSS Water Supply and Sanitation YWSC Yerevan Water and Sanitation Company YWWP Yerevan Water and Wastewater Project

Senior Global Practice Director: Junaid Ahmed Country Director Mercy Tembon Practice Manager Dina Umali-Deininger Project Team Leader: Anna Cestari/Zaruhi Tokhmakhian ICR Team Leader: Anna Cestari

iv

1.

COUNTRY Republic of Armenia

CONTENTS Data Sheet

A. Basic Information ....................................................................................................... v B. Key Dates ................................................................................................................... v C. Ratings Summary ....................................................................................................... v D. Sector and Theme Codes .......................................................................................... vi E. Bank Staff .................................................................................................................. vi F. Results Framework Analysis ..................................................................................... vi G. Ratings of Project Performance in ISRs ................................................................... ix H. Restructuring (if any) ................................................................................................ ix I. Disbursement Profile .................................................................................................. x

Main Document 1. Project Context, Development Objectives and Design ............................................... 1 2. Key Factors Affecting Implementation and Outcomes .............................................. 4 3. Assessment of Outcomes ....................................................................................... 10 4. Assessment of Risk to Development Outcome ......................................................... 18 5. Assessment of Bank and Borrower Performance ..................................................... 19 6. Lessons Learned ....................................................................................................... 22 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners .......... 23 Annex 1. Project Costs and Financing .......................................................................... 25 Annex 2. Outputs by Component ................................................................................. 26 Annex 3. Economic and Financial Analysis ................................................................. 30 Annex 4. Bank Lending and Implementation Support/Supervision Processes ............ 39 Annex 5. Beneficiary Survey Results ........................................................................... 41 Annex 6. Stakeholder Workshop Report and Results ................................................... 45 Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR ..................... 46 Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders ....................... 55 Annex 9. List of Supporting Documents ...................................................................... 56 Annex 10. Additional Background Information ........................................................... 57 Annex 11. Donor Projects Active in the Water Sector in Armenia, 2012-2015 ........... 59 MAP .............................................................................................................................. 61

v

A. Basic Information Country: Armenia Project Name: Municipal Water Project Project ID: P126722 L/C/TF Number(s): IBRD-81290 ICR Date: 12/04/2015 ICR Type: Core ICR Lending Instrument: SIL Borrower: ARMENIA Original Total Commitment:

USD 15.00M Disbursed Amount: USD 14.93M

Revised Amount: USD 14.93M Environmental Category: B Implementing Agencies: Armenia Water and Sewerage Company Cofinanciers and Other External Partners: B. Key Dates

Process Date Process Original Date Revised / Actual Date(s) Concept Review: 11/09/2011 Effectiveness: 06/23/2012 Appraisal: 01/17/2012 Restructuring(s): Approval: 02/21/2012 Mid-term Review: 04/01/2014 04/08/2014 Closing: 06/30/2015 06/30/2015 C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Moderately Satisfactory Risk to Development Outcome: High Bank Performance: Moderately Satisfactory Borrower Performance: Moderately Satisfactory

C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings

Quality at Entry: Moderately Satisfactory Government: Moderately Satisfactory

Quality of Supervision: Moderately Satisfactory Implementing Agency/Agencies: Moderately Satisfactory

Overall Bank Performance: Moderately Satisfactory

Overall Borrower Performance: Moderately Satisfactory

vi

C.3 Quality at Entry and Implementation Performance IndicatorsImplementation

Performance Indicators QAG Assessments (if

any) Rating

Potential Problem Project at any time (Yes/No):

No Quality at Entry (QEA):

None

Problem Project at any time (Yes/No):

No Quality of Supervision (QSA):

None

DO rating before Closing/Inactive status:

Moderately Satisfactory

D. Sector and Theme Codes

Original Actual Sector Code (as % of total Bank financing) Public administration- Water, sanitation and flood protection 31 31 Water supply 69 69

Theme Code (as % of total Bank financing) Rural services and infrastructure 35 35 Urban services and housing for the poor 65 65 E. Bank Staff

Positions At ICR At Approval Vice President: Cyril E Muller Philippe H. Le Houerou Country Director: Laura E. Bailey Asad Alam Practice Manager/Manager: Dina Umali-Deininger Pier Francesco Mantovani Project Team Leader: Anna Cestari Zaruhi Tokhmakhyan ICR Team Leader: Anna Cestari ICR Primary Author: Amanda Joan Goksu F. Results Framework Analysis Project Development Objectives (from Project Appraisal Document)In line with the Government of Armenia's strategy for continual improvements in service delivery, the Project Development Objective (PDO) is "To support improvement of the quality and availability of water supply in selected service areas of the Armenian Water and Sewerage Company (AWSC)." The selected service areas are: Masis town, Ashtarak town, and Echmiatsin town, and their surroundings. Revised Project Development Objectives (as approved by original approving authority)

vii

(a) PDO Indicator(s)

Indicator Baseline Value

Original Target Values (from

approval documents)

Formally Revised

Target Values

Actual Value Achieved at

Completion or Target Years

Indicator 1 : Increased weighted average daily supply of drinking water service in selected areas from 12.3 hours daily to 16.6 hours daily Value quantitative or Qualitative)

12.3 hours 16.6 hours 17.6 hours

Date achieved 02/21/2012 02/22/2012 09/30/2015 Comments (incl. % achievement)

Target Exceeded. As of the end of project (6/30/15) the result was 17.4 hours; the number increased to 17.6 by 9/30/15.

Indicator 2 : Decreased weighted average non-revenue water in selected areas from 83.50 percent to 70 percent Value quantitative or Qualitative)

83.5% 70% 70%


Target Met. As of the end of project (6/30/15) the result was 74%; the number decreased to 70% by 9/30/15.

Indicator 3 : Increased ratio of billing and collection from 93.0 percent to 98 percent Value quantitative or Qualitative)

93.5 98 98


Target Met. As of the end of project (6/30/15) the result was 96.5%; the number increased to 98% by 9/30/15.

(b) Intermediate Outcome Indicator(s)

Indicator Baseline Value

Original Target Values (from

approval documents)

Formally Revised Target

Values

Actual Value Achieved at

Completion or Target Years

Indicator 1 : Decreased annual electricity consumption in KW/m3 Value (quantitative or Qualitative)

.23 .17 .16


Target Exceeded. As of the end of project (6/30/15) the result was .17; the number decreased to .16 by 9/30/15.

viii

Indicator 2 : Increased average domestic metered consumption per metered registered inhabitant (in l/c/d) Value (quantitative or Qualitative)

88.5 116 119


Target Exceeded.

Indicator 3 : Decreased percentage of subscribers with more than four months arrears Value (quantitative or Qualitative)

27.7 21.0 17.9


Target Exceeded. As of the end of project (6/30/15) the result was 18.5; the number decreased to 17.9 by 9/30/15.

Indicator 4 : Number of automation systems installed at pumping stations and daily regulation reservoirs Value (quantitative or Qualitative)

0.0 28 35


Target Exceeded.

Indicator 5 : Increased percentage of water disinfected Value (quantitative or Qualitative)

92.9 96 99.3


Target Exceeded. As of the end of project (6/30/15) the result was 99.5; the number increased to 100 by 9/30/15.

Indicator 6 : Decreased amount of water production (l/c/d) Value (quantitative or Qualitative)

752 489 481


Target Exceeded. As of the end of project (6/30/15) the result was 481; the number decreased to 329 by 9/30/15.

Indicator 7 : Increased domestic water metering (as a percentage of domestic subscribers provided with water meters in Service Area of AWSC) Value (quantitative or Qualitative)

81.2 87 88

Date achieved 02/21/2012 02/22/2012 09/30/2015

ix

Comments (incl. % achievement)

Target Exceeded. As of the end of project (6/30/15) the result was 95.3; the number decreased to 88 by 9/30/15.

Indicator 8 : Number of people in urban areas provided with access to improved water sources under the project Value (quantitative or Qualitative)

0 99263 100978


Target exceeded

Indicator 9 : Number of people in rural areas provided with access to improved water sources under the project Value (quantitative or Qualitative)

0 33328 31089


Target not met. As of the end of project (6/30/15) the target was exceeded (33,300) but after confirmation of customers was reduced to 31,089 on 9/30/15.

Indicator 10 : Piped households (customers) with water connections that are benefiting from rehabilitation works undertaken by the project Value (quantitative or Qualitative)

0 33609 32848


Target not met. As of the end of project (6/30/15) the target was exceeded (33645) but after confirmation of customers was reduced to 32,848 on 9/30/15.

G. Ratings of Project Performance in ISRs

No. Date ISR Archived DO IP Actual Disbursements

(USD millions) 1 10/15/2012 Satisfactory Satisfactory 1.00 2 01/07/2013 Satisfactory Satisfactory 2.94 3 08/07/2013 Satisfactory Satisfactory 5.67 4 01/24/2014 Satisfactory Satisfactory 7.77 5 05/27/2014 Satisfactory Satisfactory 8.87 6 11/19/2014 Satisfactory Satisfactory 11.78 7 06/05/2015 Moderately Satisfactory Moderately Satisfactory 13.48 8 07/08/2015 Satisfactory Satisfactory 14.96

H. Restructuring (if any) Not Applicable

x

I. Disbursement Profile

xi

Executive Summary The Municipal Water Project (MWP) aimed at supporting improvement in the quality and availability of water supply in selected service areas of the Armenia Water and Sewerage Company (AWSC). The operation followed two other successful operations, the Yerevan Water and Wastewater Project (YWWP) and the Municipal Water and Wastewater Project (MWWP). The project is rated Moderately Satisfactory having substantially achieved its development objectives. All three key outcome indicators targets have been met, and 80% of intermediate indicator targets have been met. The investment component of the project was successful. It supported the improvement of water supply services in small towns and rural areas, which are less affluent than the capital city Yerevan, and have higher rates of poverty. In total, about 130,000 people benefitted from the project’s intervention. On average, availability of water supply for these beneficiaries has improved from 12.3 to 17.6 hours/day, exceeding the set target (16.6 hours/day); non-revenue water has been reduced from 83.5% to 70%; and billing and collections increased from 93.5% to 98%. The project has certainly contributed to also improving AWSC’s efficiency as many of the intermediate results indicators show. Water production has decreased while service availability has increased. Energy consumption has decreased, metering has increased beyond set targets and water disinfection as well as the methodology for measuring it (sampling) have improved. The contribution of MWP to the higher goal of improving overall WSS sector sustainability is less substantial. Since the late 1990s the Government of Armenia has started a reform process which has led to improved service coverage and noticeable efficiency gains. However, this was not accompanied by a tariff reform toward full cost recovery. In the service area of AWSC, tariffs have remained stagnant since 2009, leaving the utility indebted and dependent on Government’s subsidies. Armenia is the only country in the Former Soviet Union (FSU) that has decided to open the WSS service sector to substantial private sector participation. Over the years, it has also gradually agglomerated service provision under a limited number of utilities. In order to improve WSS service financial sustainability in a rather complex politico-economic context, GoA has decided to move forward with a second wave of reforms and tender for a national lease, possibly with a single tariff for the entire country. GoA hopes this move would provide further efficiency gains and allow tariffs to be kept comparatively low. It is difficult to assess whether this step will yield the expected outcomes. In all cases, the design of the tariff should consider the optimal level of subsidy, and the affordability of such fiscal outlays, as well as the ability of the operator to generate sufficient revenues especially as the customer base may reduce due to out migration.

1

1. Project Context, Development Objectives and Design 1.1 Context at Appraisal 1.1.1 Country Background 1. Armenia is a small, landlocked country with a relatively limited economy based on small-scale agriculture and mining. Ethnically, religiously and linguistically distinct from its neighbors, Armenia has a history of conflict and tension with bordering countries, which further restricts its access to markets and opportunities to benefit from regional trade. After the severe economic recessions that followed independence from the Soviet Union, the country’s strong commitment to market reforms and stabilization policies ushered in 6% average annual growth in the second half of the 1990s. Despite such rapid growth, jobs creation lingered and efforts to expand export sectors and increase Foreign Direct Investment have been slow. The economy of the country remains heavily dependent on the remittances provided by a worldwide diaspora that is 8 million people strong, or nearly three times the 3 million nationals residing in Armenia. GDP contracted 14% as a result of the global financial crisis, and poverty rates increased by 4.5% in rural areas and 5% in urban areas between 2008 and 2012. More than one third of Armenians now live in poverty. The government response to this crisis has focused on safeguarding social programs, increasing public investment, short-term job creation and credit and guarantee facilities for private enterprises, resulting in slow but steady growth since 2010. 1.1.2 Sector Background 2. Armenia is a water-rich country with abundant, high-quality surface and groundwater resources to supply its three million inhabitants. On average, water resources total 10.2 billion cubic meters per year, of which only 840 million cubic meters are used for drinking water.1 Water delivery infrastructure was built over 30 years to account for large domestic consumption using Soviet design standards determining very high operation and maintenance (O&M) costs. A decade ago, the infrastructure under Armenia Water and Sewerage Company (AWSC)’s management was in great need of repair and rehabilitation, and most households and businesses received only a few hours of water per day, with leakages and loss of pressure (broken half-filled pipes) compromising water quality and safety.2 3. Local governments are responsible for water and wastewater service in their territories. The water sector (including water supply, wastewater and irrigation) is administered by the State Committee for Water Economy (SCWE) which was created in 2001, and since 2005 has been operating under the Ministry of Territorial Administration. The PPP model was first used for the Yerevan water utility under a Management Contract (2000-2005) which then became a Lease contract in 2006. The 2002 Water Code allows for various forms of private sector participation (PSP) ranging from service contracts to concessions. The sector is regulated by the Public Services Regulatory Committee (PRSC), which sets service standards and tariffs. The majority of Armenians have water supply services provisioned through one of 5 PPP arrangements. Services

1 2006. FAO Aquastat. http://www.fao.org/nr/water/aquastat/countries_regions/Profile_segments/ARM-WU_eng.stm 2 2014. AWSC/Saur. AWSC Total Management Plan. Page 29.

2

for the 1 million inhabitants of the capital city, Yerevan, are managed by the Yerevan Water and Sanitation Company (YWSC). Around 600,000 people in 37 secondary towns and 283 rural areas are managed by the Armenia Water and Sewerage Company (AWSC). The towns of Gyumri, Vanadzor and Armavir (total population about 0.32 million), have created their own utilities. While each of the 5 utilities benefits from some form of PSP, they all remain public, with assets held by SCWE. 1.1.3 Rationale for Bank Assistance 4. Over the course of the last 17 years, the Bank has taken a large role in driving the shift from a Soviet “social service” model to a modern approach that promotes efficiency and sustainability in service delivery. This has required a sea change in the thinking around water pricing, ownership and sovereignty. Through this period, the Bank has helped to lay the foundation for gradual private sector participation (PSP), helping to enhance government capacity to regulate and monitor before moving the sector up the PSP ladder. 5. The Municipal Water Project (MWP) supported Strategic Objective II of the Country Partnership Strategy (CPS) for Armenia (2009-2013) – Strengthening Competitiveness and Post-Crisis Growth. MWP is the capstone of a nearly two-decade Bank engagement in Armenia’s water sector (see Annex 10 for details). Its primary objective was to support the completion of important infrastructure investments as to improve access to water services in small towns and surrounding rural areas. At the time the Project Appraisal Document (PAD) for MWP was written, investment needs in AWSC’s service area for 2012-2016 (including networks as well as other assets) were about $100 million. The Bank’s $15 million contribution leveraged another $85 million from the European Bank for Reconstruction and Development (EBRD), KfW and the Asian Development Bank (ADB) to fill the gap (see Annex 11 for details). 6. Secondarily, the project has continued to support the modernization process of the water supply and sanitation (WSS) sector in Armenia by financing part of the Management Contract for AWSC. The project, unlike its predecessors, the Yerevan Water and Wastewater Project (YWWP) and the Municipal Water and Wastewater Project (MWWP), briefly described in Annex 10, did not provide specific technical assistance to the Government of Armenia (GoA) with the implementation of reforms in the sector. The actions that had to be taken to move the water sector towards sustainability had been already laid forward in a Policy Note issued by GoA in January 2012 and mostly pertained to strategic decisions the GoA had to take regarding tariffs adjustments and other efficacy gain measures, which went well beyond the scope of MWP. 1.2 Original Project Development Objectives (PDO) and Key Indicators (as approved) 7. In line with the Government of Armenia’s strategy for continual improvements in service delivery, the Project Development Objective (PDO) is “To support improvement of the quality and availability of water supply in selected service areas of the Armenian Water and Sewerage Company (AWSC).” The selected service areas are: Masis town, Ashtarak town, and Echmiatsin town, and their surroundings. 8. The Key Outcome Indicators are:

3

Increased weighted average daily supply of drinking water service in selected areas

from 12.3 hours daily to 16.6 hours daily Decreased weighted average non-revenue water in selected areas from 83.50 percent

to 70 percent Increased ratio of billing and collection from 93.0 percent to 98 percent

1.3 Revised PDO (as approved by original approving authority) and Key Indicators, and reasons/justification 9. The PDO was not revised. 1.4 Main Beneficiaries, 10. The project aimed to provide direct benefits to about 133,000 AWSC service customers in the three selected cities receiving project investments (Ashtrak, Masis and Echmiadzin) and neighboring rural settlements. Benefits are described as increased duration of daily service, improved water quality, decreased human vulnerability to risks of water pollution, and improved customer service. An estimated 20,600 customers were to benefit from installation of new meters. 11. The project’s design also anticipated “measurable gains in AWSC sustainability” (PAD, page 5) through improved performance, higher revenue generation, and lower costs. Capacity was to be enhanced through the administration, operations, maintenance and implementation of the investments. 1.5 Original Components (as approved) 12. Component 1: Investments in Water Supply Systems (IBRD: $8.6m; Borrower: $1.72m): Design, implementation and supervision of the rehabilitation of water systems in the cities of Masis, Echmiadzin, and Ashtarak, including their neighboring rural settlements. The work included rehabilitation of water networks linked to operational efficiency and reduction of non-revenue water. The goals of the investments were to finance physical infrastructure that is technically sound, cost effective, reliable, and that reduces technical losses. 13. Component 2: AWSC Investments (IBRD: $1.76m; Borrower: $0.36): This component financed: (i) water meter replacement and installation of meter chambers in several cities and villages throughout the AWSC Service Area to reduce commercial losses; (ii) automation of pumping stations and reservoirs; and (iii) procurement of equipment and machinery for the improvement of O&M system of the AWSC. 14. Component 3: AWSC Management Strengthening (IBRD: $4.6m; Borrower: $0.92): This component financed: (i) about two years of fees for an Enhanced Management Contract (EMC) to improve AWSC administration, operations, maintenance, and ensure optimal implementation of the program investment fund; and (ii) technical studies, project management and monitoring, including independent technical audits, financial audits and operating costs.

4

1.6 Revised Components 15. There are no revised components 1.7 Other significant changes 16. The project accumulated savings of US$2 million which was used to: (a) extend on-going rehabilitation works in water supply systems rehabilitation in Echmiatsin and Ashtarak (US$1.11 million under Component 1); (b) procure chlorination equipment (US$0.22 under Component 2); and (c) procure vehicles to improve operational efficiency of the utility by introducing a new approach in the methodology of water meter data collection processes which would in turn improve billing accuracy. The Bank’s economic analysis recommended that vehicle procurement (43 vehicles were proposed) be self-financed by the operator and that a pilot approach be used with a reduced number of vehicles to measure impact before going to scale. The pilot approach was taken and results are pending. 17. Moreover, seven additional (excluding the 28 originally planned) automation systems were installed in reservoirs and pumping stations following additional demand.

2. Key Factors Affecting Implementation and Outcomes 2.1 Project Preparation, Design and Quality at Entry 18. Leveraging Earlier Operations. The MWP is a natural continuation of the MWWP and its additional financing, so much so that according to the PAD, it is considered a “repeater operation” (pg. 4). This is further evidenced by the fact that the government requested retroactive financing of feasibility studies for some of the works under MWP (conducted during the MWWP). The government also requested that the Bank advance the appraisal process for faster approval, which it did. 19. Targeted Scope. The MWP, as a $15m investment mainly targeted to infrastructure rehabilitation, could not directly influence the reform agenda given its small size and narrow scope. For example, the institutional support to the utility (Component 3) was designed to cover only year 1 of the project. The Bank was ensured that second generation reforms would be implemented in parallel with the project implementation period, as stated in the policy letter issued just prior to project approval, in which the Deputy Prime Minister announced the government’s commitment to reforms, including tariff increases, to begin in 2014. 20. Leveraging the Enhanced Management Contract It is worth mentioning that three of the four performance indicators of the EMC were included in the project’s design, enabling the project to piggyback on the outcomes of the EMC, which were underpinned by strong monetary incentives. The project also benefited from the use of a mandatory annual audit of the EMC, which provides more assurance on the accuracy of data reported by AWSC. Moreover, budget support to the management contract provided the cash needed to incentivize good performance. 21. Leveraging Other Partner Financing The Bank’s $15 million contribution leveraged another $85 million from EBRD, KfW and ADB to fill the gap. Please see Annex 11 for a list of

5

donor projects active in the sector during the course of this project (including disbursements between 2012 and 2015). It was clear from discussions with these donors that they were initially attracted to investing in the sector in Armenia due to the groundwork that the Bank had undertaken to introduce various PPP models over 17 years. 22. Weak Financial Analysis. While the PAD referenced the WB Water Sector Note of 2011, it did not present an analysis of the current or projected financial position of AWSC. A section in the PAD would have better highlighted the risk, and accordingly set clear expectations regarding MWP’s [limited] role in the broader reform agenda. Already the MWWP ICR pointed to this weakness, quoting “Because neither the 2004 PAD nor the 2008 PP included a financial forecast of AWSC operations, AWSC bankruptcy, as a result of accumulated losses and/or reduction of GoA subsidies was not identified as a risk.” According to operational policy (OMS 2.20), this is standard practice for water and wastewater projects implemented by revenue earning utility companies. Having been flagged already in the MWWP ICR it should have been addressed in the MWP PAD. 23. Incorporation of Lessons from Earlier Operations. The MWP provided ample opportunity to build on past successes and learn from past failures, some of which were accounted for the project’s design and others that were not. Several lessons from MWWP and YWWP were incorporated into the design. These are:

Water loss3 should be a performance indicator Sectorization (zoning) of the network and installation of meters should be conducted to

improve operational and financial metrics Broken meters should be replaced, new meters should be installed at apartment block

buildings, and management of meters should be transferred to AWSC. Typical covenants for water and wastewater utilities request some level of cost recovery

(YWWP ICR). The project covenant on indebtedness (see footnote number 6) implies that the entity will cover sufficient costs so as not to incur new debt.

The Armenia Water Sector Note – ESW, 2011: Reform focus should shift to reduce NRW, strengthen institutional development and the financial position of AWSC (to cover O&M and some capital expenditure (CAPEX)), and rehabilitate poor quality infrastructure.

24. There were also other lessons from earlier operations that were not transferred into the design of the MWP, including:

The YWWP ICR stated that a lack of sector financing (either from GOA budget contributions or commercial debt in local currency) was a major risk factor at the time. The debt burden and financing needs of AWSC were even greater than those of YWSC, putting them in the same precarious situation, yet this was not factored into the sustainability of the project.

3 Water loss implies physical (technical) loss. It is an indicator in the EMC, and according to the independent auditor, is taken to mean non-revenue water (NRW), which includes both commercial and technical losses.

6

The MWWP ICR noted that a large portion of losses were from authorized, unbilled consumption (public taps; public commons of apartment buildings) that required changes in local condominium regulations and behavior change for improved demand management. It is estimated today that in AWSC’s service area more than 50% of NRW is due to such local regulations and customs.4 This is a regional challenge common across ECA countries, and requires a mental shift in management of common areas, which is beyond the scope of this project.

2.2 Implementation 25. Implementation Arrangements. The implementation arrangements were well designed and based on experiences working with the same institutions in Armenia since 1998. AWSC was the Project Implementing Entity (PIE), which leveraged the capacity of AWSC in implementing ADB and EBRD projects since 2007 and helped achieve economies of scale by managing all donor funded programs together. This arrangement also strengthens in-house capacity to prepare and implement investment projects and contributes to their sustainability. Key staff in the PIE include: Project Manager, WSS Engineer/ Design Works Specialist, WSS Engineer/Construction Supervision Specialist, WSS Engineer/Construction Supervision Specialist, Procurement Specialist, Financial/Disbursement Specialists, Accountant, and Environmental and Social Impact Specialist. SCWE retained the Contract Monitoring Unit (CMU). 26. The operator was responsible for, among other issues, preparation of the annual investment program, annual procurement plan, sub-project design, construction supervision and the certification of completed works. AWSC prepared composite quarterly project reports to the Bank to describe the physical progress in all service perimeters of the ASWC, as well as outputs and outcomes delivered by the program highlighting successes, constraints, and whether planned targets were achieved or otherwise. The aim of these reports was to monitor timelines and expenditures in the completion and delivery of scheduled activities and outputs. 27. Project Timeline. As of the writing of this ICR, all pipe replacement is complete and some individual connections to households were just recently finished. AWSC could not disconnect customers from the old pipeline during the very hot summer months of June, July and August, and is only now making the final connections to the new pipeline. Thus, water was still being delivered via old deteriorated pipes until mid-September. A baseline beneficiary survey was carried out before the project’s intervention. A follow up survey using a panel approach was planned after implementation. The follow up survey took place only in two areas of the three treated and the beneficiary feedback was collected less than six months from completion of the works. The responses show improvements in levels of satisfaction, but also remaining dissatisfaction, likely linked to the fact that some beneficiaries were only recently connected at the time of the second Survey. Reconnection has been a step by step process. 28. Parallel Efforts at Reform. During the course of project implementation, the Bank supported the sector through two separate operations:

4 Regional Workshop on NRW and Water Operator Partnerships. July 1, 2015. “Non-Revenue Water in Armenia”

7

1. A $75m Development Policy Operation signed on November 12, 2014, which was contingent upon the government’s adoption of an Action Plan on Drinking Water Sector Reform, and the tender of a transaction advisor for the national lease contract. The Action Plan was adopted through Government Decree No. N883-N dated August 14, 2014.

2. A Water Sector Tariff Study, financed by the Bank’s Public-Private Infrastructure Advisory Facility (PPIAF), which was published in 2014.

2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization 29. M&E Design and Implementation. The indicators selected to monitor progress toward the PDO on improved water availability and ‘quality of service’ were well designed with direct and indirect linkages with the EMC. Quality of service is measured directly by Key Indicator 1 (increased hours of supply); and indirectly by Key Indicator 3 (increased ratio of billing and collection). Key Indicator 3 assumes more water is being purchased, meaning that water is more available and/or customers are happier with the quality of service. Quality of the water itself, a component of the service, is also measured through the intermediate result indicator on percent of water disinfected.5 AWSC, through the life of the project, and based on recommendations made by the independent technical auditors, has improved its controls of water potability, focusing on the service areas where it is more likely (given infrastructure age) that water may be affected by secondary contamination. 30. Improved water availability is also measured directly by Key Indicator 1 (increased hours of supply); and indirectly by Key Indicator 2 (reduced non-revenue water), under the assumption that reductions in non-revenue water lead to more water available in the network. Several intermediate indicators also measure progress toward quality and quantity of services (Datasheet indicators 2, 5, and 10). 31. This project used well-tested monitoring and evaluation methods, including baseline and final surveys as well as regular monitoring of indicators well aligned with the Performance Indicators included in the EMC. Three of the four performance indicators of the EMC were included in the project’s design, enabling the project to piggyback on the outcomes of the EMC, which were underpinned by strong monetary incentives. The project also benefited from the use of a mandatory annual audit of the EMC, which provides more assurance on the accuracy of data reported by AWSC, including for areas outside the domain of this project. The audit also offered specific recommendations in the cases where targets were not met. These processes helped AWSC and GoA improve procedures and quality checks. 32. M&E Utilization. The collection methods used were adequate and leveraged the know-how of the Project Implementation Unit (PIU), which has significant experience with M&E on Bank and other donor-funded projects. Data was collected on time and submitted to the Bank on a quarterly basis. The EMC was appropriately used to reinforce data collection on key indicators to

5 The ICR team has assumed that the definition of “improve quality of water services” to mean service level improvements, not improvements in water quality itself. This is also supported by the lack of a key indicator on water quality.

8

be submitted (some monthly, some quarterly) to the AWSC Company Management Board, as results were tied to the contractor’s performance bonus. 33. The quality and reliability of data is reinforced by the annual independent audit, which included several other key performance indicators for the utility, and which continues to measure progress even after the project has closed. 2.4 Safeguard and Fiduciary Compliance 34. Safeguards. An environment category B project, compliance with safeguards requirements was consistently found to be satisfactory during the project. AWSC maintained adequate safeguards capacity with the mandate to provide environmental and social oversight of works under MWP throughout implementation. Technical supervision of works under the MWP, including environmental oversight, was provided by a German limited liability company (CES Consulting Engineers Salzgitter). The supervisor provided monthly reports from each of the active works sites. The quality of environmental supervision reports improved over the last year of project implementation, following recommendations from the Bank’s task team. 35. No resettlement or land acquisitions were envisioned under the Project and OP/BP 4.12 was not triggered. AWSC confirmed no incidences of temporary or permanent land acquisition or resettlement had taken place during Project implementation. Quarterly project progress reports by the AWSC included a write-up of safeguards supervision over the previous quarter, including supervision of land acquisition and resettlement. 36. With respect to beneficiary feedback and grievance redress mechanisms, the AWSC confirmed that beneficiaries and customers have multiple ways of providing feedback on the Project, including through AWSC service centers, a telephone hot-line and by email. 37. With respect to the project’s gender informedness, the data collected by the project was disaggregated by gender and the project beneficiary survey included attention to gender issues and vulnerable groups (including focus groups with women and youth, for example). 38. Procurement. Procurement administration under the project was generally found to be satisfactory. Fiduciary support was provided by qualified procurement and financial staff. The Procurement Specialist for the project had substantial experience with Bank’s procedures and regularly attended procurement trainings and workshops organized by the Bank. 39. Financial Management. The Financial Management (FM) arrangements at AWSC for the MWP, including budgeting and planning, accounting, financial reporting, funds flow, internal controls, external audits, and staffing were adequate and acceptable to the Bank, and were found satisfactory through the project.

9

40. The Loan Agreement included a Covenant6 to monitor the financial sustainability of AWSC. The covenant set a limit for the Project Implementing Entity’s (PIE, i.e. AWSC) indebtedness at 1.2 times the estimated maximum debt service requirements of the Implementing Entity for any succeeding fiscal year on all debt of the PIE, including the debt to be incurred. Exceeding such level should have happened only with the Bank’s prior approval. After the GoA fell into non-compliance (as noted in the 2012 management letter from the auditor), the Bank requested (letter of July 18, 2013) to meet with the GoA to “assess remedial measures”. GoA responded that to cover the short term indebtedness of AWSC it would provide subsidies, while for the longer term sustainability, it would move forward with the second wave of reforms, including a national lease and nation-wide tariff. The Bank acknowledged the Government's commitments and efforts toward sector sustainability, especially in a difficult economic period. The financial sustainability of AWSC is discussed further in the Efficiency (Section 3.3) as well as in Annex 3. The final FM rating is Moderately Satisfactory, reflecting the good performance of the project financial management while acknowledging that the financial situation of AWSC was still weak at the time of project closing. It is also understood that with a national lease AWSC will cease to exist in its current form. 2.5 Post-completion Operation/Next Phase 41. Five water companies currently serve 75% of the country’s population under PPP contracts. All of these contracts will end in 2016 and the government plans to tender a national lease. Thus, the lessons learned from the past 15 years of Bank support to PPPs in Armenia will culminate in a new lease contract, and AWSC will continue to benefit from private sector participation. Key to the success of the new lease will be commitment to a new generation of sector reforms, such as condominium legislation and tariff reform, which presents a potential opportunity to continue modernizing the sector, but also a potential risk to sustainability if the reforms are not implemented fully and quickly. 42. As of the writing of this ICR, several other donors, including EBRD and ADB, have entered the sector. The EMC is also now funded by a KfW grant under Amendment 5, which was signed on December 31, 2013 and is valid through 2016. Under this new contract, Saur has also taken over management of three regional utilities (known as CJSC service area) as a leader of a consortium with German MVV and AEG Services LLC. 43. Additional Bank support will likely come through technical assistance in the design of the new lease contract. The government plans to hold a limited competitive bid for a national lease. The Bank’s PPIAF trust fund is finalizing plans to send a PPP specialist for advisory support. 44. One tariff is being proposed for the entire country in an effort to bring the quality of services outside Yerevan to a higher standard while cross-subsidizing to enable the repayment of

6Unless net revenue for the preceding FY is at least 1.2 times the estimated maximum debt service requirement for any succeeding fiscal year on all debt of the entity including the debt to be incurred.– Section II.B.4 of Schedule 2 of Loan Agreement IBRD 8129). In 2012 net revenue was negative, thus the covenant is not met in 2013.

10

loans, several of which come due in 2016. The perception that block tariffs create a corruption risk has prevented a quality discussion from taking place on tariff setting. 45. Tariff adjustment to achieve cost recovery is being supported by the Development Policy Operation Series. The PPIAF-financed Armenia Water Sector Tariff Study published in spring 2015 provided recommendations in line with this objective, with an eye to preventing potential shocks for vulnerable populations. While water tariffs are some of the lowest in the region, 41% of people outside of Yerevan are willing to pay more than they do now for improvements. The PPIAF-funded proposal recommends a transitional tariff increase coupled with subsidies to target poor households, which could be administered through the existing Family Benefits Program.

3. Assessment of Outcomes 3.1 Relevance of Objectives, Design and Implementation 46. Objectives. Rating: High. The project’s objectives remain highly relevant at the time of the writing of this ICR. Improvement of services and modernization of the water sector in Armenia is and continues to be a process that requires significant and long-term engagement between donors, the public and private sectors. The project components remained relevant to the 2014-1017 CPS and in particular to Engagement Area 1.3: Improved access, quality, and sustainability of key infrastructures. The CPS objective is to enhance the effectiveness and impact of infrastructure. WBG work on improving transport, energy and water infrastructure is to support effective targeting of the benefits to the bottom 40 percent of the population. The project also remained aligned with the Armenia Development Strategy 2025, which envisages enhancement of human capital through better access to quality services, including healthcare, education, culture, and basic infrastructure. 47. Design and Implementation. Rating: Substantial. The project’s components were adequately designed to achieve the PDO. Components 1 and 2 contributed directly to improved services by way of new infrastructure, better metering, and improved O&M. Component 3 had both a direct impact on the PDO for the entire service area of AWSC (including the project area) by supporting the management contract; as well as an indirect impact on services in the project area by way of improved company administration and operation. 48. Component 1 also aimed to reduce non-revenue water (a key indicator). Although not fully documenting it with an economic and financial analysis on its impacts on availability or quality of services in the AWSC’s service area, the PAD recognized that very high levels of NRW could substantially undermine other efficiency gains achieved by AWSC. A problem also flagged in the MWWP’s ICR, which stated that NRW was a major concern (Armenia having one of the highest NRW levels in the world) that should be addressed in subsequent projects. The MWP results framework proactively included an NRW indicator and target but not a clear justification of why the target was selected as optimal (from a cost/benefit perspective). Despite not focusing project investments specifically on NRW reduction, results on NRW reduction have been remarkable, with NRW reduced by more than 10% (a substantial reduction) through activities aimed to reduce

11

commercial losses, 7 and the target is expected to be achieved eventually as a result of the investments made under this project. 49. The project was also well designed in terms of its focus on bringing improved services to the poor. The three towns selected for infrastructure improvements are a mixture of urban and rural areas outside of Yerevan. On average, poor urban areas outside Yerevan spend the highest percentage of their income on WSS services. It can be extrapolated from the final beneficiary surveys in Masis and Astharak, that 60% of the (sample) population is under the poverty line, and one-fifth of the population is considered extremely poor8. 50. The MWP did not include long-term institutional strengthening, which was taken up by other donors. KfW took over financing of the management contract after year 1 of the MWP, and EBRD supported the transaction advisor role for the planning of the national lease contract. The lack of Bank support to the broader institutional reform agenda had two consequences. On one hand, there was no mechanism to mitigate the risk of financial sustainability of the operator. On the other hand, the project objectives, which had a narrow focus on infrastructure and service improvements, were met within the initial 3-year time frame. 51. The project’s successful implementation depended on timely infrastructure development and system component replacement over a very short three year period. All three key indicators depended highly on the successful rehabilitation of more than 200 kilometers of distribution network and installation of more than 9,000 individual meters across three towns, in addition to continual implementation of the terms of the EMC by the operator. 52. The project was disbursed on time and delivered within budget and the team gave sufficient consideration to problems as they arose, including flagging the financial sustainability concern in supervision missions and requesting expert advice. The majority of issues raised during supervision missions were addressed quickly and adequately, including:

Improvements in gender disaggregated data Customer feedback mechanisms/surveys used to monitor impacts Environmental monitoring presence bolstered at project sites Change in procurement plan for vehicle purchases

3.2 Achievement of Project Development Objectives Rating: Substantial

7 Monetary losses from illegal connections, under metering, faulty meter readings, etc. (pg. 28 of 2012 independent audit). The technical audits of 2012 and 2013 confirmed that nearly all reductions in NRW up to that point could be attributed to reductions in commercial losses.

8 Page 37. Survey of Project Beneficiaries in Masis, Ashtarak and Echmiatsin Subprojects. Final Report (second stage survey) AVAG Solutions, LTD. June, 2015.

12

53. The project has undoubtedly supported the improvement of the quality and availability of water supply services in the three towns. 54. All three key indicators for the project have been met9.As a result of the project, there was an increase in the weighted average daily supply of drinking water from 12.3 to 17.6 hours per day (key indicator 1); a decrease in the weighted average non-revenue water from 83.5% to 70% (key indicator 2); and an increase in the ratio of billing and collection from 93.5% to 98% (key indicator 3.) 55. Eight out of the 10 intermediate results targets have been met. Combined, these achievements demonstrate improved quality and quantity of service and point to improved utility performance via efficiency gains: reduced electricity consumption, fewer proportion of subscribers with arrears greater than 4 months; more water disinfected; more domestic water metering; an increase in average metered consumption; and more than 131,000 people provided with access to improved water services. 56. Regarding the unmet indicator targets (intermediate indicators 9 and 10), the number of people with connections in rural areas was 2,239 people less than the target of 33,328; and 761 fewer households were connected than the target of 33,609. The inability to meet the target is, according to AWSC, due to out migration from rural into urban areas over the course of the three year project. The target was also designed using informal data collected from mayors in 2011, rather than from the national statistical service, and thus may have been somewhat inaccurate. 57. AWSC has achieved improved operational efficiency in the areas targeted by the project. NRW reduction results are substantial, and have been achieved through the following interventions:

AWSC has managed to cut production from 752 l/c/d (liters per capita per day) to 481 l/c/d (surpassing the target of 489) while providing more hours of service (17.4 hours per day up from 12.3 hours per day) to the same customer base. This infers a significant reduction in NRW.

In a similar vein, increased average domestic metered consumption and increased domestic water metering with the same customer base and reduced production point to water being saved in the network. The 2012 audit confirms that all loss reductions are commercial.

The rehabilitation of degraded and inefficient pipes reduces physical leakages; illegal connections have been identified and removed during the course of the project which will further reduce commercial losses.

Commercial losses are being reduced through a variety of methods and will help AWSC better understand customer demand and locate service problems, including leakages.

Investments in sectorization in the three towns and improved metering lay the groundwork for improved leak detection and a full-scale NRW reduction plan that tackles technical losses.

9 Final numbers are as of September 30, 2015. Due to weather constraints impeding construction in the winter and reconnection in the summer, final data for key indicators 2 and 3 was not available until all works were completed September 30, 2015, which was 3 months after the closing of the project.

13

58. Improvements in billing and collections prove that commercial programs to improve payment are working. AWSC continues to use and publicize a very popular lottery for which only paying customers can purchase a ticket (the lottery is said to have increased the number of paying customers from 30,000 to 200,000). AWSC also continues to provide several payment options for easy access, including payment via SMS, at post offices, banks, or via the company’s website. 59. AWSC also has a program to reduce illegal connections by looking for customers who purchase energy yet are not a water customer; new software to analyze consumption patterns; verification through SMS phone images and staff using GPS; and the installation of thousands of new, more accurate meters (the former Class B meters did not register the first 50 liters of water consumed; new meters register after 5 liters consumed). All of this has led to an increase in average domestic metered consumption from 88.5 l/c/d to 119 l/c/d (surpassing the target of 116 l/c/d). Thus, both billing and collections are increasing simultaneously.

60. The results of the end surveys indicate overall customer satisfaction with the project. Nearly 66.7% of households mentioned that some positive changes in their water supply occurred between 2014 and 2015. The proportion of households that are dissatisfied with the short duration of water supply decreased 1.7 times after project completion, and those dissatisfied with low pressure decreased 2.4 times after project completion. Twenty percent of customers surveyed in Masis after project implementation reported being fully or partially satisfied with the operation of the centralized water supply system, a 47% increase from before the project. In Ashtarak, 70.5% were fully or partially satisfied, a 38% increase from before the project. 3.3 Efficiency Rating: Modest 61. The physical investment implemented under the project have certainly contributed to improved revenues and reduced costs for AWSC. Indicators consistently show improved service efficiency (reduced energy consumption and water production) and improved revenues in the project area. The project itself was implemented efficiently, within the allocated timeframe, three years, and with budget saving, which were invested in further infrastructure and equipment. The financial sustainability of AWSC, however, did not improve, on the contrary, the annual financial results are getting more negative each year and the subsidies for the company are increasing (AWSC tariffs have not changed since 2009). 62. The fact that AWSC would likely soon cease to exist, replaced by a new company operating under a countrywide lease arrangement, make future assumptions and calculations of an estimated economic rate of return (ERR) and financial internal rate of return (FIRR) rather arbitrary and speculative (see further details in Annex 3). Nevertheless, in order to provide an analysis of the impacts of the investments, an ERR has been calculated using similar assumptions as per the PAD. 63. The ERR calculated after implementation is 25.5% and the net present value (NPV) is AMD 4,269 million (USD 9 million at current exchange rate), assuming a discount rate of 10% and 30 years of asset life, with corresponding benefits to be realized starting 2016, substantially in line with PAD projections. As in the PAD, this calculation was done only for component 1.

14

Components 2 and 3 may add further to the project’s NPV and ERR but the team has limited data and there are too many additional investments into the company leading to results, which cannot be associated entirely with the project (see Annex 3, table 1). The project has also low per capita investment cost of AMD 39,120 (USD 94 at current exchange rate) per capita. This is good given the population density of the project areas. As a result, the economic benefits from the project Component 1 investments are considered to be satisfactory with respect to its economic efficiency, but the fact that overall the AWSC financial situation is extremely unsustainable despite the project financing of the management contract leads overall modest results. 64. Economic estimate of project benefits. The ex-post economic analysis includes: (a) the cost of all project components, including capital and operating costs; and (b) all measurable economic benefits, per the PAD. These are mainly: (1) time saved in fetching water; (2) cost savings on non-incremental water consumption; and (3) the value of increased water consumption. As with all economic analyses, the costs are perfectly observed while the benefits are not. Lacking proper data the team had to use the PAD's assumptions for annual cost savings on non-incremental water and time saved10 (see PAD, page 47) as well as the uncertainties about projections (see above). 65. Benefits are obtained from the reduced time to collect water: the PAD indicator envisaged increased supply from baseline of 12.3 hours (weighted average) to 16.6 hours at the end of the project. The company is currently reporting 17.4 hours, which means that the end target is exceeded. In monetized terms this is additional revenue of AMD 64 million per year for the project area (at the current tariffs). There is also a benefit from decreased electricity consumption: PAD baseline indicator is 0.23 KW/m3 and the end-project target was set at 0.17 KW/m3. The company is currently reporting 0.17 KW/m3, which means that the end target is met. In monetized terms, this means significant cost savings. Despite electricity price increase over MWP implementation period, the company contained its electricity costs by reducing consumption. It should be mentioned though that the overall operation and maintenance costs in the project area increased by AMD 13.7 million between 2012 and 2014.. It should be mentioned though that the overall operation and maintenance costs in the project area increased by AMD 13.7 million between 2012 and 2014. Another benefit is increased metered consumption: the baseline of this indicator is 88.5 l/c/d and the end-project target was set at 116 l/c/d percent. The company is reporting 119 l/c/d, which exceeds the end target. This indicator reveals avoided coping costs. The team used the PAD’s projections to monetize the savings. Additional information and calculations are presented in Annex 3. 66. Financial estimate of project benefits. The main challenge of estimating of the financial benefits of the project is the going concern of AWSC: the going concern accounting principle is the assumption that an entity will remain in business for the foreseeable future. An entity is assumed to be a going concern in the absence of significant information to the contrary (for more information see Annex 3). In the latest audited financial statements of the company (FS 2014) note 3.2 states that "in accordance with decree No1366-A dated 05.12.2013, the Government of the

10 1 hour less than 24 hour daily supply of water corresponds to 5 minutes time spent fetching water.

15

Republic of Armenia approved the approach to transfer water system operating rights from "Armenian Water and Sewerage" CJSC to a lessee on tender basis. However, the implementation of the decree was postponed, and it was decided to extend the contract with "Saur" SA organization by prolonging the functions of the executive body in the company till the end of May 2016." With this in mind, it does not make much sense to project future financial benefits from the project using the company’s future revenue stream. Nevertheless, an analysis in Annex 3 shows that AWSC’s financial situation is extremely unsustainable. 3.4 Justification of Overall Outcome Rating Final Rating: Moderately Satisfactory 67. The PDO remains highly relevant to the current priorities of the Government of Armenia. It is also well aligned with the World Bank’s strategy for Armenia. However, the project did not fully take into account lessons from previous experiences, and there were minor shortcomings in project design. Thus, relevance of objectives is rated high and relevance of design is rated substantial. In terms of achievement of objectives, the improvement in quality and availability of water supply is rated substantial. The overall efficiency is rated as modest. Benefits include time savings, reduced energy consumption and increased metered water consumption (and therefore increased revenue). 3.5 Overarching Themes, Other Outcomes and Impacts

(a) Poverty Impacts, Gender Aspects, and Social Development 68. Fifty-three percent of project beneficiaries were women, and an estimated 60 percent of beneficiaries were poor.11 The project targeted communities with a significant poor population that were also receiving few hours of supply per day, and who were located in the poorer urban and rural areas of Armenia, outside of Yerevan. 69. Poverty and gender impacts were assessed through the first and second stage beneficiary surveys in Masis and Ashtarak subproject areas, including a study of water consumption by wealth quintile, and a series of Focus Group Discussions to identify perceptions of various groups including women, youth and the poor. These assessments aided in understanding the changes as a result of project implementation (gender) and provided a baseline for future interventions (poverty). Further information is provided in Annex 5. 70. The study by wealth quintile showed that the richest population consumes from the AWSC network 1.85 times as much as the poorest quintile. As a share of their income, the highest wealth quintile pays 5 times less than the poor in Masis and 3.8 times less than the poor in Ashtarak. The conclusion that access to drinking water for the poor is more costly and thus reduces demand is key information in the design of the future water tariff, which should include sufficient subsidies for the poor.

11 Estimated from sample of Masis and Ashtarak subproject area end surveys, using monthly per capita expenditures as a proxy for poverty rates (page. 37).

16

71. Gender disaggregated data and Focus Group Discussions show that in general the living conditions of working age women are mostly influenced by the current condition and peculiarities of water supply. A large portion of households surveyed (61.9% average of both towns) think that the actual changes in the water supply which happened after the Project implementation have had a factual influence (significant influence and influence at some extent) on living conditions of their working age female members, 56.5% - working age male members, 28.9% - female children, 33% - male children, 20% - female elders, 13% - male elders. 72. Moreover, the surveys asked women, the poor, youth, and mixed groups about the key problems in the AWSC system, which will allow AWSC to better understand its customer base.

(b) Institutional Change/Strengthening (particularly with reference to impacts on longer-term capacity and institutional development)

73. AWSC has excelled at improving performance even in this constrained environment and since 2005 has been on a positive trajectory toward continual performance improvements.12 Over the course of the MWP, AWSC has made strides toward more sustainable operations by building staff technical skills, bringing in international best practices, and devising practical solutions to cut costs and increase revenues in the absence of much needed second stage sector reforms. Bringing the PIU into AWSC has improved the capacity of those staff, who interface daily with management, which increases opportunities for learning. 74. Under Component 2, the utility automated the pumping station and reservoir in Masis Town, cutting costs in terms of staffing the facilities, and enabling staff to develop more modern operating skills. The procurement of vehicles for staff improved their capacity to respond to complaints and conduct other oversight activities. 75. Customer management and relations have dramatically changed for the better. The establishment of a Customer Service Center and the centralization of billing in the headquarters of AWSC in 2012 have helped build a modern platform for customer management, in which customer feedback mechanisms and responsiveness are institutionalized in the business model. Despite the direct linkage between revenue generation and profit (given the flat tariff) AWSC has a strong customer-service oriented identity that promotes efficiency and quality services. 76. Under Component 3, the Enhanced Management Contract has led to several institutional improvements. Before the management contract began, 75% of AWSC’s staff didn’t speak English. Today, English is a priority and staff are encouraged to take language courses. Staff capacity on tendering and managing sub-contracts (such as for facility maintenance and accounting services), engineering design (from the International Federation of Consulting Engineers), environmental and social safeguards, safety measures and information technology proficiency training have also been enabled under the EMC. The development of a Total Management Plan (TMP) also

12 Nov 2012 Aid Memoire. Annex I shows continual improvement.

17

engendered a new skill set within the company and a more business-oriented focus. The TMP was thus critical for professionalizing operations and ensuring sufficient capacity to undertake a lease contract. The document is also a tool for AWSC and GoA to identify and plan for future investments, as well as to compare the affordability of tariffs with the investment requirements13. 77. For a former-Soviet country, the shift from a socialist view of public goods to a one where the private sector can make a positive contributions to service delivery improvements, is extraordinary from a psychological and political viewpoint. A lot has been achieved considering the political economy and the specific challenges posed by the influence of the Soviet system. 78. Further actions can be taken to improve performance, as detailed in the EMC and the technical audit of 2013, including: Use of the TMP as a long-term investment planning tool; improvements in leak detection; and increasing water supply in the areas with the fewest hours per day. Once enacted, these will further improve service performance.

(c) Other Unintended Outcomes and Impacts (positive or negative) 79. Exogenous factors related to political economy and sector reform have had various types of impacts on the outcome of the project. 80. First, legal reforms to allow meter installation by AWSC enabled the company to start billing based on actual use rather than piped diameter. Similarly, new laws enabled AWSC to disconnect non-paying customers, providing an incentive for customers to pay their bills to the utility. However, the reforms stopped short of changing condominium laws or meter ownership laws, which prevents AWSC from having full control in installing, maintaining and reading meters. Nevertheless, AWSC has modeled best practice approaches from Europe in auditing meter reading by taking pictures with smart phones and uploading them to a central database to verify accuracy of the self-reported data. Overall, AWSC has seen a dramatic improvement in its level of autonomy as a result of these and other, broader, legal reforms regarding PSP, from decision making on staffing to more direct customer management. 81. Second, the lack of an increase in the tariff since 2009 has led to increasing financial losses for AWSC. Likewise, prices are volumetric and are the same for residential and commercial customers, and there are no plans to introduce differentiated pricing. 82. As a result of the sustained commitment from the Bank over the last 15 years, several donors have entered the water sector in Armenia. MWP alone leveraged $85 million in donor investment from ADB, EBRD and KfW. With the Bank’s retreat from the sector, these donors continue to fund much needed infrastructure investments and support to the management contract. 83. Finally, the project’s completion impacts the decision to move to a lease contract. The decision to contract one operator for the entire country comes from a desire to improve the quality

13 Various options for levels of investment and their corresponding levels of tariff increases are presented in the TMP, as noted in the Borrower’s ICR

18

of services outside of Yerevan and to cross-subsidize from the capital to the poorer urban and rural areas. Under this new structure, the operator would be able to repay IFI loans coming due starting in 2016. Assuming tariffs are increased to the optimal level, this arrangement has the potential to bring more and better services to disadvantaged areas, including the poor. 3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops 84. In Masis, 220 households were included in the baseline survey, 93% of which were re-surveyed on the second round. In Ashtarak, 211 households were included in the baseline, 96% of which were re-surveyed on the second round. Unfortunately the survey took place very soon after works were completed and thus cannot reflect the full benefits of the project. At least six months should have passed from completion of work to obtain a more objective assessment of the beneficiaries’’ opinion. . The main findings include: Averaging the results of the two sub-project areas, 45% of customers surveyed after the

Project’s implementation reported being fully or partially satisfied with the operation of the centralized water supply system. Before the Project implementation this figure comprised 32%. Thus, the share of satisfied customer households increased by 43% on average (66% in urban areas and 27% in rural areas). Some household still reported being dissatisfied, likely due to the delayed connections, which are being done through a step by step process.

Likewise, the proportion of households that are dissatisfied with the short duration of water supply decreased 1.7 times after Project completion, and those dissatisfied with low pressure decreased 2.8 times after Project completion. Low pressure is a bigger problem in hot summer months than in the winter.

Average hours of supply per day increased to 16.7 hours from 12.7 hours on average across the project area; however, for Masis town the increase was larger for rural areas, while for Ashtarak the increase was larger for urban areas.

Households pay 10% more water now than they did before the project, given a 9.5% increase in daily per capita consumption.

Water quality remains a concern even after the project; 52.4% of customers surveyed are still dissatisfied with water quality, down from 63.3% before the project. Moreover, 30.8% of households still consume other sources of water for drinking, washing, cooking etc., a 27% decrease from 2014.

However, it is telling that there was also a 20% decrease in the proportion of households treating their water (boiling, filtering, adding chlorine, settling, or other) after the project, which infers an improved perception of the water quality.

4. Assessment of Risk to Development Outcome Rating: High 85. The successes of MWP and predecessor projects has fomented a strong commitment to PPPs in the sector. The efficiency gains made and the increased revenues of AWSC yield a low risk that the economic and political winds will change in the future. It is very likely that innovations in service delivery under the new lessee contract will continue to build stability in the sector.

19

86. Historically, PPPs are successful when risk is transferred slowly to the private sector and managerial responsibilities are transferred gradually to local staff. While this was the case with Yerevan Jur, which is now run by Armenian nationals,14 AWSC is comparatively much farther behind with day to day operations highly dependent on ex-patriot staff and very little risk transferred to the operator. The EMC is still being paid for by donor contributions, and gaps in the budget are filled through GoA’s unconditional subsidies. A jump from this volatile financial situation to a national lease is likely too large of a step given the size and complexity of AWSC’s service area and investment requirements vis-a-vis those of Yerevan. 87. The planned tariff adjustments may also bring significant social risk, as was seen earlier in 2015 with the intention to raise energy tariffs. A communication strategy will also be required to prevent “rate shock” and unrest. This risk has been known for many years and mitigation plans are well document in the 2011 Water Sector Note as well as the 2014 Water Sector Tariff Study. 88. Private companies are interested in bidding on the national lease contract, but the competition will be limited to the four companies that have already worked in Armenia’s water sector. Thus there is some risk that a suitable bid will not be proposed, and is highly dependent on the commitment to tariff reform, including the transparency of the regulator in approving changes, and the expectations of the lessee with regard to what costs they must cover. If bidding is limited, attention should be given to promoting transparency and fairness and adhering to international best practices.

5. Assessment of Bank and Borrower Performance 5.1 Bank Performance (a) Bank Performance in Ensuring Quality at Entry Rating: Moderately Satisfactory 89. The project was a repeater operation of MWWP, aimed to complete urgent infrastructure rehabilitation in underserved communities. The project’s design built on past successes, incorporated many (but not all) lessons learned, and had adequate indicators and monitoring mechanisms that leveraged those in the management contract. Regarding the financial position of AWSC, the PAD well referenced the Sector Note prepared by World Bank in 2011, which discussed the financial situation of AWSC and possible scenarios to improve its viability. A more robust analysis of the government’s constraints /ability to pursue tariff increases and the risks associated could have been done and included in the PAD. 90. The path from a highly centralized social service-like approach, typical of the FSU, to a more modern, efficient and sustainable model is long and fraught of complexities. Armenia is performing fairly well when compared with other neighboring countries, which twenty years ago started from very similar circumstances. The Bank is to be commended for its perseverance and support to the Government of Armenia. The ICR team feels that such support has been very

14 According to the YWWP ICR, “it is usually admitted that the transfer of know-how has already taken place”. Page 18.

20

valuable in the improvement of WSS services in Armenia. The financial sustainability of the sector has not yet been achieved, but the transformation process has started. An increase in tariffs, which is currently deemed necessary to move the sector towards cost recovery, has been lagging due to economic and political considerations. In the preparation documents of MWP, the Bank could have been more candid about these objective constraints. The Government’s ability to act on tariff increases should have been analyzed in light of the macro economic conditions of the country and expectations on the reforms that would improve financial sustainability of the sector should have been adjusted accordingly. 91. After 15 years of engagement, the Bank made a cautious move in limiting the investment period and loan amount of this project to give time to get the sector on a more sustainable footing without making further major commitments. Part of the reason for the shift away from investment loans was due to Armenia reaching its borrowing limit, and the CMU’s call for fewer, larger projects to generate efficiencies. The Bank’s continued engagement did, however, attract an additional $85 million in financing from other donors who remain active in the sector. (b) Quality of Supervision Rating: Moderately Satisfactory 92. The Bank provided substantial guidance, feedback and support to the client throughout implementation. The Bank team comprised the skills necessary to monitor all aspects of the project. Supervision missions were conducted on a regular basis with participation of technical, engineering, procurement, financial management and safeguards specialists. Having a TTL in the field helped with regular follow up and addressing issues as they arose. The team was critical for catalyzing additional trust funds to supplement support on the tariff reform agenda. The PPIAF Tariff Study was published in 2014 in time to inform the lease strategy. 93. The financial unsustainability of AWSC was raised repeatedly in Bank reporting and communication to the borrower, as it was also linked to a financial covenant (see paragraph 40). The 2014 decision by GoA to move to a national lease, shifted the focus from moving tariffs toward cost recovery levels to an even broader discussion on institutional reform and financial sustainability through homogenized services for the entire country, with the aim of improving efficiencies while also promoting equity in service delivery. In the meantime, GoA committed to cover the revenue gap for AWSC. The Bank considered this approach a plausible medium term solution, more socially acceptable, under the circumstances, than a sudden tariff increase. Hence the Bank decided not to act (suspend disbursement) on the noncompliance with the covenant regarding AWSC’s indebtedness; a suspension was not deemed justified nor a reasonable option on such otherwise well performing project with a concerned client intent on sector reform. 94. Finally, considered the higher level decisions made and the overall fiscal situation in Armenia, during implementation of the MWP, the Bank decided to shift its support to the WSS sector from specific investments to a reform focused approach, supported by a specific trigger for

21

the Second Development Policy Operation (DPO) and targeted technical assistance such as the PPIAF tariff study to help the client prepare to implement its reform agenda15. (c) Justification of Rating for Overall Bank Performance Rating: Moderately Satisfactory 95. Due to minor shortcomings in the design, overall Bank performance is rated as Moderately Satisfactory. The Bank’s supervision of the project was adequate and as concerns arose over the sustainability of AWSC supplemental support outside the project was provided to ensure the best policies would be adopted in the long-run. The Bank continued to receive assurances from the GoA that needed reforms were afoot, steered by the design of the national lease contract, justifying its decision not to enforce the loan covenant. 5.2 Borrower Performance (a) Government Performance Rating: Moderately Satisfactory 96. The Government of Armenia (GoA) has been a pioneer in water sector reform in the region. The Lease of Yerevan and the MC/EMC of AWSC are the only examples of substantial private sector engagement in the former Soviet Union countries. Substantial steps have been taken through the years to implement reforms that would improve service levels, efficiency and also allow private participation. For example, legal reforms to allow meter installation by AWSC enabled the company to start billing based on actual use rather than piped diameter. Similarly, new laws enabled AWSC to disconnect non-paying customers, providing an incentive for customers to pay their bills to the utility. 97. Before and during the implementation of this project, GoA had committed to take steps to limit AWSC debt and improve sustainability, see paragraph 40 on the financial covenant. On the tariff increase side, however, a slow recovery from the economic crisis and unemployment remaining above 16% capped the willingness and ability of the GoA to act decisively on tariffs. This, in turn, has limited the progress of AWSC towards more sustainable and self-sufficient operations. Moving forward, even with a national lease contract, the GoA would accelerate service improvements by acting on the following aspects: i) a substantial tariff reform, which is to take into account the evidence provided by the existing analyses on affordability as well as recommendations to decouple tariffs from social assistance programs; ii) discontinuing the practice of providing subsidies to the operator upon request (usually through annual budgeting process); iii) limiting borrowing concessional loans to replace aging infrastructure without a plan for repayment; and iv) ensuring independent and merit based human resource decisions of the utility.

15 As a prior action for DPO-2, the Borrower has: (i) adopted an Action Plan on Drinking Water Sector Reforms, including the decision to create a new, nation-wide lease contract through the approval of Government Decree No. N883-N dated August 14, 2014, and (ii) has started the Action Plan’s implementation by issuing the tender for the transactions advisory for a new lease contract, as evidenced by the announcement published on: www.scws.am, ww.procurement.am, and in the government’s official newsletter.

22

(b) Implementing Agency or Agencies Performance Rating: Moderately Satisfactory 98. AWSC delivered quality construction in a timely fashion and within budget, while meeting fiduciary and safeguard requirements. It was, however, very dependent on the government for operating subsidies. There is a semiannual IFR requirement under the project. The International financial Reporting Standards (IFRs) have always been received on time and were mostly acceptable to the Bank. IFR for S2, 2014 included inaccurate posting of certain actual expenditures to improper cost lines. The issue was corrected and the IFR re-submitted. The only lapse in on-time reporting has occurred with the latest (2014) technical audit, which is still not completed. The PIU had sufficient capacity to carry out the project on time and within budget. They did an excellent job of coordinating donor programs and leveraging their M&E expertise to report efficiently and effectively on results. 99. Managing the PIU within AWSC has drastically improved the implementation of all donor-funded projects due to easier communication between the parties. This shortened the time for decision making as there was not a third party through which to communicate. The utility integrated PIU is much more efficient, professional and less bureaucratic than the external PIU. (c) Justification of Rating for Overall Borrower Performance Rating: Moderately Satisfactory 100. Although the implementation of the project went well and all investments were completed on time, lack of timely reforms in the sector have continued to hamper the overall sustainability of AWSC.

6. Lessons Learned

101. Armenia is an absolute pioneer in using extensive PPP for providing water supply and waste water services, in the former Soviet Union. No other country in the region has utilized this instrument to this extent. Most countries have a fully public supply system or outsource only small parts of the service, such as maintenance or collection of payments (very popular payment kiosks). The following two lessons can be learned from the Armenia’s experience: 102. Strengthen enabling environment to encourage PSP and service sustainability. In all FSU countries, the consumer’s price of water supply and sanitation services remains very low. Most utilities, whether public or private, struggle to achieve financial sustainability because the tariffs are far too low to cover operating costs. In most cases tariffs are decided at the local council level and are hence affected by politics. In general, the regulators should be strengthened to provide guidance not only on maximum affordability level for the consumers but to also on the minimum requirements for a utility to cover its operations and maintenance costs. 103. Designing the PPP and setting the right incentives for risk transfer. PPPs come in various shapes and sizes and details matter. Under PPP models, risk should be transferred to the private

23

sector at a level commensurate with the scale of private participation. PPPs should not be assessed, in general, as a good or bad model for a specific country as any recommendation would depend on the specific needs not only of the sector, but of the service area. In terms of this project, one must differentiate between the experience of Yerevan Jur and of AWSC in implementing PPPs in Armenia, especially given the disparities in asset base, customer base and O&M requirements. To ensure sustainability of service, the provider’s solution, should be accompanied by a tariff reform that allows progress towards cost recovery. In the case of AWSC financial sustainability significantly lagged during