new world bank document · 2016. 7. 8. · 0 6,240 7,500 7,566 date achieved 07/31/2008 03/31/2011...

Document of

The World Bank

Report No: ICR00003440

IMPLEMENTATION COMPLETION AND RESULTS REPORT

(IBRD-75040, IBRD-76640, IDA-43840)

ON A

LOAN

IN THE AMOUNT OF US$52.68 MILLION

AND

AN ADDITIONAL FINANCING LOAN

IN THE AMOUNT OF US$115 MILLION

AND

A CREDIT

IN THE AMOUNT OF SDR78.6 MILLION

(US$125.0 MILLION EQUIVALENT)

TO THE

REPUBLIC OF INDONESIA

FOR THE

NATIONAL PROGRAM FOR COMMUNITY EMPOWERMENT

IN URBAN AREAS

June 19, 2015

Global Practice for Social, Urban, Rural and Resilience

Urban and Disaster Reduction Management Unit

East Asia and Pacific Region

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

3

PMU Project Management Unit

PNPM Program Nasional Pemberdayaan Masyarakat (National Community Empowerment

Program)

PROPENAS Program Pembangunan Nasional (National Development Plan)

RLF Revolving Loan Fund

TA Technical Assistance

UPP Urban Poverty Project

VLD Voluntary Land Donation

Vice President: Mr. Axel van Trotsenburg

Country Director: Mr. Rodrigo A. Chaves

Sector Manager: Mr. Abhas Jha

Project Team Leader: Mr. George Soraya

ICR Team Leader: Ms. Evi Hermirasari

2

CURRENCY EQUIVALENTS

(Exchange Rate Effective December 31, 2014)

Currency Unit = Rupiah

IDR 1.00 = US$ 0.000081

US$ 1.00 = IDR 12,407

FISCAL YEAR

January 1 – December 31

ABBREVIATIONS AND ACRONYMS

ADB Asian Development Bank

AF Additional Financing

BAPPENAS Badan Perencanaan Pembangunan Nasional (National Planning Board)

BKM Badan Keswadayaan Masyarakat (Board of Community Trustees)

CAS Country Assistance Strategy

CDD Community-Driven Development

CDP Community Development Plan

CPS Country Partnership Strategy

FM Financial Management

FY Financial Year

GoI Government of Indonesia

IBRD International Bank for Reconstruction and Development

ICR Implementation Completion and Results Report

ICT Information and Communication Technology

IDA International Development Association

IDB Islamic Development Bank

IDR Indonesian Rupiah

ISR Implementation Status Report

KDP Kecamatan Development Project

Kel. Kelurahan (Ward)

Kec. Kecamatan (Sub-district)

Kotamadya Municipality

KSM Kelompok Swadaya Masyarakat (community group)

LAR Loan at Risk

M&E Monitoring and Evaluation

MIS Management Information System

MPW Ministry of Public Works

ND Neighborhood Development

NGO Non-Governmental Organization

NMC National Management Consultant

OC Oversight Consultant

O&M Operations and Maintenance

PAD Project Appraisal Document

PAPG Poverty Alleviation Partnership Grant

PDO Project Development Objective

4

INDONESIA

NATIONAL PROGRAM FOR COMMUNITY EMPOWERMENT

IN URBAN AREAS

CONTENTS

Data Sheet

A. Basic Information

B. Key Dates

C. Ratings Summary

D. Sector and Theme Codes

E. Bank Staff

F. Results Framework Analysis

G. Ratings of Project Performance in ISRs

H. Restructuring

I. Disbursement Graph

1. Project Context, Development Objectives and Design ...................................... 12 2. Key Factors Affecting Implementation and Outcomes ...................................... 17 3. Assessment of Outcomes .................................................................................... 24 4. Assessment of Risk to Development Outcome .................................................. 28 5. Assessment of Bank and Borrower Performance ............................................... 28 6. Lessons Learned ................................................................................................. 31 7. Comments on Issues Raised by Borrower/Implementing Agencies/Donors ..... 32

Annex 1. Project Costs and Financing .......................................................................... 33 Annex 2. Outputs by Component .................................................................................. 34 Annex 3. Economic and Financial Analysis ................................................................. 40 Annex 4. Grant Preparation and Implementation Support/Supervision Processes ....... 41 Annex 5. Beneficiary Survey Results ........................................................................... 43 Annex 6. Stakeholder Workshop Report and Results ................................................... 43 Annex 7. Borrower's ICR and Comments on Draft ICR .............................................. 43

Annex 8. Comments of Cofinanciers ............................................................................ 53 Annex 9. List of Supporting Documents ...................................................................... 56

MAP No 33420R3

5

DATA SHEET

A. Basic Information

Country: Indonesia Project Name:

National Program for

Community

Empowerment in Urban

Areas (PNPM UPP)

Project ID: P096921 L/C/TF Number(s): IBRD-75040,IBRD-

76640,IDA-43840

ICR Date: 06/19/2015 ICR Type: Core ICR

Lending Instrument: SIL Borrower: GOVERNMENT OF

INDONESIA

Original Total

Commitment: USD 177.68M Disbursed Amount: USD 286.88M

Revised Amount: USD 291.96M

Environmental Category: B

Implementing Agencies:

Ministry of Public Works

Cofinanciers and Other External Partners:

B. Key Dates

Process Date Process Original Date Revised / Actual

Date(s)

Concept Review: 05/16/2007 Effectiveness: 07/22/2008 07/22/2008

Appraisal: 12/10/2007 Restructuring(s):

05/01/2009

12/12/2011

06/28/2013

Approval: 05/20/2008 Mid-term Review:

Closing: 03/31/2011 12/31/2014

C. Ratings Summary

C.1 Performance Rating by ICR

Outcomes: Satisfactory

Risk to Development Outcome: Moderate

Bank Performance: Satisfactory

Borrower Performance: Moderately Satisfactory

6

C.2 Detailed Ratings of Bank and Borrower Performance (by ICR)

Bank Ratings Borrower Ratings

Quality at Entry: Satisfactory Government: Moderately Satisfactory

Quality of Supervision: Satisfactory Implementing

Agency/Agencies: Satisfactory

Overall Bank

Performance: Satisfactory

Overall Borrower

Performance: Moderately Satisfactory

C.3 Quality at Entry and Implementation Performance Indicators

Implementation

Performance Indicators

QAG Assessments (if

any) Rating

Potential Problem Project

at any time (Yes/No): No

Quality at Entry

(QEA): None

Problem Project at any time

(Yes/No): No

Quality of Supervision

(QSA): None

DO rating before

Closing/Inactive status:

Moderately

Satisfactory

D. Sector and Theme Codes

Original Actual

Sector Code (as % of total Bank financing)

General education sector 27 27

Health 27 27

Other social services 13 13

Sub-national government administration 6 6

Urban Transport 27 27

Theme Code (as % of total Bank financing)

City-wide Infrastructure and Service Delivery 50 50

Participation and civic engagement 50 50

E. Bank Staff

Positions At ICR At Approval

Vice President: Axel van Trotsenburg James W. Adams

Country Director: Rodrigo A. Chaves Joachim von Amsberg

Practice Manager/Manager: Abhas Kumar Jha Sonia Hammam

Project Team Leader: George Soraya George Soraya

ICR Team Leader: Evi Hermirasari

ICR Primary Author: Andre Oosterman

7

F. Results Framework Analysis

Project Development Objectives (from Project Appraisal Document)

The project development objective is to ensure that the urban poor in PNPM-Urban locations

benefit from improved socio-economic and local governance conditions.

Revised Project Development Objectives (as approved by original approving authority)

(a) PDO Indicator(s)

Indicator Baseline Value

Original Target

Values (from

approval

documents)

Formally

Revised

Target Values

Actual Value

Achieved at

Completion or

Target Years

Indicator 1 : Improved household expenditure rates or improved access to economic and social

services in 80% of participating kelurahans (wards)

Value

quantitative or

Qualitative)

0 7,800 7,500 8,511

Date achieved 07/31/2008 03/31/2011 12/31/2011 12/31/2014

Comments

(incl. %

achievement)

With the approval of the AF, the targeted number of kelurahans increased from 7800 to

9457, and the proportion of those experiencing improvement was reduced from 100% to

80%. The revised target was rounded off to (80% x 9,457=) 7,500.

Indicator 2 : Infrastructure built is 20% less expensive than that built by non-community based

approaches in 80% of participating kelurahans

Value

quantitative or

Qualitative)

0 6,240 7,500 7,566

Date achieved 07/31/2008 03/31/2011 12/31/2011 12/31/2014

Comments

(incl. %

achievement)

Actual achievement based on annual survey conducted by the PNPM Urban consultants

and facilitators on the completed subprojects between 2009-2014.

Indicator 3 : 80% satisfaction levels from beneficiaries regarding improved services and local level

governance

Value

quantitative or

Qualitative)

0 80% 80% 87%

Date achieved 07/31/2008 03/31/2011 12/31/2011 12/31/2014

Comments

(incl. %

achievement)

Actual achievement based on results of quantitative survey of 2,400 households in 80

kelurahans. The survey assessed satisfaction of beneficiaries both with completed

subprojects and planning process at community level

8

(b) Intermediate Outcome Indicator(s)

Indicator Baseline Value

Original Target

Values (from

approval

documents)

Formally

Revised Target

Values

Actual Value

Achieved at

Completion or

Target Years

Indicator 1 : Min. 40% participation rate of the poorest and vulnerable community members in

planning and decision making meetings

Value

(quantitative

or Qualitative)

0 ≥40% ≥40% 35%

Date achieved 07/31/2008 03/31/2011 12/31/2011 12/31/2014

Comments

(incl. %

achievement)

The achievement (35%) was lower than the target (at least 40%) because socialization

was sometime less effective, and the poor and vulnerable community members tended

to rely on neighborhood representatives instead of participating in meetings themselves.

Indicator 2 : Min. 40% participation rate of women in planning and decision making meetings

Value

(quantitative

or Qualitative)

0 ≥40% ≥40% 44%

Date achieved 07/31/2008 03/31/2011 12/31/2011 12/31/2014

Comments

(incl. %

achievement)

Indicator 3 : Min. 30% of the adult population voting in BKM elections at the neighborhood level

Value

(quantitative

or Qualitative)

0 ≥30% ≥30% 33%

Date achieved 07/31/2008 03/31/2011 12/31/2011 12/31/2014

Comments

(incl. %

achievement)

Indicator 4 : BKMs formed in minimum of 90% of kelurahans

Value

(quantitative

or Qualitative)

0 ≥90% ≥90% 99%

Date achieved 07/31/2008 03/31/2011 12/31/2011 12/31/2014

Comments

(incl. %

achievement)

Indicator 5 : Min. 90% of kelurahans with Community Development Plans (CDPs) completed and

ratified

Value

(quantitative

or Qualitative)

0 ≥90% ≥90% 97%

Date achieved 07/31/2008 03/31/2011 12/31/2011 12/31/2014

Comments

(incl. %

achievement)

9

Indicator 6 :

Min. 80% of local governments provide cost-sharing: 20% cost-sharing for local

governments with low fiscal capacity, and 50% cost-sharing for local governments with

high fiscal capacity

Value

(quantitative

or Qualitative)

0 ≥80% ≥80% 92%

Date achieved 07/31/2008 03/31/2011 12/31/2011 12/31/2014

Comments

(incl. %

achievement)

Indicator 7 : Number of each type of infrastructure, economic, and social activities completed in

minimum 80% of participating kelurahans (wards)

Value

(quantitative

or Qualitative)

0 7,800 7,500 9,457

Date achieved 07/31/2008 03/31/2011 12/31/2011 12/31/2014

Comments

(incl. %

achievement)

Infrastructure, economic, and social activities were completed in all 9,457 participating

kelurahans.

Indicator 8 : Min. 70% of infrastructure works evaluated as of good quality

Value

(quantitative

or Qualitative)

0 ≥70% ≥70% 97%

Date achieved 07/31/2008 03/31/2011 12/31/2011 12/31/2014

Comments

(incl. %

achievement)

Actual achievement was confirmed through an infrastructure certification process

carried out by local consultants based on national guidelines.

Indicator 9 : Min. 70% of kelurahans with revolving loan funds (RLFs) having a loans at risk (LAR)

ratio ≥ 3 months of <10%

Value

(quantitative

or Qualitative)

0 ≥70% ≥70% 28%

Date achieved 07/31/2008 03/31/2011 12/31/2011 12/31/2014

Comments

(incl. %

achievement)

The achievement (28%) is lower than the target of min 70%, mainly because of limited

capacity and high turnover among the financial unit (UPK) staff, and the small capital

base.

Indicator 10 : Min. 90% of kelurahans with RLFs have a cost coverage ratio of >125%

Value

(quantitative

or Qualitative)

0 ≥90% ≥90% 76%

Date achieved 07/31/2008 03/31/2011 12/31/2011 12/31/2014

Comments

(incl. %

achievement)

The achievement (76%) is than the target of min 90%, for the same reasons as the

mentioned in the comment on Indicator 9.

Indicator 11 : Min. 90% of kelurahans with RLFs have annualized return on investment of >10%

Value

(quantitative

or Qualitative)

0 ≥90% ≥90% 26%

10

Date achieved 07/31/2008 03/31/2011 12/31/2011 12/31/2014

Comments

(incl. %

achievement)

The achievement (26%) is lower than the target of min 90%, for the same reasons as the

mentioned in the comment on Indicator 9.

Indicator 12 : Min. 30% participation rate of women as member of community groups (KSMs)

Value

(quantitative

or Qualitative)

0 ≥30% ≥30% 38%

Date achieved 07/31/2008 03/31/2011 12/31/2011 12/31/2014

Comments

(incl. %

achievement)

Indicator 13 : 90% of OCs providing timely & accurate data through MIS

Value

(quantitative

or Qualitative)

0 ≥90% ≥90% 94%

Date achieved 07/31/2008 03/31/2011 12/31/2011 12/31/2014

Comments

(incl. %

achievement)

Actual achievement is the result of quarterly assessments conducted by OCs.

Indicator 14 : 70% of BKMs with completed annual financial audits

Value

(quantitative

or Qualitative)

0 ≥70% ≥70% 98%

Date achieved 07/31/2008 03/31/2011 12/31/2011 12/31/2014

Comments

(incl. %

achievement)

G. Ratings of Project Performance in ISRs

No. Date ISR

Archived DO IP

Actual Disbursements

(USD millions)

1 11/27/2008 Satisfactory Satisfactory 80.00

2 05/08/2010 Satisfactory Satisfactory 201.18

3 06/17/2011 Moderately Satisfactory Moderately Satisfactory 226.18






11

H. Restructuring (if any)

Restructuring

Date(s)

Board

Approved PDO

Change

ISR Ratings at

Restructuring

Amount

Disbursed at

Restructuring

in USD millions

Reason for Restructuring & Key

Changes Made DO IP

05/01/2009 S S 121.68

12/12/2011 MS MS 226.18

06/28/2013 MS MS 263.69

I. Disbursement Profile

12

1. Project Context, Development Objectives and Design

1.1 Context at Appraisal

From the late 1960s until the mid-1990s, Indonesia experienced a sustained and uninterrupted

reduction in poverty. The share of the population below the national poverty line dropped from over

40% in 1976 to less than 12% in 1996. However, the East Asian financial crisis, which started in July

1997, wiped out much of these gains. By 1998, the poverty rate had doubled to over 24%, and the

number of Indonesians living below the poverty line increased to almost 50 million. The financial

crisis did not only cause economic hardship but also led to greater demand for public participation in

government affairs, both by individual citizens and by sub-national governments. In 1999, the

National Parliament passed two new decentralization laws, partially in response to the effect of the

crisis on Indonesia's political landscape. These laws shifted many responsibilities for government

affairs from the center to the provincial and district (kota and kabupaten) governments, and also pro-

vided these sub-national governments with the financing needed to implement their newfound

responsibilities.

Against this background, the Government of Indonesia (GoI) created a national development

program (Program Pembangunan Nasional or PROPENAS) for 2000-2004 aimed at increased public

participation, improved governance, and economic recovery. The economic recovery program

prioritized poverty alleviation and the development of a community-based economic system.

PROPENAS 2000-2004 emphasized the need to establish partnerships between government,

communities and the private sector, and to increase efforts to overcome poverty and social

instabilities. To help achieve these objectives, the Bank assisted GoI with the implementation of a

series of poverty reduction projects, initially consisting of the Urban Poverty Project (UPP) series in

urban areas and Kecamatan Development Project (KDP) series in rural areas. Implementation of the

first UPP and KDP was commenced in 1998.

In 2007, GoI launched the National Program for Community Empowerment, which is better known

by its Indonesian acronym PNPM (Program Nasional Pemberdayaan Masyarakat). PNPM was

established as the operational umbrella for all community-based development (CDD) programs in the

country, including the KDP and UPP series. An inter-ministerial working group (Kelompok Kerja

Pengendali PNPM Mandiri or “Tim Pengendali”) provides oversight of PNPM at the national level.

The group is chaired by the Coordinating Minister for Community Welfare and its members include

representatives from the Ministry of Home Affairs, the National Planning Board (BAPPENAS), the

Ministry of Finance, and relevant line ministries.

Rationale for Bank involvement. In 2008, the Bank had financed––or was financing––CDD

programs in urban areas through three projects: UPP1, UPP2, and UPP3 (Table 1). This ICR covers

PNPM Urban (“the project”), which was mainly a continuation of UPP2 and UPP3 to help

consolidate PNPM as a national program. The project would cover 7,800 wards (kelurahans) in all

provinces of Indonesia, about 7,250 of which were already covered by its predecessor projects.

PNPM Urban not only addressed GoI’s development objectives, but was also fully consistent with

the Bank’s Country Assistance Strategy (CAS) for 2004-2007, which aimed at sustaining economic

recovery, promoting broad-based growth, and delivering better public services for the poor.1

1 The strategy was in place during most of the project preparation period, and was extended to also cover 2008. The

CAS was superseded by the Country Partnership Strategy for 2009-2012.

13

Table 1: Key Features of Community Driven Development Projects in Urban Areas

(at time of appraisal of PNPM Urban)

Key Feature UPP1

(Cr. 3210)

UPP2

(Ln. 4664,

Cr. 3658/1)

UPP3

(Ln. 4779,

Cr. 4063)

PNPM Urban

(Ln. 7504,

Cr. 4384)*

Implementation Period 1999-2004 2002-2009 2005-2011 2009-2011

# Provinces covered 6 13 15 33

# Kelurahans covered 2,548 7,273 2,396 7,800

Bank financing (US$ million)* 100 235 139 167

Closing month Jun 2004 Dec 2009 Mar 2011 Mar 2011

Source: World Bank (2015)

* At appraisal.

1.2 Original Project Development Objective and Key Performance Indicators

The PAD stated the project development objective (PDO) as “to ensure that the urban poor in PNPM

Urban locations benefit from improved socio-economic and local governance conditions”. The PDO

in the legal agreements was identical.

Key Indicators

1. Improved households expenditure rates or improved access to economic and social services in

7,800 kelurahans (wards) in 2009,

2. Infrastructure built is 20% less expensive than that built by non-community based approaches in

80% of participating kelurahans, and

3. 80% satisfaction levels from beneficiaries regarding improved services and local level

governance (impacts taken from representative sample).

Intermediate results indicators

The PAD identified 14 results indicators, aimed at measuring intermediate results in the three

components described in Section 1.5 (indicators are in brackets):

1. Organized community groups increasingly voice demands and local governments deliver

improved services to the urban poor (participation rate of the poorest and vulnerable community

members in planning and decision-making meetings, participation rate of women in planning and

decision-making meetings, percentages of adult population voting in elections for Community

Trustee Boards––Badan Keswadayaan Masyarakat or BKM––at the neighborhood level,

percentage of kelurahans with BKM formation, percentage of kelurahans with Community

Development Plans completed and ratified, percentage of local governments providing cost-

sharing of at least 20%).

2. BKMs deliver improved services to the poor (number and type of infrastructure, economic, and

social activities completed, percentage of infrastructure works evaluated as of good quality,

percentage of kelurahans with revolving loan funds having a loans at risk ratio ≥ 3 months of less

than 10%, percentage of kelurahans with revolving loan funds having a cost coverage ratio

exceeding 125%, percentage of kelurahans with revolving loan funds having an annualized

return on investment exceeding 10%, participation rate of women as member of community

groups).

14

3. Consultants provide technical assistance and implementation support to the project (percentage of

oversight consultants providing timely and accurate data through the MIS, percentage of BKMs

with completed annual financial audits).

1.3 Revised PDO and Key Indicators

The PDO and the three outcome indicators remained unchanged during project implementation.

However, upon approval of the Additional Financing (see Section 1.7), the target for outcome

indicator #1 “Improved households expenditure rates or improved access to economic and social

services in 7,800 kelurahans (wards) in 2009” was revised in three respects:

1. Change in overall target. The number of kelurahans to be covered by the project was increased

from 7,800 to 9,457.2

2. Change in calculation of target. The targeted number of kelurahans with improved household

expenditure rates or improved access to services was no longer set as an absolute number, but as

a percentage of the total number of participating kelurahans. The target was set at 80%, which

was deemed more realistic that the 100% used in the PAD. As a result, the original project would

need to achieve the target in 6,240 kelurahans, and the additional financing would need to

achieve the target in 4,900 kelurahans.3

3. Change in target year. The year in which the revised target should be achieved was moved from

2009 to 2011 in line with the shift of the closing date.

The revised outcome indicator #1 was therefore as follows: “Improved households expenditure rates

or improved access to economic and social services in 80% participating kelurahans (wards) in

2011”. Performance against this indicator, as well as against all other outcome and intermediate

results indicators, was monitored separately for kelurahans financed by the original loan and credit

and for kelurahans financed from the proceeds of the AF.

1.4 Main Beneficiaries

The originally intended primary beneficiaries of the project consisted of some 13.8 million poor

persons living in 7,800 target kelurahans located in 33 provinces. The population of these kelurahans

would benefit both directly and indirectly from improved community infrastructure and services,

grant assistance for poorest and most vulnerable groups (such as scholarships and training courses),

and improved access to microcredit––all planned and implemented through a community-driven

development (CDD) process. Secondary beneficiaries would be: (i) the Directorate General of

Human Settlements in the Ministry of Public Works (MPW, the implementing agency), which would

benefit from increased capacity, (ii) ward-level facilitators and partner agencies providing

implementation support, (iii) the kabupaten and kota governments in which the 7,800 target

kelurahans were located, who would benefit from training, field studies, and institutional

strengthening aimed at improving their planning capabilities, and (iv) Community Trustee Boards,

which would benefit from technical assistance and advisory services in the process of establishing

and strengthening their role in channeling CDD funds to appropriate community activities and

investments.

2 There were about 4,600 repeated kelurahans. The total number of participating kelurahans was therefore (7869 +

6168 –4,600 =) 9,457.

3 The figures were calculated as follows: 80% x 7,800 = 6,240, and 80% x 6,168 = 4,934. The latter figure was

rounded off to 4,900. The total of (7,800+4,900=12,700 is higher than 9,457 because of overlapping kelurahans.

15

1.5 Original Components

Component 1: Community and Local Government Capacity Building. Project Cost: US$31.62

million (100% Bank financing)

This component would fund the cost of facilitators to carry out social intermediation activities,

training, workshops, press releases/conferences, and other mass communication activities, meetings

and focus group discussions, and production of socialization materials and publications. This

component would also support the training and socialization of local government staff on PNPM and

leverage support from local governments to share costs of subprojects. Taken together, these

activities would help ensure that infrastructure subprojects financed by Component 2 would be less

expensive than those realized by non-community based approaches and that beneficiaries would be

satisfied with improved services and local level governance.

Component 2: Kelurahan Grants and Sub-loans. Project Cost: US$248.92 million (52% Bank

financing)

To improve access to economic and social services, this component would provide block grants to

participating kelurahans to finance activities identified in Community Development Plans (CDPs),

including: (i) specific high-priority infrastructure investments, (ii) microcredit loans channeled

through community-managed revolving loan funds (RLFs), and (iii) direct assistance through social

programs for the benefit of the poorest and most vulnerable groups or individuals to pay for

scholarships, health care, training and similar activities. Kelurahan grants would not finance activities

on the project’s negative list, such as weapons, religious buildings or government offices. Each

participating kelurahan would receive a one-time block grant of IDR 150 million to IDR 350 million

to finance investments in one or more of the above categories, in accordance with their community

development plans (with a maximum allocation of 30% to RLFs for newly participating kelurahans

and 20% to well-performing RLFs in other kelurahans). The block grant amount would depend on

the population of the kelurahan.

Component 3: Implementation and Technical Assistance. Project Cost: US$23.51 million (67%

Bank financing)

The project would be managed by a Project Management Unit (PMU) in MPW. The project would

finance consultants and facilitators to assist in project implementation. Technical assistance (TA)

would be provided through National Management Consultants (NMC) at the central level, and

Oversight Consultant (OC) teams at the province level, with OC offices in the participating district

governments, and facilitators and community cadres at the kelurahan level. In addition, this compo-

nent would finance the project’s monitoring and evaluation program, and a complaints handling

system to help ensure that the local governance aspect of the PDO would be achieved.

1.6 Revised Components

The components remained unchanged during project implementation.

1.7 Other significant changes

Additional financing. On May 1, 2009, the Bank approved an IBRD Loan of US$115 million for

Additional Financing (AF) to scale up the project geographically and enhance its impact. The AF

would be consistent with the PNPM Urban objectives and design. The proceeds of the AF would be

used to provide supplemental block grants to 4,593 kelurahans already supported by UPP2, UPP3 or

16

PNPM Urban itself, and to 1,575 kelurahans not previously covered by any of these projects. The AF

would also finance community and local government capacity building and implementation and TA

to support investments in the 6,168 additional kelurahans, incorporate awareness of disaster risk

management and emergency preparedness in the community socialization process, and institute

enhancements to the RLF component to make it more sustainable.

The original financing agreements were amended to extend the closing date by nine months to

December 31, 2011 (the same closing date as the AF) and to revise the target of one of the project’s

three outcome indicators (see Section 1.3). Since GoI had decided that PNPM Urban would only

receive block grant funding on an annual basis, it was not possible for it to commit to a longer

closing date, even though it was anticipated that the new closing date would not be sufficient to

complete the additional activities in full. The AF was estimated to reach about 8.5 million primary

beneficiaries, thereby increasing the total number of beneficiaries to 22.3 million. (The number of

kelurahans and beneficiaries mentioned in this paragraph refer to Bank-financed locations only; they

do not include locations co-financed by IDB.)

Other significant changes. Other important changes improvements that during project

implementation were the integration of all NMCs into a single NMC, the decentralization of

operations to 19 provincial working units in charge of overseeing project implementation and

managing the 4,500 kelurahan facilitators, and improved protocols for disbursement of block grants

to the BKM bank accounts.

Table 2: Project Cost and Financing Plan at Appraisal and after Approval of AF

(US$ million)

Item Appraisal

Estimate (A)

After AF

Approval (B)

Increase

(B-A)

Project Cost

Community and Local Government Capacity Building 31.6 76.1 44.5

Kelurahan Grants and Sub-loans 248.9 454.1 205.2

Implementation and Technical Assistance 23.5 42.6 19.1

Front-end fee 0.1 0.1 -

Financing Plan

Borrower 126.5 201.1 74.6

Bank financing 177.7 292.7 115.0

Islamic Development Bank - 79.0 79.0

Total Project Cost and Financing* 304.2 572.8 268.6

Source: World Bank (2015)

* Totals may not add up due to rounding off

Parallel financing. In 2009, GoI wished to increase the total financing for the project by about

US$ 269 million (Table 2). GoI would provide additional funds of about US$75 million from its own

resources, and US$115 million in Additional Financing would be provided by the Bank. This left a

financing gap of US$79 million, for which GoI requested financing from the Islamic Development

Bank. GoI decided that IDB would cover the 14 westernmost provinces (consisting of DKI Jakarta,

West Java, Banten, West Kalimantan, Riau Islands, Bangka-Belitung and all eight provinces on the

17

island of Sumatra). The utilization of the IBRD Loans and IDA Credit would be confined to the

remaining 19 provinces, instead of to all 33 provinces in the country (as originally envisaged).4

Further closing date extensions. The closing date of the financing agreements was revised two

more times, as follows:

As anticipated, the time was not sufficient to complete the additional activities financed from the

AF. On December 12, 2011, the Bank extended the closing date of both PNPM Urban and AF by

18 months, from December 31, 2011 to June 30, 2013. The Bank also approved a reallocation of

about US$34 million from kelurahan grants (for which GoI had already secured funding from

other sources) to goods and consultants’ services in order to finance community facilitators and

consultant services to support the implementation of CDPs in 2012.

Shortly before the revised closing date, about US$25 million in AF for kelurahan grants

remained undisbursed. Since GoI had some funds in their annual budget remaining that had to be

used by the end of the fiscal year, GoI gave priority to the use of these funds over external funds,

which necessitated a loan extension. GoI requested to use these funds for the further development

of the Neighborhood Development (ND) scheme (see Section 2.2 for details). In response to this

request, on June 28, 2013 the Bank extended the closing date of the AF by an additional 18

months to December 31, 2014 and reallocated part of the loan funds for kelurahan grants to the

expenditure category goods and consultants’ services.5 As a result of AF and extensions, the project closed almost four years later than originally planned.

Neighborhood Development (ND) and Poverty Alleviation Partnership Grant (PAPG). The

PNPM Urban and Additional Financing supported two advanced programs that were first piloted

under UPP2 and UPP3, and which have since become an integral part of PNPM (see Annex 2 for

details). ND targeted slum areas and involved the development of integrated spatial plans designed

with extensive community participation. It provided larger, more concentrated investments than

regular block grant amounts to improve the impact of infrastructure spending. PAPG provided

matching grants to local governments, which then worked together with BKMs to develop tertiary as

well as connecting infrastructure between kelurahans (refer to Annex 2 for details on NG and PAPG)

It is important to note that both ND and PAPG were based on PNPM’s core components (block

grants for infrastructure, economic and social activities implemented by communities with the help

of facilitators).6

2. Key Factors Affecting Implementation and Outcomes

2.1 Project Preparation, Design and Quality at Entry

Rating: Satisfactory

Soundness of background analysis. The design of PNPM Urban was based on UPP2 and UPP3,

which were being successfully implemented at appraisal, but with several modifications to reflect

lessons learned from their implementation. The most important of these lessons were the following:

(i) build institutional sustainability through active involvement of local governments, (ii) make future

kelurahan grant payments contingent upon collection of delinquent RLF loans, (iii) close poorly

4 In October 2012, the province of North Kalimantan was created. Project activities in this 34th province were co-

financed by the Bank, not by IDB. 5 It was not practical to finance ND development from the proceeds of loans for the successor project PNPM Urban

III because this project used different fund channeling methods. 6 For more information about these advanced programs, refer to Section 2.2 and Annex 2.

18

performing RLFs, (iv) improve the role of NMCs in capacity building, and (v) build sustainable

oversight and monitoring functions at community level (see PAD, pp. 8-9).

Assessment of project design. The PDO was relevant and corresponded to GoI’s own plans and

priorities. The components and implementation arrangements were well designed, realistic, and not

overly complex, having been largely adapted from proven approaches developed under the UPP

series. The components were also likely to result in the achievement of the PDO: through the

provision of kelurahan grants and sub-loans, Component 2 would improve access to economic and

social services in project locations. Components 1 and 3 would provide capacity building and

oversight aimed at improving local governance conditions to ensure that the economic and social

services would be provided in a cost-effective and sustainable manner. The results framework is

deemed adequate; it consisted of a relatively small number of well-defined and relevant performance

indicators. However, a weakness of the results framework was the lack of an indicator that measured

increases in access to services or household expenditure rates (the results framework simply

measured the number of kelurahan where an increase was reported). It is worth noting that GoI

wished to develop a uniform monitoring and evaluation (M&E) framework for both PNPM Urban

and PNPM Rural (although differences continue to persist, the design of the framework was

influenced by this policy objective). The social and environmental safeguards policies applied were

based on policies used for UPPs, and safeguards measures put in place were deemed adequate.

Adequacy of Government commitment, stakeholder involvement, and participatory processes.

GoI’s commitment to the project was strong and involved stakeholders at central as well as local

level. GoI prepared the project in accordance with a roadmap for the development of the nationwide

PNPM until 2015. This roadmap was prepared with the involvement of stakeholders at the central

and local levels, including NGOs and community-based organizations. The former President Soesilo

Bambang Yudhoyono also issued a decree to make all planning, budgeting, implementation, manage-

ment and funding mechanisms for PNPM projects subject to a single set of operational guidelines,

and a single overarching management structure.

Assessment of risks and risk mitigation measures. The overall project risk was rated “moderate”.

The risks assessment and mitigation measures were generally on target, and while some were more

successfully mitigated than others, there were no unanticipated risks or missed opportunities for

mitigation that seriously limited achievement of project outcomes.

2.2 Implementation

Regular program.

The project provided grants for small-scale infrastructure and social programs in just over 9,500

kelurahan in 19 provinces, and financed RLFs in the majority of these. About 78% of all kelurahan

grants were invested in physical infrastructure. The rest of the funds were used for social programs

(12%) and RLFs (10%).

The technical quality of construction was consistently rated as good, while the cost of construction

was at least 20% lower compared to local government contractors in the vast majority of the

participating kelurahans.7 The social programs cover training courses and other social programs

(including aid to the elderly, health care, additional nutrition for children of poor, scholarships

for young and disadvantaged students. The quantitative study shows that about 35% of social

7 Technical quality was measured annually through spot checks by NMC and OCs, and also covered by surveys

conducted by PT. Shiddiq Sarana Mulya in 2009, and the RAND Corporation in 2011 (see Annex 9).

19

program beneficiaries perceived the social program as very useful, and 61% as useful. About

69% BKMs interviewed manage to continue providing social grants. The RLFs were less

successful than anticipated even though a new strategy was adopted to improve the performance of

RLFs.

The project also financed the consultant and facilitators to help communities plan and implement the

infrastructure, social and RLF programs, provided training and capacity building to communities and

local governments. The AF also financed modules on community-based disaster risk management

and mainstreamed tools for reducing vulnerabilities against disaster risk and strengthening the role of

women in decision-making processes. The quality of project facilitation was not always consistent

but certainly sufficient to meet minimum standards. The implementing agency worked diligently to

meet challenging output targets and significant capacity was built at the community and local

government level.

Advanced programs.

Neighborhood Development (ND). The program provided a bigger grant than regular kelurahan

grants, for the planning and implementation of larger scale infrastructure. Because of their

complexity, ND locations required more facilitation and collaboration with local government.

During project implementation, ND was expanded from 18 to 255 kelurahans.

Poverty Alleviation Partnership Grant (PAPG). This was a matching grant to local governments

for financing poverty alleviation subprojects that were: (i) too big to be funded by regular

kelurahan grants or required local government involvement, (ii) located in more than one

kelurahan, (iii) not on the negative list for kelurahan grants, and (iv) jointly prepared, proposed

and implemented by BKMs in collaboration with local governments. The project supported

PAPG especially to enhance its sustainability in PAPG cities, such as facilitating cities that

replicated PAPG from their own funding.

Factors outside the control of the government or the implementation agency

There were no factors outside the control of the Borrower or implementing agency with a material

impact on the performance of the project.

Factors subject to the control of the government or the implementation agency

Positive factors

Strong political commitment. The project was able to secure support at the highest levels of

government. Virtually all participating district governments co-financed part of the project cost

from their own resources, and a substantial minority provided funds over and above the minimum

required amount. GOI also leveraged the project to deliver other programs, such as the

Neighborhood Upgrading and Shelter Sector Project funded by the Asian Development Bank

(ADB), for slum upgrading and post-disaster community-based housing.

Streamlined management. During implementation, the implementing agency integrated all NMCs

into a single NMC, reduced the number of OC contracts from 36 to 9, and decentralized

operations to 19 provincial working units in charge of overseeing project implementation and

managing kelurahan facilitators. The implementing agency also required all development

partners (in this case the Bank and IDB) to support PNPM in urban areas through the same

platform. As mentioned in Section 1.7, IDB financed PNPM in urban areas in 14 provinces not

20

covered by the Bank, and therefore operated independently from the Bank, albeit using the same

standard operating procedures.

Streamlined eligibility criteria. Starting in 2009, the implementing agency disallowed the use of

block grants for charity and requested that all block grants be invested in economically

sustainable activities. The implementing agency also encouraged the use of block grants for

larger and more focused investment for infrastructure to increase the project’s impact on poverty

alleviation.

Flexibility. From the start of implementation, the implementing agency and the Bank were wil-

ling to modify the design of the project in response to changing circumstances (by, for example,

providing basic technical assistance to kelurahans covered by earlier UPPs to enhance their

sustainability after project closure, and mainstreaming tools for reducing vulnerabilities against

disaster risk). The CDD design of the block grant component and flexibility in its implementation

also allowed the communities to allocate available funds where it mattered most to them. GoI and

the Bank also prepared and approved the AF to help scale up the project and modify the design to

enhance its effectiveness.

Effective anti-corruption strategy. See Section 2.4 for details.

Negative factors

Delays in disbursement. At the central level, there were delays in disbursement of funds,

especially for payment of consultants and facilitators. These delays only occurred in the first

quarter of each financial year because of late issuance of supporting documents for the annual

budget document. This problem was beyond the control of the implementing agency and affected

projects other than PNPM Urban.

Weaknesses in financial planning. At the beginning of every fiscal year, the Parliament approved

an annual budget ceiling for PNPM Urban. The budget would be financed from GoI’s own

resources, supplemented with financing from the Bank (and, in a later stage, IDB). During the

project implementation period, the budget ceiling was routinely set too low, which necessitated a

budget revision in the second half of the fiscal year. In some years, the budget revision was

approved too late to spend the full amount of GoI and Bank funds planned for that year.

Problems with assessing fiscal capacity. At appraisal, local governments classified by the

Ministry of Finance (MoF) as having high fiscal capacity were required to provide at least 50%

cost sharing, while other local governments were required to provide 20%. As a result of

inaccurate calculations and an initial overestimation of local government’s capacity, some local

governments were unable to provide the required counterpart funding. In 2012, MoF revised cost

sharing percentages to 20% and 5% respectively. After the revision, virtually all local

governments provided the minimum required cost sharing.

Constraints to improving RLF performance. The RLF strategy did not yield the positive results

expected, mainly because of limited capacity of communities to manage RLFs (see Annex 2 for

details). The performance of RLFs continued to deteriorate, in spite of restrictions on eligibility

for block grants and measures aimed at improving repayment and sustainability.

Mid-term review. In December 2010, the Bank conducted a mid-term review, about one year behind

the original schedule (January 2010). The review resulted in a series of recommendations, the most

important of which can be summarized as follows:

21

consistently apply identical policies for the implementation of PNPM Urban in areas covered by

the Bank and by IDB,

recruit facilitators using multi-year contracts to improve efficiency and ensure continuity of

facilitation in the field,

ensure that local governments fulfill their commitments to provide counterpart funds,

improve effectiveness of complaints handling mechanism to minimize misuse of funds (see

Section 2.4 for details),

increase the involvement of local governments in ND and improve the selection criteria to ensure

that ND grants benefit kelurahans most in need; the involvement of local governments was

important to ensure compatibility of ND with the local development plans and also helped secure

commitment (see Annex 2 for details), and

modify the strategy to improve RLF performance, especially through more frequent spot checks,

and improve targeting of the poor.

At the closing date, the implementing agency had implemented most of the above recommendations

except multi-year contracting of facilitators.

Effectiveness of risk mitigation. Risk assessment and mitigation measures were generally well

defined and effective. The key identified risks were: (i) constraints to the management capacity of

MPW for the scaled up program, (ii) delays caused by cost-sharing arrangements with local

governments, and (iii) insufficient budgets approved by parliament to meet full funding for PNPM

Urban were well mitigated during implementation. However, risks related to delays in budget

releases, delays in procurement of consulting services, and poor performance of RLFs were not

mitigated as successfully as expected, though these shortcomings did not fundamentally undermine

achievement of the project outcome.

2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization

Rating: Substantial

The project measured progress against key performance indicators (KPIs) using a management

information system (MIS), augmented by a series of qualitative and quantitative surveys and the

results of regular supervision missions.

(a) M&E design. M&E was based on a results framework that consisted of a series of well-

defined and realistic outcome and intermediate results indicators, with plausible time-bound targets

for each indicator. The exception was the PDO level indicator that measured increases in access to

services and household expenditure rates which should have been defined to measure actual increases

towards achieving these objectives instead of merely counting the number of kelurahans where an

increase was observed. At appraisal, a comprehensive management information system was in place

to monitor and evaluate UPPs. The same system would be used to monitor and evaluate the

implementation of PNPM Urban through the measurement of most indicators in the results

framework. In addition, the PAD listed a series of spot checks, surveys and studies that would be

undertaken during project implementation to complement the data to be collected by the MIS, and

evaluate the outcomes of the project. It is worth noting that since 2008, when PNPM Urban became a

national program, it was no longer possible to compare the performance of participating kelurahans

with a control group of non-participating kelurahans.

(b) M&E implementation. In general, the monitoring of the project was well implemented

through the MIS, spot-checking by consultants, and by the complaints handling system (complaints

could be submitted via the project website, by phone, text message or in writing). The MIS and

22

website gradually evolved into an integrated and user-friendly information system (also recording,

inter alia, payment advice from the State Treasury office, detailed RLF performance records, and

audit findings). Data quality improved over time. In addition, the Bank team together with the

Government also established a teleconference supervision system, which significantly expanded

supervision coverage. The project financed several studies and evaluations, including a quantitative

evaluation of PNPM Urban (see Annex 5 for details), a rapid assessment of gender-sensitive CDD

and disaster risk management, and a rapid appraisal of the ND scheme. Unfortunately, several

evaluations were of a qualitative nature or assessed perceptions (as opposed to facts) and were

therefore of limited use in helping to measure progress against key indicators. Also, the preparation

of most studies took longer and started later than envisaged.

(c) M&E utilization. The data collected by the MIS and the monitoring of progress helped the

project stakeholders at central as well as local level to: (i) identify and remedy quality problems and

incorporate better controls as the project progressed, (ii) fine-tune the targeting of resources to the

areas with the greatest needs and to help achieve the greatest impacts, and (iii) ensure transparency

and strengthen public confidence in the openness and fairness of the project. Significantly, the Bank

and implementing agency agreed to consider only data entered into the MIS as relevant for

monitoring and evaluation purposes. The project also shared common data with other CDD projects

through the MIS under the umbrella of PNPM that provides a reference for ministries to identify

active projects and allocate resources. By closing, the project’s website did not only provide access to

data on PNPM Urban itself, but also on the follow-up projects PNPM Urban III (Ln. 7866) and

PNPM Urban 2012-2015 (Ln. 8213). GoI is now using the MIS to monitor its nationwide CDD

program, which strongly suggests that the system will be sustainable in the long term. In 2009, the

World Bank selected the project website from 42 submissions in the category “most innovative use

of internet and cell phones” during the East Asia Pacific Innovation Days. Since 2011, the website

has also been used to distribute training and socialization materials. The system has since been

adopted for other projects in MPW.

2.4 Safeguard and Fiduciary Compliance

Safeguards. The project was classified as Environmental Category “B”, and overall complied with

the environmental and social safeguards requirements. Although some project activities triggered

safeguards related to environmental assessment (OD/BP 4.01), indigenous peoples (OD/BP 4.10) and

involuntary resettlement (OD/BP 4.12), none of these activities caused significant adverse impacts.

The safeguards frameworks documents were disclosed in Bahasa Indonesia and English. The

Indigenous Peoples Plan of this project was dated March 10, 2008 and disclosed on March 13,

2008. The Resettlement Plan was dated February 25, 2008 and disclosed on February 27, 2008. The

Environmental Assessment was dated December 14, 2007 and disclosed on March 13, 2008. The AF

did not trigger any additional safeguards.8 The project did not finance land acquisition, but did

receive voluntary land donations. Given the small size of the subprojects (the cost of which often did

not exceed US$2,000 equivalent), it was a priori expected that any adverse social and environmental

impacts of such subprojects would be small or negligible. Indeed, the project generated substantial

social and environmental benefits, especially through investments in drainage, water supply and sani-

tation. However, the recording of potential social and environmental impacts was initially poor,

especially with regards to voluntary land donations (VLD). The VLD documentation improved at the

8 For the original financing agreements, two safeguard documents were disclosed: (i) Safeguards Policy Frame-

works (Jakarta, 2008), and (ii) Indonesia National Program for Community Empowerment in Urban Areas Project:

Additional Financing – Integrated Safeguards Data Sheets (Washington DC, 2009). After approval of the AF, a

third safeguards document was disclosed locally: the Safeguards Frameworks PNPN Urban III (Jakarta, 2010).

23

final stages of project implementation. By the end of 2014, about 70% of sub-project proposals had

complete information about VLD. There was no independent monitoring of compliance with

safeguards, given the small, scattered nature of the interventions.

Fiduciary Compliance

Financial management. The project’s financial management generally complied with the Bank’s

minimum requirements under OP/BP10.02 and 13.20. Several mechanisms were put in place to

improve the financial management capacity of government, consultants, facilitators, local govern-

ments and communities, including training and coaching. At community level, a community-based

control was applied. This includes the facilitator’s review and certification of community reports,

establishment of oversight consultants, display of financial reports in public areas, and an extensive

complaints-handling system. Furthermore, community group financial statements were audited

annually by independent audit firms. The project received an unqualified opinion from an external

auditor in all years from 2009 to 2013, and the project’s financial management was generally accep-

table at closing. Also, the implementing agency ensured that local government auditors conducted

audits and followed up on their findings (which were recorded in the MIS). The involvement of local

government auditors helped increase the total audit coverage from 16% in 2009 to 34% in 2010 in

the 19 provinces covered by Bank financing, exceeding the target of 20% sample. However, the

project also encountered problems with financial management, primarily on RLF management,

delays in issuance of budget documents, and uneven capacity of the community and facilitators. The

project website has also been extensively used to support financial management.

Procurement. The project has generally complied with the World Bank’s Guidelines.9 Community

contracting worked well. Nonetheless, procurement remained moderately satisfactory throughout

project implementation, mainly because of avoidable delays in the recruitment of consultants. This

affected the quality of project implementation, such as mobilization of consultants and facilitators,

delayed distribution of socialization and training materials, and most importantly resulted in delays in

the training of facilitators and consultants. To overcome this problem, the Bank attempted to simplify

procurement procedures to avoid multiple amendments, and agreed to work with the same NMC and

OCs that have been working in the same areas, subject to performance evaluation. Also, rotation of

facilitators was minimized to allow for a more efficient procurement timeline, and online invoice

monitoring system has been developed and incorporated into the project website along with all

consultants’ contracts, which allow the consultants and government to monitor potential fiduciary

risks and invoice payment.

Compliance with Better Governance Action Plan. The project took steps to improve governance,

enhance transparency, and reduce opportunities for corruption and fraud. More specifically, the Bank

and the implementing agency agreed on a map of corruption risks and measures to mitigate such

risks. The areas of mapping consisted of procurement of consultants, project management, and

project implementation at community level which were regularly monitored and updated. A

comprehensive complaint handling mechanism was used to process inputs, complaints, and feedback

from the public by using project website, text-messaging (SMS), mail, telephone, or consultants

/facilitators. During 2008-2014, the PMU resolved 126,609 of 128,027 (99%) reported complaints10,

9 World Bank’s Guidelines: Procurement under IBRD Loans and IDA Credits dated May 2004 (revised October

2006), and Guidelines: Selection and Employment of Consultants by World Bank Borrowers dated May 2004

(revised October 2006) 10 Complaints included questions, suggestions and other inputs ..

24

including 1,127 corruption complaints, and had recovered about US$1.5 million (67%) of misused

funds totaling an estimated US$2.3 million equivalent. Follow-up actions, including local govern-

ment coordination and withholding of funds distribution, proved effective in combating corruption.

The very low level of misuse of funds was confirmed by external audits of the project. To improve

transparency and reduce corruption, the project website publishes details on consultant contracts,

payment status of the consultants, audit results and follow-up actions, and BKM financial reports.

2.5 Post-completion Operation/Next Phase

(a) Transition arrangements. For infrastructure constructed, the project required the

establishment of an O&M committee at the neighborhood level, and that maintenance costs to be

financed by community contributions. RLFs would continue to be managed by community members,

the administrative cost being financed from a portion of funds under management.

Presidential elections were held in July 2014, shortly before the loan closing date. At the time of

writing, the newly elected President had expressed an interest in continuing PNPM Urban on a

nationwide basis, but with modified implementation arrangements. The implementing agency intends

to use PNPM to help achieve its “100-0-100” objective in urban areas by the end of 2019. (“100-0-

100” means 100% access to safe water, 0% slum areas and 100% access to improved sanitation).

(b) Follow-up projects. In the second half of 2009, GoI requested the Bank to co-finance the

Third Program for Community Empowerment in Urban Areas.11 An agreement for US$150 million

IBRD loan (Ln. 7866) was signed in May 2010. In November 2012, GoI signed a loan agreement for

an IBRD loan (Ln. 8213) in an amount of US$292 million for the successor project PNPM Urban

2012-2015. The project also influenced the design of the ADB-funded Neighborhood Upgrading and

Shelter Sector Project for slum upgrading and post-disaster community-based housing.

(c) Future impact evaluation. Not applicable.

3. Assessment of Outcomes

3.1 Relevance of Objective, Design and Implementation

Relevance of objective

Rating: Substantial

The PDO of improving socio-economic development and local governance conditions remained

relevant throughout implementation. The project complemented the Government’s efforts to improve

socio-economic development and empower local communities. The newly elected President recently

confirmed his commitment to community-driven development projects, and intends to continue the

implementation of PNPM beyond 2014, albeit with modified implementation arrangements. The

CDD approach is also a key component of the Bank’s Country Partnership Strategy for 2009-2012,

which emphasized engagements with government counterparts and other stakeholders who are

committed to addressing critical governance and institutional challenges. The CPS for 2013-2015

identifies six thematic areas that form the core of the Bank’s engagement in Indonesia, three of which

are directly relevant to PNPM Urban: (i) Promoting Communities, Protecting the Vulnerable and

11 The original name of the project was “Second National Program for Community Empowerment in Urban Areas”

(PNPM Urban II). At the request of GoI, the name was changed to “Third National Program for Community

Empowerment in Urban Areas”, because GoI had already used the term “PNPM Urban II” to refer to the

Additional Financing to PNPM Urban.

25

Improving Health Outcomes, (ii) Ensuring Sustainable Development and Improving Disaster

Resilience, and (iii) Gender and Governance. Therefore, at completion the project was still

substantially relevant to the objectives of both the Bank and the Government of Indonesia.

Relevance of design

Rating: Substantial

Over 80% of the project cost consisted of investment in small-scale infrastructure, social services and

RLFs, which directly improved access to economic and social services to the intended beneficiaries,

thereby improving socio-economic conditions in the project area. Most of the remainder of the

project cost was allocated to capacity building and technical assistance aimed improving local

governance conditions, primarily through: (i) social accountability mechanisms, (ii) financial

management, (iii) facilitation, (iv) complaint handling mechanisms, and (v) management information

systems and reporting. The implementation of the Better Governance Action Plan also contributed to

improving local governance by identifying and mitigating corruption risks. The project’s design was

therefore highly relevant to the achievement of the project’s objectives throughout the

implementation period, although it is acknowledged that the PAD could have identified more specific

actions aimed at improving local governance.

3.2 Achievement of Project Development Objective (Efficacy)

Rating: Substantial

Outcome indicators. PNPM Urban was successful in achieving its development objective of

ensuring that the urban poor in project locations were able to benefit from improved socio-economic

and local governance conditions. All three outcome indicators exceeded targets set. Most

intermediate results indicators were achieved, except those relating to the participation of the poorest

of the poor and RLFs. The outputs and outcomes related to the indicators are discussed below.

Outcome indicator #1: Improved household expenditure rates or improved access to economic

and social services in 80% of participating kelurahans (wards). Outputs: through Component 2,

the project financed a large number of infrastructure subprojects, including over 12,280 km of village

roads, 3,400 km of drains, 12,600 water supply facilities and 1,100 educational facilities. It also

financed revolving loan funds and social programs, benefitting approximately 670,000 and 340,000

poor persons, respectively (see Annex 2 for details). Outcome: a study undertaken in 2012 estimated

that 90% of participating kelurahans enjoyed improved access to economic and social services

financed by the project, well above the target of 80%.12 All intermediate results indicators for

Component 2 were achieved, except those related to RLF performance. RLFs had lower returns on

investments and higher loans-at-risk ratios than was envisaged at appraisal, even though about 80%

kelurahans managed to recover their cost (see Annex 2).

Outcome indicator #2: Infrastructure built is 20% less expensive than that built by non-

community based approaches in 80% of participating kelurahans. Output: Component 1

financed facilitators to help communities build infrastructure using community contracting.

Component 3 monitored the quality of the infrastructure. Outcome: Quarterly spot checks found that

infrastructure financed by the project was on average 33% lower than the cost of equivalent works

undertaken by local government contractors in 87% of participating kelurahans. Given that the

12 Quantitative Evaluation the National Community Empowerment Program - Urban (NCEP-Urban), Ifa

Isfandiarni, Kementerian Pekerjaan Umum, 2012. Because of methodological difficulties, no attempt was made

to estimate project-induced increases in household expenditure rates.

26

project invested over US$300 million in infrastructure, this amounted to a cost saving of about

US$100 million.

Outcome indicator #3: >80% satisfaction levels from beneficiaries regarding improved services

and local level governance. Output: Components 1 and 3 financed means to improve local level

governance, notably facilitation and an advanced MIS (see also Section 3.1). ). For both components,

all targets were achieved, except for the participation of the poorest and most vulnerable groups

(35%), which was slightly below the target (40%), partly due to less effective socialization.13

Although the participation rate of women remained above target, the Bank and the implementing

agency took steps to improve the quality of female participation through targeted facilitation. About

one-third of the adult population voted in BKM elections at the neighborhood level, slightly higher

than the target of 30% (and substantially higher than for elections of community leaders unrelated to

PNPM Urban). In 99% of the project locations, BKMs were established and CDPs completed.

Outcome: the 2012 study estimated that 87% of project beneficiaries were satisfied with improved

services and local governance, significantly above target (there was no baseline for satisfaction levels

in repeater locations as the impact evaluation of UPP2 only measured satisfaction levels related to

RLFs only). The quantitative evaluation also showed that 75% respondents viewed the BKM election

as transparent, participatory and fair. According to a process evaluation, which was conducted in

2011, the vast majority of subprojects financed by PNPM Urban are functional and of good quality.14

This finding may serve as an explanation for the relatively high satisfaction levels: communities do

not merely state that they are satisfied, but actively use the subprojects for the intended purpose. In

relation to local governance, the survey measured beneficiaries’ satisfaction about the information

provided in neighborhood meetings and by BKMs, and the performance of BKMs in selecting

subproject proposals and inviting people to annual meetings.

The project had reached about 33.7 million poor beneficiaries, exceeding the revised target of 22.3

million (original loan and credit plus AF) by a substantial margin. The project also made a significant

contribution to the achievement of core sector indicators by providing improved access to clean

water (benefitting about 619,000 households), sanitation (822,000 HHs), electricity (110,000 HHs)

and health clinics (211,000 HHs).

3.3 Efficiency

Rating: Substantial

Efficiency of project management. The implementing agency improved the efficiency of the

project by reducing the number of NMC and OC contracts, decentralizing operations, and

establishing a single platform for Bank and IDB-financed operations (see also Section 2.2). As a

result of this streamlining operation, average personnel costs (per year per kelurahan covered by the

project) dropped by about 25 percent. Although the implementation period was four years longer

than envisaged at appraisal the additional time was used to cover a much larger number of

kelurahans and pioneer the development of advanced programs at a relatively modest increase in the

project management cost.

13 A substantial number of respondents lived in kelurahans that not received block grants before PNPM Urban

(reflecting the fact that the majority of kelurahans receiving PNPM Urban had not received any of the UPPs).

The participation of women and poor people in PNPM Urban community meetings would have been

significantly higher compared to the participation of these groups in community meetings prior to PNPM Urban. 14 Process Evaluation of the Program Nasional Pemberdayaan Masyarakat (PNPM)-Urban. RAND Corporation,

August 2011.

27

Efficiency of infrastructure subprojects. Quarterly spot checks undertaken by the NMC confirmed

the cost-effectiveness of community-based infrastructure financed by the project. By the end of 2011,

in 87% of all participating kelurahans, infrastructure built was at least 20% less expensive than infra-

structure that was constructed using non-community based approaches, well above the target of 80%

(see Annex 3 for details). The lower cost did not adversely affect quality, as over 95% of infra-

structure works were rated as being of “good quality”.

Efficiency of social programs and RLFs. The efficiency of investments in social programs was not

measured by the MIS or studies conducted by MPW or the Bank.15 The efficiency of investments in

RLFs was measured by the intermediate results indicators and it did not meet the targets set. The

results for the indicator return on investments (RoI) showed only about 30% of kelurahans managed

to achieve a RoI of more than 10%, out of the 90% targeted kelurahans. The cost coverage ratio

(CCR) showed better results. About 80% of kelurahans achieved a CCR of 125% (meaning that they

managed to recover their costs), out of the 90% targeted kelurahans. The RLFs and social programs

together accounted for less than 10% of the project cost, and so do not impact the overall rating too

much.

3.4 Justification of Overall Outcome Rating


The PDOs remained highly relevant throughout the implementation period. The relevance of the

design was rated substantial, especially with regard to improving socio-economic conditions. The

project’s efficacy was substantial as measured by its three outcome indicators and intermediate

results indicators linked to the achievement of the PDOs. The project exceeded most targets set for

the outcome and intermediate results indicators except those related to RLF performance. As for the

project’s efficiency, the cost of CDD infrastructure projects tends to be significantly lower than

similar programs financed by government contractors, although cost savings were to some extent

reduced by limited capabilities for O&M and heavy reliance on facilitators, which resulted in

increased project implementation costs. On the positive side, the implementing agency took steps to

streamline the management of the project, thereby reducing administrative overheads.

Because of the relevance of the project, the efficacy of the outcome and minor shortcomings, to

efficiency, the overall outcome of the project was rated “satisfactory”.

3.5 Overarching Themes, Other Outcomes and Impacts

Gender aspects. To strengthen the role of women in the preparation and execution of CDPs, the

project not only set a target for female participation in decision-making meetings, but also required

that at least 30% of facilitators be women (the actual figure was about 40%), collected MIS data on

gender-disaggregated basis, and increased awareness of gender aspects through the community

socialization process. The AF also financed a gender mainstreaming program, which helped to

overcome the initial underperformance on the project in this area. The implementing agency also

required its contractors to grant paid maternity leave to pregnant facilitators.

Other outcomes and impacts. A positive but unintended outcome of PNPM Urban and its

predecessor projects is an increased interest of the implementing agency in using community-based

approaches to achieve its strategic objectives (MPW is now using community-based systems for

15 The quantitative evaluation contains a review of the perceptions of the intended beneficiaries of the training

programs and RLFs, both of which were highly favorable. The evaluation did not attempt to compare costs with

benefits and recognized norms (as required for an efficiency analysis).

28

delivery of water supply and sanitation services in addition to local government institutions). The

implementing agency has also adopted PNPM’s management information and financial management

systems in community-based housing (REKOMPAK) and water and sanitation (PAMSIMAS)

projects.

3.6 Summary of Findings of Beneficiary Survey and Stakeholder Workshops

Beneficiary surveys. The implementing agency and Bank undertook evaluations of the project’s

impact, and reviewed the technical and economic performance of the subprojects used by project

beneficiaries. The key findings of these studies are summarized in Sections 3.2 and 3.3 and will not

be repeated here (see Annex 5 for details on the beneficiary survey and process evaluation).

Stakeholder workshops. The implementing agency organized annual meetings with NGOs,

community members and other stakeholders to exchange views about project implementation.

4. Assessment of Risk to Development Outcome

Rating: Moderate

The “Risk to Development Outcome” is defined as the risk that the (expected) development outco-

mes of the project—at the time of this assessment—will not be maintained or realized. The most

important constituent risks can be summarized as follows:

Technical risk. These were rated “low”, because the project’s investments were concentrated in

physical infrastructure that is relatively inexpensive to maintain and not technically complex.

To further mitigate the risk of inadequate O&M, the project required the establishment of an

O&M committee at the kelurahan level.

Financial risk. The risks associated with financial management and the ability of the

communities to monitor the use of funds provided in the future was deemed “moderate”.

Although the management of RLFs was less than satisfactory throughout the project as the

revolving funds were often unable to comply with the financial management guidelines, grant

usage for revolving funds were not significant (accounting for about 10% of the total funds for

block grant).

Political risk. This risk was also rated “moderate” given the fact that both central and local

governments showed a strong and long-term commitment to the financing of PNPM (albeit in a

different format; see Section 2.5). If the central and local governments would withdraw their

support for community-driven development plans, a major portion of the project’s investment

will have been wasted, especially investments in facilitation and training for community-driven

development. Although the impact of such a withdrawal would be significant, the likelihood of

this happening is in the near future is considered low given GoI’s long-term commitment to

CDD-type programs.

Based on these considerations, the overall “risk to development outcome” is rated “moderate”.

5. Assessment of Bank and Borrower Performance

5.1 Bank Performance

(a) Bank Performance in Ensuring Quality at Entry


The strengths of the original project design are:

29

Strategic relevance. The project successfully contributed to the further development of PNPM,

as outlined in the Government’s roadmap for the program, and helped implement the Bank’s

strategy for engagement with communities (refer to the Bank’s 2009-2012 CPS for Indonesia).

It also helped mainstreaming the Neighborhood Development program, a program designed to

integrate local government and community-based planning (PAPG) and incorporate disaster

risk management in the community socialization process.

Robustness. The design of the project was based on the design of UPP2 and UPP3, with minor

modifications to reflect lessons learned from these projects (notably the need to increase the

involvement of local government and make the disbursement of block grants to RLFs

contingent on performance) but otherwise using the same implementation arrangements. The

Bank deliberately kept design changes to a minimum, in order to consolidate PNPM Urban as a

full-fledged nationwide program, introducing substantial design changes during this process

would not have been prudent.

Strategic use of information and communication technology (ICT). The project design

promoted the use of ICT for planning, implementation and monitoring evaluation, not only to

be able to manage investments in a very large number of locations, but also to maintain a high

level of transparency in all aspects of the project cycle.

Direct communication with client. The task team leader was field-based, which facilitated

discussions on project preparation with the Borrower. This also helped keep preparation cost

low (US$200,000, or about 0.1% of the loan amount at appraisal).

On the other hand, the results framework did not contain indicators to explicitly measure the

outcomes of investments in revolving funds or social programs, and the PAD could have identified

more specific actions aimed at improving local governance. This shortcoming was deemed minor,

however, and the Bank’s performance in ensuring quality at entry was therefore rated “satisfactory”.

(b) Quality of Supervision


The Bank mobilized a multi-disciplinary supervision team with expertise in project management,

financial management, procurement, monitoring, and safeguards. Intensive supervision enabled the

Bank to identify and proactively address key issues adversely affecting achievement of the PDOs in

an early stage, notably problems with delays in budget execution and revision, consultant

mobilization, compliance with safeguard policies, and the mainstreaming of the ND scheme. Because

of these supervision efforts, performance in these areas improved over time (as described in detail in

the ISRs, which were generally of good quality and candid about problems encountered during

supervision missions). The Bank also flexibly responded to requests for Additional Financing and

project extensions to expand the project geographically and accelerate the development of the ND

and PAPG schemes. The Bank was able to keep supervision costs low, because most missions did

not only cover the project itself, but also other related projects (UPP3, PNPM Urban III and PNPM

Urban 2012-2015). This enabled the Bank to field typically two full supervision missions per year, in

addition to many shorter site visits between such missions (locations to be supervised were selected

using the MIS). Virtually all team members were based in the country office, which allowed for close

and continuous supervision. However, in spite of these efforts, the Bank was unable to halt the

deterioration of RLF performance, and to resolve some shortcomings in financial management.

These shortcomings were, however, deemed moderate, and the quality of Bank’s supervision was

consequently rated “satisfactory”.

30

(c) Justification of Rating for Overall Bank Performance


In light of the “satisfactory” ratings for the Bank’s performance in ensuring quality at entry and the

quality of supervision, the Bank’s overall performance was also rated “satisfactory.”

5.2 Borrower Performance

(a) Performance of the Government

Rating: Moderately Satisfactory

Throughout project implementation, the project enjoyed the strong support of the Government, which

together with local governments and communities financed about 30% of the total project cost. GoI

supported the Bank’s efforts for continuous innovation of PNPM to improve the impact of the

program on poor communities throughout Indonesia. It also maintained good relationships with other

development partners involved in PNPM, made a timely request for the AF, and helped increase the

audit coverage from 16% to 34%. At the same time, it is felt that the Government could have been

more pro-active in preventing delays in budget execution, and a better definition of fiscal capacity

could have avoided problems with providing the minimum required amount of local government

counterpart funding. In addition, better financial planning may have avoided the need to repeatedly

extend the project. These shortcomings were deemed moderate, however, and the performance of the

Government was consequently rated “moderately satisfactory”.

(b) Implementing Agency Performance


Through project implementation, MPW supported a bottom-up, community-driven approach to

poverty alleviation.16 It strongly supported the continuous improvement of a comprehensive M&E

system to maintain a high level of transparency in relation to the outcomes of the project. Following

the agreed Better Governance Action Plan, it also took steps to modify implementation arrangements

where needed and responded to complaints or suspicions about the misuse of funds with great vigor

(as part of these efforts, MPW developed a system for computerized verification of facilitator

payments and requested its contractors to pay all salaries by bank transfer). There were, however,

several challenges that stronger management may have been able to resolve such as delays in

consultant recruitment, incomplete safeguards documentation and delays in submission of interim

financial reports. These shortcomings were deemed minor and did not have a significant adverse

impact on the progress of the project.

(c) Justification of Rating for Overall Borrower Performance

Rating: Moderately Satisfactory

In light of the “moderately satisfactory” rating for the performance of the Government and the

“satisfactory” rating of the implementing agency, the overall performance of the Borrower was rated

“moderately satisfactory.”

16 The PAD and other Bank documents refer to the Ministry of Public Works as “Executing Agency”. This was

taken to mean “implementing agency” in the context of this ICR.

31

6. Lessons Learned

The implementation of PNPM Urban confirmed the key lessons from the UPP series: CDD programs

can deliver small-scale infrastructure at a lower cost at a comparable quality to equivalent

infrastructure constructed by local government contractors, provided that communities are adequately

facilitated. Including a revolving fund in a large-scale CDD project runs the risk that such funds will

not receive sufficient attention, which may result in poor performance. In addition, several other

lessons were learned, the most important of which can be summarized as follows:

Active involvement from local governments in CDD programs increases the impact of such

programs. The project helped mainstream local government involvement in PNPM in urban

areas, and demonstrated that financial incentives, coupled with intensive socialization and

training of local government officials and their consultants, are an effective means to create a

sustained interest on the part of local governments in planning and co-financing small-scale

infrastructure projects for and together with communities. PAPG was particularly good at

involving local governments and making them aware of the value of working with communities.

Intensive socialization is needed to prevent elite capture of knowledge. Poor community

members have limited understanding of PNPM Urban processes, and community leaders tend to

present information that they perceive as relevant to be disseminated. To improve the

participation rate of the poor, it is necessary to spend substantial time on socialization and

community training.

The implementation of CDPs at a scale with more complexity is constrained by limited human

resources. PNPM Urban supported the mainstreaming of Neighborhood Development, a scheme

aimed at integrated planning for the improvement of slum areas. It was found that community

participation for ND was more limited than in regular program primarily because of the

perceived complexity of planning at a higher spatial scale, and therefore requires intensive

facilitation.

Larger, more concentrated investments, together with local government involvement, have a

bigger impact on slum areas than ordinary block grants. The Neighborhood Development

program provided relatively large block grant amounts to slum areas, resulting in a better

targeting of the poor. Collaboration between BKMs and local governments during the preparation

of ND helped with the integration of tertiary infrastructure provided by the project with

secondary infrastructure provided by local governments.

The intensive use of ICT helps to maintain a high level of transparency in all aspects of the

project cycle. The implementing agency instituted the policy to make all information about the

project available on the project website, including payment advice from the State Treasury office,

detailed RLF performance records, and audit findings. This policy was effective in strengthening

public confidence in the project’s openness and fairness. The benefits of the MIS and website can

be further enhanced by using the system for the prevention of misuse of funds (for example, the

project website now shows all invoices paid by the project).There is no single optimal approach

to preventing misuse of funds. Instead, multiple approaches at multiple layers should be

promoted.

CDD programs can be effective mechanism for creating awareness of disaster risk management.

The project adapted the existing community socialization process to increase awareness of

disaster risks and help develop mitigation measures. This approach worked well and now forms

part of the PNPM in both urban and rural areas.

32

The application of principles of successful livelihoods program may help improve RLF

performance. In livelihoods programs, members of a group are required to meet certain

conditions, such as a minimum about of regular saving, internal lending and bookkeeping, before

they are permitted to borrow. The application of these conditions, coupled with more intensive

facilitation and capacity buildings of savings group, could help improve RLF performance.

Most of the above lessons have already been incorporated into successor projects, and are applicable

in developing countries with governance systems similar to Indonesia.

7. Comments on Issues Raised by Borrower/Implementing Agencies/Donors

(a) Borrower/Implementing agencies

The Government rates the process, outcome and implementation of the project “satisfactory”. GoI

considers PNPM Urban and its predecessor projects a model for providing support to the poor on a

nationwide scale, using a demand-driven and community-based planning process. The

implementation of the project was deemed “satisfactory,”, as it delivered much-needed basic

infrastructure and social support to the urban poor. The implementing agency rates its working

relationship with the Bank as satisfactory, and has expressed a keen interest in the further

development of PNPM Urban as a means to help achieve its “100-0-100” target by the end of 2019

(see Section 2.5). At the same time, the Government feels that overly detailed Bank supervision was

sometimes counterproductive to the smooth implementation of the project (see Annex 7 for detailed

comments of GoI on this ICR).

The Bank concedes that supervision was tight and sometimes detailed, but wishes to point out that

intense supervision was deemed necessary to identify and proactively address key issues adversely

affecting achievement of the PDOs in an early stage, and help ensure the timely achievement of

project targets.

(b) Cofinanciers/Development Partners

No issues were raised by the Islamic Development Bank.17

(c) Other partners and stakeholders

No issues were raised by partners and stakeholders other than representatives of the Borrower and the

implementing agency.

17 As mentioned in Chapter 1, IDB provided parallel financing for PNPM in urban areas in 14 provinces not covered

by the Bank and was therefore technically not a co-financier of Bank-financed activities.

33

Annex 1. Project Costs and Financing

a) Project Cost by Component (in US$ million equivalent)

Components Appraisal

Estimate

(US$ millions)

Actual/Latest

Estimate

(US$ millions)*

Percentage of

Appraisal

Community and Local Government Capacity

Building 31.62 105.26 333%

Kelurahan Grants and Sub-loans 248.92 346.92 139%

Implementation and Technical Assistance 23.51 37.39 159%

Designated Account - (0.45)

Total Baseline Cost 304.05 489.12 161%

Physical Contingencies - - -

Price Contingencies - - -

Total Project Costs 304.05 489.12 161%

Front-end fees 0.13 0.13 100%

Total Financing Required 304.18 489.25 161%

* Provisional figures (minor differences may arise between these data and final loan closing data)

b) Financing

Source of Funds Appraisal

Estimate

(US$ millions)

Actual/Latest

Estimate

(US$ millions)*

Percentage of

Appraisal

Borrower 75.00 } 141.95 } 112%

Communities 51.50

International Bank for Reconstruction

and Development (IBRD) 52.68 167.43 318%

International Development Association (IDA) 125.00 100.89 81%

Islamic Development Bank ** - 78.98 100%

Total 304.18 489.25 161%

* Provisional figures (minor differences may arise between these data and final loan closing data)

** IDB provided parallel financing for PNPM in urban areas in 14 provinces not covered by the Bank.

34

Annex 2. Outputs by Component

This annex summarizes the outputs of the project, and is largely based on a review of the

achievements of the outcome and intermediate results indicators mentioned in the PAD.

Component 1: Community and Local Government Capacity Building (actual cost: US$ 105.26

million)

The project aimed to establish representative and accountable community organizations that would

be able to provide services to the urban poor and increase the voice of the poor in public decision-

making. BKMs were established in virtually all participating kelurahans, and the vast majority of

these were deemed representative, effective and operating in a participatory manner, as measured by

the involvement of women, and poor and vulnerable persons in planning and decision meetings. On

average about 33% of the adult population in kelurahans voted in BKM elections (substantially

higher than the targeted value of 30%). As for local government capacity building, over 90% of local

governments covered by the project co-financed at least 20% of the project cost from their own

resources (higher than the target of 80%).

To build capacity at community and local government level, the project financed two types of

training: (i) basic training and (ii) thematic training. Basic training was compulsory for project

stakeholders at all levels, and aimed to give a comprehensive understanding of the project concept,

participatory planning, community development, budgeting, financial management, accountability

and monitoring systems. Thematic training was given to certain groups of participants working in

particular areas. Local government officials were trained in participatory planning, the project

organization and monitoring system, and audits of CDD-type projects (the latter was of particular

relevance to local inspectorates). BKM, community groups (Kelompok Swadaya Masyarakat or

KSM) and community volunteers were trained prior to the planning and implementation process to

learn how to facilitate community meetings, prepare subproject proposals and budgets, conduct

community procurement, monitor and assess the construction of infrastructure development, and

prepare project and financial reports. Consultants also received a series of thematic trainings in

accordance with their responsibilities, on subjects such as leadership, facilitation skills, socialization,

bookkeeping, financial management, infrastructure assessment, and MIS.

Training was delivered using a “training-of-trainers” model. The project trained about 400 national

trainers, both government officials and consultants. At the community level, the project provided

training to around 100,000 BKM members, 120,000 community volunteers and 70,000 community

groups (KSM) per year. In addition, the project trained about 800 local government officials, mostly

from planning and public works departments, and from local inspectorates.

Component 2: Kelurahan Grants and Sub-loans (actual cost: US$ 346.92 million)

The project was designed to provide grants to communities to finance activities aimed at poverty

alleviation, especially small-scale infrastructure. About 78% of all kelurahan grant funds (totaling

about US$175 million equivalent) were invested in physical infrastructure. The rest of the funds went

to social infrastructure and services (12%) and revolving loan funds (10%). About 87% of

beneficiaries were satisfied with the improved services and local level governance.

35

REGULAR PROGRAM

Infrastructure. Most of the grants for physical infrastructure were invested in village roads,

drainage and rehabilitation of houses (Table A2.1). Together, investments in these three types of

infrastructure accounted for over 80% of total spending of infrastructure block grants. In 87% of the

participating kelurahans infrastructure subprojects were at least 20% less expensive than that of

similar infrastructure built by non-community based approaches. Spot checks showed that over 97%

of the subprojects were of good quality. As reported in various aides-memoires, the technical quality

of the community infrastructure was generally good. In every sub-neighborhood where project-

financed infrastructure was built, the project required the establishment of an O&M committee, as a

means to instill a sense of ownership for the infrastructure financed by the project. Although this is

considered a useful step towards improved sustainability of community infrastructure, it is uncertain,

and in many cases even unlikely, that formation of these committees will provide the commitment

and resources needed. The Bank recognizes that long-term engagement is needed to resolve this

long-standing problem of inadequate O&M.

Social programs. The project financed about IDR 231 billion (or about US$20 million equivalent) in

training courses and other social programs (including aid to the elderly, health care, additional nutrition

for children of poor, scholarships for young and disadvantaged students), benefitting approximately

830,000 persons, of which over 670,000 were poor. Of total 2,400 HH respondents, about 13% (318 HH)

received social programs.

Revolving loan funds. The project financed about IDR 214 billion (about US$18 million equivalent,

or about 3.6% of the total project cost) of block grant allocations to RLFs. This investment benefitted

over 340,000 poor persons in 65,000 community groups. Of these, about 217,000 (or 63% of the

total) were women. As described in the main text, the performance of RLFs has deteriorated over

time and did not meet the targets set by the results framework. Even though less than 5% of the

project cost was used for RLFs, future projects may benefit from an understanding of the factors that

caused poor performance (see Box 1).

Box 1: REVIEW OF RLF PERFORMANCE

Background. RLFs were first established under UPP to serve urban poor people that otherwise would not have

access to credit (other than credit provided by informal lenders at extremely high rates). At present, there are

over 11,000 RLF management units (Unit Pengelolaan Keuangan or UPKs) that were established with support

of PNPM Urban or its predecessor projects. UPKs are small by most standards; by the end of 2014, over 90% of

UPKs had capital of less than IDR 300 million (US$25,000). Loan sizes range from IDR 0.5 million (about

US$40 equivalent) to IDR 2 million (about US$160), considerably smaller than loans provided by formal

financial service providers. Interest rates vary from 1.5% to 3% per month, with the upper end of the range

comparable to rates charged by Bank Indonesia Rakyat, the country’s largest micro-credit provider. Because

community groups, and not individuals, are responsible for loan repayment, and because borrowers do not need

(and usually are unable) to provide collateral, lending decisions are largely based on assessments of the character

of the borrower.

Performance over time. To measure RLF performance over time, the Bank used three performance indicators in

the project’s result framework, namely the loans at risk (LAR) ratio, the cost coverage ratio and the annualized

return on investment. Performance as measured by all three indicators deteriorated over time. Bank supervision

missions and evaluation studies list a large number of possible explanations, including:

high turnover of UPK staff (who are unpaid volunteers),

a small capital base, which makes it difficult to raise additional capital,

limited financial management capacity of UPK staff (although bookkeeping was generally up to standard),

limited capacity of facilitators to provide adequate training and oversight,

36

limited willingness of borrowers to repay funds (as investments in RLFs are often seen as central

government grants, not as funds that need to be repaid),

limited capacity of borrowers to set up and manage a financially viable business,

utilization of loan funds for non-revenue generating activities (such as consumptive use),

no requirement for borrowers to save (which is generally an effective tool for instilling financial

discipline), and

Implications for project design and implementation. Based on lessons learned during implementation of

predecessor projects, PNPM Urban generally did not allow kelurahans to allocate more than 20% of block grants

to RLFs, except in the case when RLF performance was rated “satisfactory” (a threshold of 30% was applied for

kelurahans that had not previously participated in PNPM). As performance of RLFs declined over time, block

grant allocations for RLFs also declined, dropping from 15% of total block grant funds in 2007-2009 to 10% in

2010. The Bank is currently exploring ways to link revolving loan funds directly to livelihood development

projects. Through the Financial Services Authority (Otoritas Jasa Keuangan or OJK), the Government is

planning to rationalize and possibly consolidate RLFs established under PNPM.

Table A2.1: Types of Infrastructure Financed by the Project (PNPM Urban and AF)

Type of

Infrastructure

Unit Volume Project Cost

PNPM

Urban

AF Total US$

million

% Total

Village roads km 7,882 4,998 12,280 113.0 52

Drainage km 1,809 1,643 3,452 38.0 17

Houses rehabilitated unit 33,280 24,317 57,598 24.8 11

Solid waste and communal

sanitation facilities

unit 30,023 32,767 62,790 15.4 7

Water supply facilities unit 7,414 5,228 12,642 8.7 4

Small bridges m 27,120 15,415 42,536 7.0 3

Education facilities unit 357 758 1,114 3.0 1

Community health facilities unit 1,656 578 2,235 3.0 1

Public markets unit 97 241 338 1.3 2

Electrification unit 7,857 15,057 22,914 1.1 52

Other activity 1,419 1,165 2,584 4.1 3

Total 219.4 100

Source: Project MIS (31 December 2014)

ADVANCED PROGRAMS

Neighborhood Development. ND was developed in 2007 during implementation of UPP2 and UPP3

and scaled up with financing from PNPM Urban to 255 kelurahans in 2009. Each participating

kelurahan received a grant of IDR 1 billion (about US$90,000), to be used for infrastructure (85%)

and capacity building for communities (15%). The implementation of ND was supported by an urban

planner and social marketing expert both at local government and community level. Communities

were facilitated to prepare a community settlement plans in consultation with local governments.

Over half of the infrastructure grants were used for roads (37%), drainage (13%) and water supply

and sanitation facilities (8%). At the project closing date, ND covered more than 700 kelurahans.

37

Since 2012, local governments have led the selection of ND locations and the implementation of the

program. Starting in 2014, ND started to focus on slum improvement in metropolitan areas for bigger

impact, with continuous strengthening provided to local government and increased collaboration

between communities and local government. Following the recently announced government policy to

eliminate slum areas by the end of 2019, the government is preparing a national program for slum

upgrading that will significantly draw on the model developed under the Neighborhood Development

program. Poverty Alleviation Partnership Grant (PAPG). The project supported the completion of PAPG

subproject activities in about 30 cities. Various initiatives were implemented to institutionalize the

PAPG mechanism into local government planning and budgeting systems, and also to encourage

partnership in ND locations. Despite the limited capacity of PAPG to fund large scale infrastructure,

partnerships between local governments and communities were generally effective. At project

closing, more than 30 local governments chose to replicate PAPG from their own funding.

Component 3: Implementation and Technical Assistance (actual cost: US$ 37.39 million)

The project was managed by a Project Management Unit (PMU) assisted by an administrative unit

(Satuan Kerja or SatKer). The PMU hired consultants and a large number of facilitators to assist with

project implementation. Technical assistance was provided through a National Management

Consultant (NMC) at the central level, and Oversight Consultant (OC) teams at the regional level,

with district offices in the participating kota/kabupaten, and facilitators and community cadres at the

kelurahan level. This component also covered the monitoring and evaluation of the project through

the MIS and a series of evaluation reports (see Annex 9 for details). Implementation support

primarily consisted of the following activities:

Complaints handling. During 2008-2014, the PMU resolved 126,609 of 128,027 (99%) reported

complaints18, including 1,127 corruption complaints, and had recovered about US$1.5 million

(67%) of misused funds totaling an estimated US$2.3 million equivalent. The complaint handling

system recorded complaints as well as misused of funds related findings as a result of financial

audit by BPKP, Local Government Inspectorate, as well as independent audit to BKM.

Monitoring of progress against development targets and performance indicators (supported by

reports of independent evaluators). About 94% of OCs provided timely and accurate data through

MIS (91% for kelurahans covered by the AF), which was higher than the target of 90%. The

NMC and OCs also helped ensure that almost all BKMs had completed financial audits by the

end of each year, exceeding the target of 70% by a substantial margin. However, due to delays in

consultant recruitment, evaluation reports were issued later than expected, and there was

substantial variation in the quality of the reports.

As described in the main text, the implementation of PNPM Urban took significantly longer than

anticipated at the time of appraisal. Refer to Table A2.2 for an overview of the amendments to the

original financing agreements.

18 ‘Complaints’ include questions, suggestions and inputs as well as complaints.

38

Table A2.2: Overview of Amendments to Original Financing Agreements for PNPM Urban

Key Feature PNPM Urban

(Ln. 7504, Cr. 4384)

Additional Financing

(Ln. 7664)

Original Loan/Credit agreement

Signing June 6, 2008 May 1, 2009

Effective July 22, 2008 July 29, 2009

Closing date March 31, 2011 December 31, 2011

Bank financing

(US$ million)*

177.68 115.00

# Kelurahans covered 7,800 6,168 (of which 4,600 repeater locations)

Loan/Credit Amendment

#1

Reason: Additional Financing

Signing date: 1 May 2009

Changes: - Closing date of PNPM Urban to December 31, 2011

- Outcome Indicator #1 of PNPM Urban

Before: Improved household expenditure rates or improved access to economic

and social services in 7,800 participating kelurahans (wards)

After: Improved household expenditure rates or improved access to economic

and social services in 80% participating kelurahans (wards)

- Target outcome indicator #1 of PNPM Urban from 7,800 to 6,240

- Target year in which the revised target should be achieved from 2009 to 2011

#2

Reason: The time was not sufficient to complete additional activities financed by AF and GoI

had secured funding for kelurahan grants from other resources

Signing date: December 12, 2011

Changes: - Revised closing date of PNPM Urban and AF to June 30, 2013

- Reallocation of US$34 million from kelurahan grants to good and consultants’

services in order to finance community facilitators and consultant services to

support the implementation of CDP

#3

Reason: GoI had secured funding for kelurahan grants from other resources so about US$25

million in AF for kelurahan grants remained undisbursed by the project closing date

and will be utilized for ND

Signing date: June 28, 2013

Changes: - Revised closing date of AF to December 31, 2014

Diagram A2.1 Evolution of PNPM Urban

UPP 2

1 x kelurahan grants

Small infrastructure

(group proposals).

2 x of kelurahan

population election

that voted in final stage of BKM

election.

Max 30% RLF.

Economic KSM consist of 2/3 poor.

Generalist facilitator

team.

Active LG

involvement on

PAPG only in year

3,4, 5 (extra block grants 50% from GoI

contribution

PNPM URBAN AND AF

3 x kelurahan grants.

Change composition of facilitator: economic

facilitator, technical facilitator, social facilitator

(since UPP 3).

Larger size infrastructure (financing kelurahan

level proposal).

Min 30 % of the adult population voting in BKM

election at the neighborhood level.

Quota 30% for women facilitator

New Strategy of RLF:

100% beneficiary is the poor (requirement).

New output indicators of RLF, put LAR, PAR,

CCR, and Rol.

MIS based online monitoring; incl. monthly

reporting system, on RLF has been developed.

Separated management of RLF and financial

management at national level.

Improve RLF guidelines, training modules and

add economic facilitator.

Limit only RLF with RR> 90% (for second round

and the rest).

Neighborhood Development (Pilot Project in

UPP 3).

Introduction of DRR.

Change of use social funds.

Local Government’s Involvement:

LG’s involvement from the beginning.

Allocate counterpart funding (min 20%).

Administer facilitator contract, kelurahan grant,

and training budget for community.

Lurah facilitate the PNPM activities together

with facilitator, BKM and volunteer.

Project Management:

Consolidation from 36 to 9 OCs.

Fixed-budget for community training.

BKM carry out annual financial audit.

Bookkeeping performance measurement of

BKM.

PNPM URBAN III

Neighborhood Upgrading

Scheme (funded by existing

projects and GoI).

Mainstreaming DRR in CSP/

PJM (GFDRR)

Allocate coordination grant to

facilitate Channeling program

(funded by existing projects and

GoI).

EV

AL

UA

TE

D

IN T

HE

UP

P 2

IM

PA

CT

EV

AL

UA

TIO

N

NO

T I

NC

LU

DE

D

IN T

HE

UP

P 2

IMP

AC

T E

VA

LU

AT

ION

IMP

RO

VE

ME

NT

FR

OM

TH

E E

VA

LU

AT

ED

CO

MP

ON

EN

TS

IMP

RO

VE

ME

NT

PNPM URBAN 2012-2015

Community Participation:

Strengthen women’s participation by providing

special grant allocation for women’s group

Increase community participation through

improvement socialization methods

Expanded Social Programs:

Increase the allocation of social funds and create exciting social programs initiated by LG

and link to the PNPM resources

Pilot livelihoods in 56 kelurahans to increase

effectiveness of social program

Neighborhood Development (ND)

Scaling up ND up to about 700 kelurahans

Focus location in slum in big or metropolitan

cities

LG proposed ND location to increase

ownership and sustainability

RLF:

Close the kelurahan grants for RLFfor the kelurahan that do not meet the indicators.

Provide assistance for capacity building and and legal basist of UPK

Local Government:

Strengthening the involvement of Local

Government by providing training and capacity building to be ble to work with BKMs and

BKM forums

Intensify collaboration with LG by developing instruments that open spaces between LG and

community.

In 2015

PNPM Urban support GoI policy on delivery of universal access of basic services and

improved living conditions to the poor,

including 100% access to safe drinking water, 0% slum areas, 100% access to basic sanitation,

Annex 3. Economic and Financial Analysis

Because the project adopted a programmatic approach, the activities financed by the project were not

pre-defined but demand-driven. For this reason, it was not possible to estimate indicators such as the

net present value or the economic internal rate of return (financial rates of return were not applicable,

given that the project exclusively financed non-revenue generating subprojects). Efficiency was

therefore gauged by unit rate norms. These are highly favorable for community-executed

infrastructure projects, as attested by the following studies:

In 2009, the implementing agency recruited a firm to review community-based infrastructure

development in PNPM Urban. The review contained an analysis of the cost-effectiveness of

infrastructure financed by the project. Based on an analysis of the cost of 28 small-scale

infrastructure projects in 12 kelurahans, it was estimated that the cost of infrastructure built by

communities was, on average, 23% lower than the cost of equivalent works built by local

government contractors. The main sources of cost savings were a reduction in the following:

contractor overheads, double or triple handling of materials, on-site design modifications, and

construction supervision in remote areas.

A recent review of PNPM Urban prepared by the RAND Corporation in 2011, called Process

Evaluation of the Program Nasional Pemberdayaan Masyarakat (PNPM)-Urban, found

favorable results for the quality of infrastructure projects undertaken and financed by previous

Bank-financed CDD projects.

In 2012, the PNPM Support Facility recruited a consultant to analyze the economic benefits of

small-scale infrastructure (the infrastructure subprojects were financed by PNPM Rural, but are

comparable to subprojects financed by PNPM Urban). The actual cost of the 48 projects was

compared to the theoretical cost of the same projects had they been provided by local government

contractors. The results of the analysis, which are described in Laporan Akhir Studi Skala Kecil

Analisis Manfaat Ekonomi Proyek Infrastruktur PNPM Mandiri Perdesaan, are comparable to

the findings of similar studies on cost savings associated with community-driven development

projects in Indonesia, especially when undertaken by communities themselves instead of using

community contracting. Cost savings were highest for projects in the roads and bridges sector

(22%-33%), followed by water supply (19%-28%), and irrigation (20%-25%).

Quarterly spot checks undertaken by the NMC confirmed the cost effectiveness of community-based

infrastructure financed by the project. By the end of 2011, in 87% of all participating kelurahans,

infrastructure built was at least 20% less expensive than infrastructure that was constructed using

non-community based approaches, well above the target of 80%. It is worth noting that the lower

cost did not adversely affect quality, as over 95% of infrastructure works were evaluated as being of

“good quality”.

The implementing agency improved the efficiency of project management by reducing the number of

NMC and OC contracts, decentralizing operations, and establishing a single platform for Bank and

IDB-financed operations It should be noted, however, that the use of single-year contracts for

facilitators made the implementation of the project less efficient that it would have been if multi-year

contracting had been applied (as recommended in the mid-term review).

41

Annex 4. Loan Preparation and Implementation Support/Supervision Processes

(a) Task Team members

Names Title Unit Responsibility/

Specialty

Lending

George Soraya Lead Municipal Engineer EASIS Task Team Leader

Zoe Elena Trohanis Infrastructure Specialist EASUR Co Task Team Leader

Fatima Shah Urban Economist EASUR Co Task Team Leader for AF

Evi Hermirasari Operations Officer EASIS Project Management and Monitoring

Yogana Prasta Operations Adviser EACIF Operations Adviser

Unggul Suprayitno Senior Financial

Management Specialist EAPFM Financial Management

Rizal H. Rivai Senior Procurement

Specialist EAPPR Procurement

Zhentu Liu Senior Procurement

Specialist COSU Procurement

Kumala Sari Operations Analyst EASIS Training and Socialization

Parwoto Tjondro

Sugianto Consultant EASIS Guidelines and Training

Melinda Good Senior Counsel LEGES Legal

Jana Halida Uno Poverty and Evaluation

Specialist EASIS Targeting and Evaluation

Indira Dharmapatni Senior Operations Officer EASIS Social Safeguards

Andrew Sembel Environment Specialist EASIS Environmental Safeguards

Sajjad Ali Shah Senior Operations Officer COSU Reviewer

Isabel Mutambe Program, Assistant EASUR Team Assistant

Ben Fisher Consultant OPCS Reviewer

Supervision /ICR

George Soraya Lead Municipal Engineer EASIS Task Team Leader

Evi Hermirasari Senior Urban Development

Specialist EASIS Project Management and Monitoring

Judy Baker Lead Economist EASUR Evaluation

Jana Halida Uno Operations Officer EASIS Evaluation

Yogana Prasta Operations Adviser EACIF Operations Adviser

Indira Dharmapatni Senior Operations Officer EASIS Safeguards

Unggul Suprayitno Senior Financial


Christina Irma Donna Senior Financial


Kumala Sari Operations Analyst EASIS Training and Socialization

Yulia Herawati Operations Analyst EASIS Evaluation

Budi Permana Procurement Analyst EAPPR Procurement

42

Names Title Unit Responsibility/

Specialty

Zenthu Liu Senior Procurement


Rizal H. Rivai Senior Procurement


Purnomo Sutantyo Consultant EASIS Procurement

Parwoto Tjondro

Sugianto Consultant EASIS Guidelines and Training

Lilis Suharti Consultant EASIS Financial Management

Patricia Yossianty Sonata Consultant EASIS Financial Management

Djumadi Achmad Consultant EASIS Financial Management

Yuli Safitri Widyawati Consultant EASIS MIS

Vivianti Rambe Consultant EASIS Environmental Safeguards

Virza S. Sasmitawidjaja Consultant EASIS Environmental Safeguards

André Oosterman Consultant EASIS ICR Report

Ratih Dewayanti Consultant EASIS Evaluation

(b) Staff Time and Cost

Stage of Project Cycle

Staff Time and Cost (Bank Budget Only)

No. of staff weeks US$ Thousands (including

travel and consultant costs)

Preparation

FY07 6.4 7.2

FY08 159.0 189.5

FY09 - -

Total: 165.4 196.7

Supervision/ICR

FY09 9 18.7

FY10 11 74.3

FY11 9 73.7

FY12 10 81.7

FY13 10 62.2

FY14 10 23.8*

FY15 12 17.7

Total: 71 352.2

* In 2014 and 2014, the supervision budget for the project was financed by the supervision budget for PNPM Urban

2012-2014, which covers the same locations as PNPM Urban.

43

Annex 5. Beneficiary Survey Results Quantitative evaluation. In 2011, the implementing agency commissioned a survey on the impact of

PNPM Urban throughout Indonesia. The results of the survey, which was based on interviews with

2,400 persons in 80 kelurahans were published in October 2012, in a report called “Quantitative

Evaluation the National Community Empowerment Program – Urban (NCEP-Urban) 2011”. The

study covered five island groups (Sumatera, Java, Nusa Tenggara, Kalimantan and Sulawesi), 17

local governments and 80 kelurahans.

The main findings of this report were as follows:

The vast majority of respondent stated that they received benefits and improved services

from the PNPM Urban. Improved road access was cited most often as a benefited of PNPM

Urban.

The majority of respondents who received microloans from KSMs indicated that the process

to acquire loans was relatively simple and that these loans improved household income. The

marginal impact of the PNPM loans on household incomes was perceived to be greater than

that from other loans.

Almost half of the respondents stated that gaining access to information about PNPM Urban

was easy. However, there was room for improvement in this area, as many respondents only

acquired information through informal channels and many did not know about subjects such

as poverty mapping and PAPG.

BKMs were generally perceived to be honest and trustworthy, and BKM elections were

generally perceived to be transparent, participatory and fair.

Trainings given to the RLF beneficiaries have increased the probability of improving the

households’ income by 0.4% (13.4% for RLF beneficiaries without training, to 13.8% for

RLF beneficiaries complemented by related trainings).

Of total 2,400 HH respondents, about 13% (318 HH) received social programs. Of the 318

HH, about 35.5% perceived the social program as very useful, and 61% as useful.

About 70% BKMs interviewed manage to continue providing social grants. About 24%

BKMs decided to discontinue social grants, because no budget allocated for social grant by

BKM (44%), not included in the priority (25%), no proposal received (13%), and other

reasons such as the long term benefit was somewhat unclear (only to buy sewing machine or

cake-making tools)

BKM meeting with social unit (UPS) is less frequent compared to other units (UPK and

UPL)

Process evaluation. Also in 2011, the Bank appointed the RAND Corporation to conduct a process

evaluation of PNPM Urban. For this evaluation, a sample of 350 beneficiaries in 16 kelurahans was

interviewed, as well as a selection of government officials and facilitators. The main findings of the

RAND process evaluation are summarized below.

Quality of infrastructure subprojects:

Subjective assessments of PNPM Urban project quality were almost universally favorable.

Almost all respondents – notably including some government officials – said that PNPM Urban

projects were of high quality, and often of better quality than similar government-constructed

projects.

44

Objective assessments, including professional engineering assessments, of PNPM Urban project

quality were also good.

Generally, respondents attributed the higher quality of PNPM Urban infrastructure to the

participatory approach and the avoidance of government management.

With respect to maintenance, some respondents said that the community was more likely to

maintain PNPM Urban projects than other government infrastructure due to the perception of

ownership of PNPM Urban projects.

Quality of non-infrastructure

A significant proportion of respondents mentioned that the poor need more social assistance

programs, such as access to jobs, loans and capital, training, and access to healthcare and

education, and affordable food. However, even respondents who expressed a desire for more

economy and social programs recognize that many of those projects may be difficult to

implement (than infrastructure project), and may not benefit as many people as in the

infrastructure projects.

There is demand for social and economic projects as well as infrastructure projects. Many

informants agreed that the current mix of infrastructure, economy and social projects was not

ideal. More emphasis should be placed on economy and social projects, or more balanced funds

allocated between infrastructure, economy and social programs. Preferences varied across and

within sites, but many stated the need of greater funds for trainings and loans.

Many are aware that the non-infrastructure projects have not worked well in the past, but the

findings raise the general issue that the low-income residents in the state of needing more

economy and social programs.

Women tend to be more participative in economy and social programs, and less in infrastructure

projects (mostly only contributes food and drink for workers).

The CDD approach:

Volunteer community organizations, including BKMs and KSMs, were generally thought to

work well, and to function independently of local government.

Community participation from the grassroots was not at ideal levels, but this must be viewed

against the scenario prior to PNPM Urban of almost zero participation. Decisions about project

selection and implementation tended to be made by delegated representatives of the community,

but respondents generally expressed satisfaction with the choices made about projects to pursue.

Participation was greater in project implementation. Respondents expressed the view that PNPM

Urban infrastructure was of good quality and that this was because the community was involved

in their implementation. In most communities, households reported making in-kind contributions

(e.g. labor or food/drinks for workers).

Targeting the poor via infrastructure project was a challenge because the urban poor tend to be

geographically spread out, even within kelurahans. This was not however a fault of the program.

Local government:

There was little or no duplication of local government activities i.e. PNPM Urban was providing

services that were not delivered by government.

The involvement of local government varied across sites, from little substantive involvement

(e.g. only signing off on projects) to substantial local government involvement (e.g. participating

in decision making and oversight). While in some sites, projects might have been aided by

improved communication between PNPM Urban and government, often both community

45

representatives and government officials were in favour of PNPM Urban being kept separate

from local government.

Capacity building and effectiveness of the facilitators:

Facilitators were playing a significant role in all of the cycles of PNPM Urban, from the

socialization phase to helping BKMs and KSMs in the writing of reports and auditing of projects.

Community members generally perceived that facilitators were significantly contributing to

project outcomes, often by providing specialized knowledge or identifying problems in the

implementation of infrastructure projects. However, it was found that facilitators could be doing

more to encourage community participation in decision-making.

In some cases, facilitators were a bit overstretched, having to supervise activities in multiple

kelurahans.

Governance and control mechanisms:

Monitoring and governance mechanisms were generally working well, despite the logistical

challenges of collecting information for a national program.

Auditing was being completed on schedule and that it was perceived to be useful for uncovering

instances of inadequate project performance, poor implementation, corruption, etc.

The PNPM Urban MIS had substantial potential, but there was room for improvement in the

selection of data collected and the validation of this data.

The adequacy of block grants and learning from ND:

The size of the block grants offered was generally adequate for project needs. However, this may

have been because project requests were already scaled to expectations about PNPM Urban

resources.

Even though it was deemed reasonable that pilot sites were determined on the basis of readiness

rather than need, the report suggested that ND should consider an alternative selection method for

future sites to enhance the program’s impact in alleviating poverty.

46

Annex 6. Stakeholder Workshop Report and Results

Not applicable.

47

Annex 7. Borrower's ICR and Comments on Draft ICR

(a) Borrower’s Completion Report

In June 2015, the Borrower submitted a 49-page project completion report to the Bank, titled “Project

Completion Report (PCR) PNPM Urban I & Additional Financing (IBRD-75040, IBRD-76640,

IDA-43840)”.The Executive Summary of the report is copied below.

1. In Mid-2007, in Palu at the centre of Sulawesi province, the President of Indonesia

announced the launch of a national program for poverty reduction. A pillar of this program

is the National Community Empowerment Program, or Program Nasional Pemberdayaan

Masyarakat Mandiri (PNPM Mandiri).

2. The PNPM Mandiri, aims to reduce poverty by empowering communities so that they will

be effective development players and partners through a comprehensive, integrated

community-based approach. The overall objectives of the project are to (i) improve the

socio-economic conditions of communities in the targeted areas, especially women and

the poorest, through (a) improving community infrastructure, both, physical and social;

(b) social activities aimed at the dependent/ vulnerable segments of the communities; (c)

facilitating economic growth and livelihood development, and (ii) build capacities & skills

of communities and their local institutions.

3. PNPM Urban I was carried out in the period of 2008, with the effective date of the project

is on July 22th, 2008 and the closing date (original) on March 31th 2011, with extent to

December 31th , 2014 (revised/actual date). The additional financing (AF) was PNPM Urban

which have been implemented in the period in 2009, with the effective date of the

project was on May 1st, 2009 and with the same closing dated. Both are financed with

funds from the World Bank, ie IBRD-75040, IBRD-76640, IDA-43840.

4. The original amount of the PNPM-Urban is US$177.68 million (IBRD loan US$52.68 million,

IDA credit SDR 78.6million/ US$125 million). With additional financing, provide an

additional loan in an amount of US$115 million to the PNPM Urban Fy 2009. The original

project is a Specific Investment Loan with an IDA/IBRD blend.The additional financing is

100% IBRD and will bring total financing up to US$292.68 million.

5. In order to close these project, the government of Indonesia (GOI) prepare a Project

Completion Report (PCR) PNPM Urban I and Additional Dinancing. The PCR of PNPM Urban

I and AF prepared to find out the achievements of the outcome and output results of

PNPM Urban III which are are identified and defined in Project Appraisal Documents (PAD)

of PNPM Urban I and AF.

6. According to data presented by the Deputy Secretary Vice President For People's Welfare

and Poverty Alleviation, as the Executive Secretary of the National Team for Accelerating

Poverty Reduction (TNP2K), the number of poor people in 2014 was 27,72 million people

or equivalent with the poverty rate reach 10.96%. For comparison, data on poverty in

Indonesia at the time of preparation of PNPM Urban I in 2008, the number of poor people

at that time was 34.96 million people with a poverty rate amounted to 15,42%.

48

7. Thereby, during the implementation of PNPM Urban I in the period of 2008 to 2014, a

general overview of the poverty in Indonesia was a reduction in the number of poor about

7,24 million people, and also decline in the poverty rate of 4,46 %. Specifically in urban

areas, it was a reduction in the number of poor people at 2,41 million or 3,49% from 2008

to 2014.

Table 1: Poverty in Indonesia Fy 2008-2014

Year

Poor People (million)

% Poverty

Urban Rural Total Urban Rural Total

2008 12.77 22.19 34.96 11.65 18.93 15.42

2009 11.91 20.62 32.53 10.72 17.35 14.15

2010 11,1 19.93 31.02 9.87 16.56 13.33

2011 10.95 18.94 29.89 9.09 15.59 12.36

2012 10.51 18.09 28.59 8.60 14.70 11.66

2013 10.63 17.92 28.55 8.52 14.42 11.47

2014 10.36 17.37 27.72 8.16 13.76 10.96

2014 - 2008 2.41 4.82 7.24 3.49 5.17 4.46

Picture of poverty reduction in Indonesia since 2003 to 2013. BPS data sources

8. PNPM Urban I and Additional Financing have been completed and finished on December

2014 and in overall, the project is rated satisfactory, with measured by objective and

outputs indicators results achievement.

9. Related with Achievements of Project Development Objectives (PDOs) of PNPM Urban I

and Additional Financing, all indicators have been achieved and meet the satisfactory

rating. In general almost all outputs indicators of PNPM Urban have succeeded to meet

indicators target results . From all indicators on the component of Community and Local

Government Capacity Building and Component 3: Technical Assistance are satisfactory,

while Component 2: Kelurahan Grants’ rating is moderate satisfactory, because t

indicators related with RLF haven not been achieved. It can be concluded that the

Kelurahan grant component has successfully disbursed and managed by community with

appropriately, efficiently and effectively, except for the revolving fund strategies that

need to be made more precise.

10. The Beneficiaries of PNPM urban 1 and Additional Financing reached 22 million

Households, including 14 million are poor households.

49

Table 1: Poverty in Indonesia Fy 2008-2014

Beneficiaries

IBRD 7504/IDA 4384 IBRD 7664

Household Poor HH % Household Poor HH %

13.666.214 8.341.572 61.04% 9.168.842 5.671.073 61.85%

Sources: MIS P2KP/PNPM Urban, April 2015

11. The success of a program, will be able to influence government policy. The Community-

based development approach in Indonesia, which introduced through PNPM Urban, has

now become the mainstream of development in Indonesia, with the integration to the

principles of community development to the Village Law No.6 Year 2014, such as Fund

Disbursement directly to Community, Community participation, Community Groups

approach, Community Organizing, Transparancy and Accountability, Gender Perspective,

and Community Self-Reliance

Figure 1: PNPM Urban and Additional Financing Achievement

50

12. As the final conclusion, the implementation of the PNPM Urban and Additional Financing

have resulted satisfactory without any significant risk for future program, due to some

positive impacts i.e.:

a. The community empowerment approaches have successfully become a platform

for several programs in Directorate General Human Settlement Ministry of

Public Work such as Pamsimas etc.

b. The community empowerment approaches have been adopted as national

policy since the PNPM/NCEP established.

c. Beside applied in poverty reduction program, the community empowerment

approaches have also been adopted in post-disaster rehabilitation and

reconstruction through Rekompak and other programs;

d. The PNPM Urban will be transformed as one of “The National Urban Slum Up-

grading based on CDD Program Fy 2015-2019”.

e. The succesfull of PNPM Urban I and Aditional Financing mainly due to the

support by closely coordination, cooperation, and intensive communication

between the executing agency with the World Bank, which at each stage of

PNPM Urban and Additional Financing implementation were always been a

subject to prior approval of both GOI and the World Bank.

51

(b) Borrower Comments on Draft ICR

55

Annex 8. Comments of Co-financiers No comments were received from the Islamic Development. (Note that IDB provided parallel

financing for PNPM in urban areas in 14 provinces not covered by the Bank and was therefore

technically not a co-financier of Bank-financed activities.)

56

Annex 9. List of Supporting Documents

Project Appraisal Document. World Bank. April 2008.

Aide Memoires (various). World Bank. 2008-2014.

Implementation Status Reports (various). World Bank. 2009-2014.

Loan agreements (various). World Bank. 2008-2009.

Project Paper on a Proposed Additional Financing Loan for the National Program for

Community Empowerment in Urban Areas. World Bank. March 2009.

Country Assistance Strategy for Indonesia 2004-2007. World Bank. August 2008.

Country Partnership Strategy for Indonesia 2013-2015. World Bank. December 2012.

Micro Credit Strategy Formulation Mission for the National Community Empowerment

Program”, Final Report, Yayasan “MICRA”, November 2008.

Consulting Services for A Study on Community-Based Infrastructure Development in PNPM

UPP, PT. Shiddiq Sarana Mulya, November 2009.

Contributions from Eight Thematic Studies to Qualitative Evaluation of the Indonesian Urban

Poverty Programme (UPP2/UPP3) 2009-2010. Betke and Ningsih. Departemen Pekerjaan

Umum, March 2011.

Process Evaluation of the Program Nasional Pemberdayaan Masyarakat (PNPM)-Urban.

RAND Corporation, August 2011.

Indonesia: Urban Poverty and Program Review. RAND Corporation, 2011.

Social Activities in Six Selected Cities, Lembaga Penelitian dan Pengabdian Kepada Masyarakat,

February 2010.

Female Participation in Eight Selected Cities, PT. Prospera Consulting Engineers, April 2010.

PNPM Gender Study 2012: Increasing the Quality of Women’s Participation – Final Report.

PNPM Support Facility. 2012.

Laporan Akhir Studi Skala Kecil Analisis Manfaat Ekonomi Proyek Infrastruktur PNPM Mandiri

Perdesaan. PNPM Support Facility. July 2012.

Technical Evaluation of Infrastructure – PNPM-Rural & Other Funding Sources. PNPM Support

Facility. July 2012.

Rapid Appraisal of Neighborhood Development (ND) and PAPG Mechanism. Schuler and

Dwiyani, 2012.

Gender Sensitive Community Driven Development and Disaster Risk Management in Indonesia.

Earth Systems, 2012.

Quantitative Evaluation the National Community Empowerment Program - Urban (NCEP-

Urban), Ifa Isfandiarni, Kementerian Pekerjaan Umum, 2012.

Consolidated Assessment of UPK Revolving Loan Funds in Indonesia. M-Cril, 2012.Peta Jalan

PNPM Mandiri – Menuju Keberlanjutan Program Pemberdayaan Masyarakat. Kelompok Kerja

Pengendali Program Penanggulangan Kemiskinan Berbasis Pemberdayaan Masyarakat (PNPM

Mandiri), Tim Nasional Percepatan Penanggulangan Kemiskinan (TNP2K). Jakarta, September

2012.

Studi Kelompok Masyarakat PNPM. Leni Dharmawan, et.al. TNP2K Working Paper 10a.

Jakarta, July 2014.

Project Completion Report (PCR) PNPM Urban I & Additional Financing (IBRD-75040, IBRD-

76640, IDA-43840), Ministry of Public Works, June 2015.