public disclosure authorized - world...

Document of The World Bank

Report No: ICR00001111

IMPLEMENTATION COMPLETION AND RESULTS REPORT (IBRD-47020/ IDA-37840)

ON

A LOAN IN THE AMOUNT OF US$ 31.1 MILLION

AND

A CREDIT IN THE AMOUNT OF SDR 54.00 MILLION

(US$ 80.53 MILLION EQUIVALENT)

TO

REPUBLIC OF INDONESIA

FOR

HEALTH WORK FORCE AND SERVICES PROJECT / PROVINCIAL HEALTH PROJECT III

June 27, 2009

Health, Nutrition and Population Human Development Sector Unit East Asia Pacific Region

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

CURRENCY EQUIVALENTS (Exchange Rate Effective June 16, 2009)

Currency Unit = Indonesian Rupiah (IDR) IDR 1,000.00 = US$0.0971 US$ 1.00 = IDR 10,300.00

FISCAL YEAR

January 1 – December 31

ABBREVIATIONS AND ACRONYMS

APBD Anggaran Pendapatan & Belanja Daerah (Regional Government Budget)

APBN Anggaran Pendapatan & Belanja Negara (Central Government Budget)

Askeskin Asuransi Kesehatan untuk orang Miskin (Health Insurance for the Poor)

Bappeda Badan Perencanaan Pembangunan Daerah (Regional Development Planning Agency)

BDEHHR Board for Development and Empowerment of Human Health Resources

CPCU Central Project Coordination Unit DCA Develpment Credit Agreement DFA District Funding Allocations DHC District Health Councils DIU District Implementation Unit IACC Inter-agency Coordinating

Committee ICR Implementation Completion ReportIDA International Development AgencyIDHS Indonesia Demographic Health

Survey IG Inspectorate General IMA Indonesian Medical Association IMR Infant Mortality Rate ISR Implementation Status Report

JHC Joint Health Councils M&E Monitoring and Evaluation MDG Millennium Development Goals MMR Maternal Mortality Ratio MOH Ministry of Health MONE Ministry of Education MTR Mid-Term Review Musrenbang Musyawarah Rencana &

Pembangunan (Development Plan Meeting)

PAD Project Appraisal Document PCIU Provincial Central Implementation

Unit PDO Project Development Objectives PHO & DHO Provincial and District Health

Offices PHP Provincial Health Project PPIU Provincial Project Implementation

Unit SIKD Sistem Informasi Keuangan Daerah

(Regional Financial Information System)

SUSENAS Survey Sosial Ekonomi Nasional (National Social Economic Survey)

TRT Technical Review Team U5MR Under-five Mortality Rate

Vice President: James W. Adams Country Director: Joachim von Amsberg Sector Manager: Juan Pablo Uribe Project Team Leader: Puti Marzoeki ICR Leader: Pandu Harimurti

INDONESIA

HEALTH WORK FORCE AND SERVICES PROJECT / PROVINCIAL HEALTH PROJECT III

CONTENTS

Data Sheet A. Basic Information B. Key Dates C. Ratings Summary D. Sector and Theme Codes E. Disbursement Profile

1. Project Context, Development Objectives and Design ..........................................................1 1.1 Context at Appraisal ............................................................................................................1 1.2 Original Project Development Objectives (PDO) and Key Indicators ...........................1 1.3 Revised PDO (as approved by original approving authority) and Key Indicators, and reasons/justification ...................................................................................................................3 1.4 Main Beneficiaries ...............................................................................................................3 1.5 Original Components (as approved) ..................................................................................3 1.6 Revised Components ............................................................................................................5 1.7 Other significant Changes ...................................................................................................5

2. Key Factors Affecting Implementation and Outcomes .........................................................5 2.1 Project Preparation, Design and Quality at Entry ...........................................................5 2.2 Implementation ....................................................................................................................7 2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization ...............8 2.4 Safeguard and Fiduciary Compliance ...............................................................................8 2.5 Post-completion Operation/Next Phase .............................................................................9

3. Assessment of Outcomes .........................................................................................................10 3.1 Relevance of Objectives, Design and Implementation ...................................................10 3.2 Achievement of Project Development Objectives ...........................................................10 3.3 Efficiency ............................................................................................................................18 3.4 Justification of Overall Outcome Rating .........................................................................18 3.5 Overarching Themes, Other Outcomes and Impacts .....................................................18 3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops ............19

4. Assessment of Risk to Development Outcomes ....................................................................19 5. Assessment of Bank and Borrower Performance ................................................................19

5.1 Bank Performance .............................................................................................................19 5.2 Borrower Performance ......................................................................................................20

6. Lessons Learned ......................................................................................................................21 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners ....................24

Annex 1. Project Costs and Financing .......................................................................................29 Annex 2. Outputs by Component ...............................................................................................30 Annex 3. Economic and Financial Analysis ...............................................................................34 Annex 4. Bank Lending and Implementation Support/Supervision Processes ......................35 Annex 5. Beneficiary Survey Results .........................................................................................37 Annex 6. Stakeholder Workshop Report and Results ..............................................................38 Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR ..............................39 Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders ................................40 Annex 9. List of Supporting Documents ....................................................................................41 MAP - IBRD 37030 ......................................................................................................................42

i

A. Basic Information

Country: Indonesia Project Name: Health Workforce & Services (PHP 3)

Project ID: P073772 L/C/TF Number(s): IBRD-47020,IDA-37840

ICR Date: 06/27/2009 ICR Type: Core ICR

Lending Instrument: SIL Borrower: INDONESIA

Original Total Commitment:

USD 105.6M Disbursed Amount: USD 55.7M

Environmental Category: B

Implementing Agencies: Ministry of Health

Cofinanciers and Other External Partners: B. Key Dates

Process Date Process Original Date Revised / Actual

Date(s)

Concept Review: 10/02/2001 Effectiveness: 12/30/2003

Appraisal: 04/03/2003 Restructuring(s):

Approval: 06/12/2003 Mid-term Review: 01/15/2007 01/15/2007

Closing: 12/31/2008 12/31/2008 C. Ratings Summary C.1 Performance Rating by ICR

Outcomes: Moderately Unsatisfactory

Risk to Development Outcome: Substantial

Bank Performance: Moderately Satisfactory

Borrower Performance: Moderately Unsatisfactory

C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings

Quality at Entry: Moderately Unsatisfactory

Government: Moderately Unsatisfactory

Quality of Supervision: Moderately SatisfactoryImplementing Agency/Agencies:

Moderately Unsatisfactory

Overall Bank Performance:

Moderately SatisfactoryOverall Borrower Performance:

Moderately Unsatisfactory

ii

C.3 Quality at Entry and Implementation Performance IndicatorsImplementation

Performance Indicators

QAG Assessments (if any)

Rating

Potential Problem Project at any time (Yes/No):

Yes Quality at Entry (QEA):

None

Problem Project at any time (Yes/No):

Yes Quality of Supervision (QSA):

Moderately Satisfactory

DO rating before Closing/Inactive status:

Moderately Satisfactory

D. Sector and Theme Codes

Original Actual

Sector Code (as % of total Bank financing)

Health 70 70

Sub-national government administration 15 15

Tertiary education 15 15

Theme Code (as % of total Bank financing)

Child health 25 25

Decentralization 25 25

Education for the knowledge economy 13 13

Health system performance 24 24

Vulnerability assessment and monitoring 13 13 E. Bank Staff

Positions At ICR At Approval

Vice President: James W. Adams Jemal-ud-din Kassum

Country Director: Joachim von Amsberg Andrew D. Steer

Sector Manager: Juan Pablo Uribe Fadia M. Saadah

Project Team Leader: Puti Marzoeki Enis Baris

ICR Team Leader: Pandu Harimurti

ICR Primary Author: Adrian Hayes F. Results Framework Analysis

Project Development Objectives (from Project Appraisal Document) Support health sector decentralization in four provinces for sustainable financing and client-centered delivery of health services. Specific Objectives: 1. Improve financing and delivery of essential health services in the provinces of Jambi, East Kalimantan, West

iii

Kalimantan, and West Sumatera to enhance access to care, quality of care and health outcomes at the district level. 2. Strengthen health workforce policy, management and development in a decentralized context in order to improve allocational efficiencies and equity in the distribution and use of health resources in the districts. 3. Empower MOH, MONE and IMA, the three key stakeholders in the sector. Revised Project Development Objectives (as approved by original approving authority) (a) PDO Indicator(s)

Indicator Baseline Value

Original Target Values (from

approval documents)

Formally Revised Target Values

Actual Value Achieved at

Completion or Target Years

Indicator 1 : Underfive Mortality rate to decrease to 35 per 1,000 live births to stay on course with the MDG target of 2/3 reduction by 2015 (West Sumatera/WS, Jambi/Jmb, West Kalimantan/WK, and East Kalimantan/EK)

Value quantitative or Qualitative)

West Sumatera 59, Jambi 51, West Kalimantan 63, East Kalimantan 50 per 1,000 live births (IDHS 2002-2003)

U5 Mortality rate to decrease to 35 per 1,000 live births

West Sumatera 62, Jambi 47, West Kalimantan 59, East Kalimantan 38 per 1,000 live births (IDHS 2007)

Date achieved 06/30/2001 10/01/2003 12/31/2007 Comments (incl. % achievement)

Indicator 2 : The proportion of births attended by a skilled health personnel to increase to 90% (West Sumatera/WS, Jambi/Jmb, West Kalimantan/WK, and East Kalimantan/EK)


WS 79.8%, Jmb 70.5%, WK 63.7%, EK 79.2% (IDHS 2002-2003)

Births attended by a skilled health personnel to increase to 90%

WS 80.5%, Jmb 69.8%, WK 62.2%, EK 75.5% (IDHS 2007)


Indicator 3 : The proportion of pregnant women receiving at least four ante natal visits to increase to 90% (West Sumatera/WS, Jambi/Jmb, West Kalimantan/WK, and East Kalimantan/EK)


WS 79.8%, Jmb 78.5%, WK 65.7%, EK 80.4%

The proportion of pregnant women receiving at least four ante natal

WS 84.8%, Jmb 82.4%, WK 78.3%, EK 80.9%

iv

(DG of Community Health, MOH, 2003)

visits to increase to 90%

(DG of Community Health, MOH, 2008)


Indicator 4 : The segment of the population belonging to the lowest income quintile to have increased their service utilization by 50% (West Sumatera/WS, Jambi/Jmb, West Kalimantan/WK, and East Kalimantan/EK)


The utilization of outpatient services by the lowest income quintile WS 36.48%, Jmb 27.34%, WK 19.67%, EK 26.63% (Susenas 2003)

50 % increase of utilization of outpatient services by the poor (all ages) by the end of the project

The utilization of outpatient services by the lowest income quintile WS 38.18%, Jmb 32.66%, WK 33.03%, EK 37.35% (Susenas 2007)


Indicator 5 :

The share of local government health spending to reach 15% of the total local government spending, except in East Kalimantan where it is to reach 8% or at least Rp. 51,000 percapita in Year 2002 prices (West Sumatera/WS, Jambi/Jmb, West Kalimantan/WK, an


Share of Health to Local Budget : WS 7.17%, Jmb 6.45%, WK 6.68%, EK 4.28% (MOF Local Government Database 2003) Per capita Spending (in 2002 prices): WS Rp 29,235 Jmb Rp 18,803, WK Rp 18,803, EK Rp 50,436

The share of local government health spending to reach 15% of the total local government spending, except in East Kalimantan where it is to reach 8% or at least Rp. 51,000 percapita in Year 2002 prices

Share of Health to Local Budget : WS 5.95%, Jmb 7.68%, WK 6.87%, EK 4.250% (MOF Local Government Database 2007) Per capita Spending: WS Rp 29,235, Jmb Rp 18,803, WK Rp 18,803, EK Rp 50,436


Indicator 6 : IMA to have established an institutional structure and instruments for a national medical examination and certification, for formal, graduate, and continuing medical education

v


IMA institutional structure needs improvement. National medical examinations, recertification and internship did not exist

IMA to have established an institutional structure and instruments for a national medical examination and certification, for formal, graduate, and continuing medical education

Guidelines for IMA institutional structure completed, National Medical. Examination. System and Certification is available, Recertification still in development, Internship program still in development


Indicator 7 : The portion of medical students in Mulawarman and Andalas Universities surveyed as satisfied with the curriculum and education provided is at least 80%


No Data

Medical students in Mulawarman and Andalas Universities surveyed as satisfied with the curriculum and education provided is at least 80%

Students satisfied with: - facilitators 67.5% - lecture rooms 76.5% - library 63% - books and journal 37% - skills laboratories 74% - Lab. equipment 54% (MONE Survey)


(b) Intermediate Outcome Indicator(s)

Indicator Baseline Value

Original Target Values (from

approval documents)

Formally Revised

Target Values

Actual Value Achieved at

Completion or Target Years

Indicator 1 : Implementation of the District funding Allocations (DFAs)

Value (quantitative or Qualitative)

No DFA was available prior to the project

Five rounds of DFAs were fully completed in the 4 provinces

Five rounds of DFA were completed by the end of project implementation period

vi


Although the target to have 5 rounds of DFA was completed, but with limited implementation with only around 25 months from expected 60 months of project implementation

G. Ratings of Project Performance in ISRs

No. Date ISR Archived

DO IP Actual

Disbursements (USD millions)

1 06/17/2003 Satisfactory Satisfactory 0.00 2 12/04/2003 Satisfactory Satisfactory 0.00 3 06/17/2004 Satisfactory Satisfactory 4.31 4 07/29/2004 Satisfactory Satisfactory 4.31 5 12/22/2004 Satisfactory Satisfactory 4.31 6 05/18/2005 Satisfactory Satisfactory 13.03 7 12/19/2005 Moderately Satisfactory Satisfactory 13.63 8 12/22/2006 Moderately Satisfactory Moderately Satisfactory 31.50 9 05/14/2007 Moderately Satisfactory Moderately Satisfactory 38.00

10 02/13/2008 Moderately Satisfactory Moderately Satisfactory 47.81 11 09/24/2008 Moderately Satisfactory Unsatisfactory 55.13

12 01/09/2009 Moderately

Unsatisfactory Moderately

Unsatisfactory 55.13

H. Restructuring (if any) Not Applicable

vii

I. Disbursement Profile

1

1. Project Context, Development Objectives and Design 1.1 Context at Appraisal The Health Work Services Project / Provincial Health Project III (known also as HWS Project or PHP III) became effective 31 December 2003, during a period of rapid political and administrative reform in Indonesia following the Asian Financial Crisis, the downfall of President Suharto, and the Reformasi movement. At the province and district levels PHP III focused on West Sumatera, Jambi, West Kalimantan and East Kalimantan. PHP I (Lampung and Yogyakarta) became effective in August 2000 and PHP II (North Sumatera, West Java and Banten) in April 2002. All three projects aimed at helping the government implement “effective decentralization” of the health sector. Sector-related CAS goals for PHP III were to (i) promote broad-based growth and income generation; (ii) support decentralization of public sector services and the involvement of the private sector in health care delivery; and (iii) contribute to improved health status of the population. 1.2 Original Project Development Objectives (PDO) and Key Indicators The PDO as stated in the PAD (dated 19 May 2003) was to support health sector decentralization in four provinces for sustainable financing and client-centered delivery of health services. More specifically, the main objectives were to:

1. Improve financing and delivery of essential health services in the provinces of Jambi, East Kalimantan, West Kalimantan, and West Sumatera to enhance access to care, quality of care and health outcomes at the district level.

2. Strengthen health workforce policy, management and development in a

decentralized context in order to improve allocative efficiencies and equity in the distribution and use of health resources in the districts.

3. Empower the Ministry of Health (MOH), the Ministry of National Education

(MONE), and the Indonesian Medical Association (IMA), the three key stakeholders in the sector, through assistance to redefine their roles and responsibilities vis-à-vis health work force policy, planning and management, and through building their institutional capacity for effective stewardship in fulfilling the functions of policy making, legislation, regulation, quality assurance/control and technical assistance to provinces and districts.

The PAD listed two sets of performance indicators. (1) Non-project-specific indicators included:

a. Health outcome indicators of Infant Mortality Rate (IMR), Under-five Mortality Rate (U5MR), Maternal Mortality Ratio (MMR), and other locally relevant disease-specific indicators (Tuberculosis, Malaria, HIV/AIDS prevalence rates).

2

b. Tracer health services indicators, including proportion of pregnant women receiving at least four ante-natal visits, proportion of births attended by skilled health personnel, and immunization coverage rate.

c. Health service utilization indicators, including outpatient visit rate, admission rate, average length of stay, and bed occupancy ratio.

(2) Project-specific indicators were largely institutional and organizational in nature, and included:

a. Preparation and use of a Health Development Master Plan for all capital investment in human, financial and physical resources.

b. Design and implementation of new provincial and district level institutional mechanisms (District Health Councils (DHC), Joint Health Councils (JHC), and Technical Review Teams (TRT)) for increased pluralism and client involvement in policy making, planning and system management.

c. Design and implementation of financial mechanisms at the district level (block funding) for increased efficiency in resource allocation.

d. Proportion of districts reaching per capita health expenditures threshold of Rp 51,000 in 2002 prices, or 15 percent of the total local government spending, except in East Kalimantan where it is to reach 8 percent, excluding the resources allocated for civil works and equipment.1

e. Increased provider and client satisfaction, as measured by surveys. f. Development and implementation of new standards for licensing, certification,

and registration of health professionals and accreditation of health facilities. g. Adoption and implementation of new medical school admission criteria, curricula,

and licensing criteria. h. Adoption of policies and financial and non-financial incentives by Provincial and

District Health Offices (PHO and DHO) to increase productivity, performance and motivation of health professionals.

i. Redefinition of MOH’s roles and responsibilities vis-à-vis health workforce policy, planning and management.

j. Strengthening of MOH’s institutional capacity through structural changes in organigramme, and increased allocation of human and financial resources for effective stewardship in fulfilling the functions of policy making, legislation, regulation, quality assurance/control and technical assistance to provinces and districts.

The Loan Agreement and Development Credit Agreement (signed 1 October 2003) established the following performance indicators:

(1) By 30 June 2008, in each of the project provinces:

a. Under-five mortality rate to decrease to 34 per 1,000 live births so as to stay on course with the Millennium Development Goal target of 2/3 reduction by 2015.

b. The proportion of births attended by skilled health personnel to increase to 90 percent.

1 This PI is not clear in the PAD but it is clarified in the DCA.

3

c. The proportion of pregnant women receiving at least four ante-natal visits to increase to 90 percent.

d. The lowest income quintile of population to have increased their service utilization by 50 percent, measured as the number of out-patient visits per capita relative to baseline data.

e. The share of local government health spending to reach 15 percent of total local government spending, except in East Kalimantan where it is to reach 8 percent, or at least Rp. 51,000 per capita in 2002 prices.

(2) By 30 June 2008, at the national level:

a. IMA to have established an institutional structure and instruments for a national medical examination and certification, for formal, graduate and continuing medical education.

b. The proportion of medical students in Mulawarman and Andalas Universities surveyed as satisfied with the curriculum and education provided to be at least 80 percent.

1.3 Revised PDO (as approved by original approving authority) and Key Indicators, and reasons/justification The PDO were not revised. The Midterm Review (MTR) noted the need to simplify the project where possible and better focus some of its activities; this resulted in more emphasis on improving maternal and child health status among the project’s activities. The re-focusing of project activities was done so that the District Grant components would primarily refer to the PDO indicators in the credit agreement. The justification was to help districts focus as the MTR found too much fragmentation which in part was due to lack of focus during project design with regards to the District Grants which were supposed to be used for many reforms. 1.4 Main Beneficiaries The main beneficiaries were identified in the PAD as some 13.7 million residents in the 4 provinces, including 3 million poor. The poor were expected to benefit from the Project through improved access to better quality health services. The project would also benefit current and future health care providers, medical students and faculties, and flow-on effects would benefit the whole sector. 1.5 Original Components (as approved) The project had 3 main components focusing respectively on the district, province and central levels of administration (Table 1). The design built on the previous PHP I and PHP II projects while adding a new component on human resources development.

4

Table 1: Project Components Component A District Health Offices (DHO) and Health Facilities

A1 Improved access to and quality of health services A2 Project management

Component B Provincial Health Offices (PHO) and Health Facilities B1 Health workforce development B2 Health system coordination, planning and management B3 Project management

Component C Central Level C1 Effective health system stewardship (MOH) C2 Enhancing the quality of medical education (MONE)

Component A – District Health Offices and Health Facilities (US$ 374.67 million; Loan/Credit amount: US$ 74.5 million) Component A1. Improved access to and quality of health services Bank support would be based on (a) health development master plan and annual health plan developed by district, endorsed by DHC, approved by JHC; and (b) use of block grants of District Funding Allocation (DFA) for increased efficiency in resource allocation and utilization. In addition districts would be expected to participate in a pilot program (under Component C) to train health care specialists, to improve the quality of hospital-based care; and West Sumatera and West Kalimantan districts would be expected to pilot a family-physician model of service delivery (also under Component C). Component A2. Project management This component would be the responsibility of the District Implementation Unit (DIU), supervised by the District Bappeda. Component B – Provincial Health Offices and Health Facilities (US$ 28.96 million; Loan/Credit amount: US$ 9.07 million) Component B1. Health workforce development Health workforce planning and management to be supported by (a) adapting the minimum service standards to local conditions; (b) determining the numbers, skills, and specializations of existing and future health workforces needed at provincial and district levels; and (c) implementing measures to ensure appropriate numbers of adequately trained staff in the health facilities. In addition the project would support studies, pilot tests, and operational research concerning incentives, motivation, performance appraisals, etc., for improving workforce performance. Component B2. Health system coordination, planning and management The PHO’s role was to strengthen the performance of districts in the following areas: (a) workforce development; (b) guidelines for health information and education campaigns; (c) epidemiological monitoring and response; (d) cross-sectoral cooperation; and (e) health system financing.

5

Component B3. Project management The project would support a Provincial Project Implementation Unit (PPIU). Component C – Central Level (US$ 25.74 million; Loan/Credit amount: US$ 22.03 million) Component C1. Effective health system stewardship (MOH) The project would support the Board for Development and Empowerment of Human Health Resources (BDEHHR), including its different centers (for Health Human Resources Management, Health Manpower Education, Health Training, and Empowerment of Health Profession and International Workforce). MOH’s capacity strengthened to (a) fulfill its human resource development responsibilities; (b) improve the quality of pre-service education programs; (c) ensure the quality of in-service training provided at the district level; and (d) promote self-regulation of the nursing and midwifery professions. The project would also support IMA through MOH. Component C2. Enhancing the quality of medical education (MONE) The project would support MONE to: (a) increase its institutional capacity to organize and manage medical education; (b) improve the quality of formal medical education; and (c) enhance the learning and teaching environment for both undergraduate and post-graduate medical education and training. 1.6 Revised Components The components of the project were not revised. 1.7 Other significant Changes No other significant changes took place. However, by 3 June 2008 the project had only disbursed 46.6 percent of the original project allocation. An extension on was entertained by both the Bank and the GOI, but the Government revised its decision late in 2008 and the project closed on 31 December as originally planned. In December 2008, The Government requested cancelation of Project funds amounted USD 48.9 million (Loan 30.8, and Credit 18.1) which was later approved by the Bank. 2. Key Factors Affecting Implementation and Outcomes 2.1 Project Preparation, Design and Quality at Entry Context A key factor which subsequently affected implementation and outcomes was that project preparation was completed at a time when the Government’s own policies and mechanisms regarding decentralization were still very much “a work in progress.” The

6

1999 Laws on Local Autonomy (which came into effect 1 January 2001)2 laid out the broad framework but many details were unclear and soon became the subject of intense discussion and negotiation. The Bank’s background analysis of pre-decentralization problems in the health sector was extensive, and the Bank also developed a deep appreciation of the possible risks and opportunities for the sector posed by decentralization (based on its international experience and consultation with Indonesian officials and experts), but no one could predict with confidence exactly how the definition and practical allocation of responsibilities for health across the main levels of government would evolve. Central level agencies were often worried about losing authority while many districts were keen to test the limits to their new powers. Much discussion in the provinces during early project preparation focused on lending conditions and the respective roles central line ministries, especially MOF, would play in project implementation; provinces and districts were interested in being involved in the project conditional on what they perceived as a satisfactory resolution of these issues; in the last analysis such resolution would depend more on the evolution of national policy than on any agreements specific to the project. The result was a project design that tried to accommodate the interests of a range of organizational and political interests at the expense of logical consistency and overall cohesion. The project documents have been aptly described as pot-pourri. Limitations A review of the PAD reveals a number of weaknesses. First, the PDO are confusing in the way each is expressed as a means to an end. “Improve financing and delivery of essential health services … to enhance access to care, quality of care and health outcomes.” Is the main objective here to improve financing or to enhance access to care? Is the relationship between the former and the latter supposed to be empirical (if so, where is the evidence?), or is the latter supposed to be a set of restrictive criteria which project management should apply to the former (i.e. limit project efforts to improving financing to those which are specifically related to enhanced access to care, etc.). The PDO are not clear. The PAD asserts, “The PHP model has proven to be viable and is readily embraced by central, provincial and district authorities alike,” but gives little consideration to whether this model is as well-suited to the project’s workforce goals as it is to the goals of protecting and improving health services. In the view of the ICR, the workforce part of PHP III, involving two ministries and the IMA at the central level, as well as provinces and districts, fits rather uncomfortably as an overlay on the PHP structure adopted from PHP I and II. During preparation MOH asked that the project be kept simple but PHP III is actually far more complex in terms of the range of its objectives and the number of organization types involved than its forebears.3

2 Law 22 of 1999 on Regional Autonomy and Law 25/1999 on Financial Balance between Central and Regional Government. 3 Given this complexity a diagram showing explicitly which subcomponents relate to which PDO would have been helpful. This design issue may have been clear in the minds of those who designed the project but it is not immediately clear in the PAD. Without making these relationships explicit it is hard for

7

Third, although the PAD included long lists of “key performance indicators” and “data collection strategies” there were few clear guidelines on how the project was to be monitored and evaluated, except for its financial aspects. It is understandable in the case of a complex project facing many uncertainties that the definition of objective criteria for assessing substantive progress towards project objectives and outputs should be left for the implementation phase, but the PAD did not make it clear who would be responsible for this, or when and how they would do it. These limitations in quality at entry disclose a situation where the key government units involved in the project (and the IMA) were able to present their varied interests and plans in their respective PIPs during project preparation but there was insufficient time for the Bank to fully harmonize all these inputs in the PAD (and fill in the remaining gaps in planning). 2.2 Implementation A number of factors in the institutional context within which the project operated at various times helped and/or hindered implementation. National policies defining and regulating decentralization continued to evolve during the course of the project. Most significant were the new decentralization laws passed in 2004 (Laws 32 and 33) to supersede those of 1999. The new laws created a firmer legal and regulatory framework for decentralization, but in the short term they produced more confusion since the associated government regulations stipulating how the new laws should be implemented were not released until 2007 (PP 38 and 41). On the positive side the 2004 laws established clearer responsibilities and authorities for the province level of government, very much in line with the philosophy enshrined in all three PHP. The most difficult obstacle frustrating implementation was the perennial lateness in the central government’s disbursement of project funds to provinces and districts. The underlying reasons for this are complicated.4 The MOF agreed in 2003 that HWS would be on-granted to districts, but it changed the channeling mechanisms it used to implement this agreement twice in 5 years. Project management and the Inter-agency Coordinating Committee (IACC) were proactive in seeking positive changes to these mechanisms for the benefit of HWS but on those occasions where MOF agreed to changes they were subsequently negated by further revisions to national financial policy. During the first 3 years of the project “actual implementation [was] only about 18 months,” according to the Mid-Term Review (MTR), resulting in disbursement of only around 38 percent after 36 months compared to an original projection of 71 percent.5 2007 was a relatively good

implementers of a given component to keep focused on the desired substantive results rather than simply on an arbitrary set of PIs. 4 See Supartini Hanafi, 2008. “Implementation Process of HWS Project 2004-2008: Supplementary Document for Final Evaluation.” Pp. 7-9. Jakarta: HWS. 5 There were other factors too contributing to the low disbursement rate, as the MTR noted (“Lack of experience, particularly among the districts, in applying Bank Guidelines on procurement and consultant selection, combined with heavy prior review requirements resulted in further delays”), but the major issue

8

year, but in 2008 disbursement of project funds was again problematic with three provinces receiving their money in September and one in October. By the end of the project actual project implementation at the province and district levels was only 25 months (in 5 years). Frequent turnover of officials, including project staff, was also cited by project staff as a hindrance to efficient implementation. At the district level exceptionally high turnover was often due to the local bureaucracy “settling down” during the first couple of years after decentralization. 2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization As indicated earlier the PAD was not specific on how data of sufficient quality would be collected for monitoring and evaluation purposes. The MTR noted, “Project M and E is weak due to several factors, however one of the most important … seems to be inadequate attention to the M and E design during project preparation. … One of the conclusions of the MTR mission is the need to clarify indicators, and to strengthen substantially the monitoring capacity of the project at all levels so that information is timely and adjustments can be made if required.” A task team for strengthening M&E was established by the Bank and an international consultant prepared a report (completed in November 2007) proposing “the use of performance measures which incorporate ‘milestones’ in such a way that, even if the data are collected late in the project’s life, a ‘historical record’ of the project can be recreated and the performance of the project components reviewed across the years 2004, 2006 and 2008 (corresponding to the dates for baseline, mid-term, and final reviews).” The project only had one year left to run, so changes in routine monitoring could hardly be used for routine management purposes; the consultant’s report “therefore recommended that most of the data needed to measure the monitoring PIs [performance indicators] be collected at the time of the final review, and responsibility for this activity be ‘folded into’ that for conducting the final evaluation.” This approach was adopted (at least to some extent) by the project in its own final review of HWS,6 but it was not formally adopted as a revision to the logframe. 2.4 Safeguard and Fiduciary Compliance Following confirmation of fraudulent practices in PHP II in 2005, the Bank conducted a special mission and proposed an action plan to strengthen financial control of all three PHP. The plan included (i) strengthening controls by moving financial and procurement management control from the Bureau of Planning to relevant units within MOH, (ii) recruiting a firm to conduct internal audits of the projects while working with the IG’s office to strengthen internal control within MOH, and (iii) appointing BPK to replace BPKP as external auditors.

which project management was powerless to control was, at least until 2007, the late release of funds by the MOF. 6 CPCU HWS, 2008. Implementation Status Results and Report HWS Project 2004-2008. Jakarta: MOH.

9

The HWS procurement post review findings for FY 2006 and FY 2007 included signs of collusion and other irregularities regarding procurement procedures, especially at the district level. The MTR rated implementation of procurement activities unsatisfactory. This could be ascribed to lack of familiarity with the Bank’s standards and procedures at the district level,7 and it was also not always easy to harmonize the Bank’s procedures with the Government’s own regulations. The Bank worked with the MOH IG and the CPCU to address this issue, and the ex-post review report for FY 2008 found the signs of collusion had decreased sharply. However the Bank’s Financial Management Review, conducted in December 2008, still concluded that in general “progress has been less than satisfactory as there are many outstanding items, such as no status of follow up action of audit findings, excessive outstanding of unclaimed expenditures and DFA balance, which should be resolved before the project closes. The progress of technical assistance in MOH IG has also been slow.” A review of procurement management conducted at the same time with the ICR Mission, to assess interventions to address procurement issues during the Project. The following were the key findings: a. There have been some improvements in the quality of procurement documents in the

last two years of the Project as the CPCU was more actively involved in preparing annual plans and procurement plans with implementing units. The CPCU and the Bank have organized several workshops and training sessions for capacity building purposes.

b. The Bank and the MOH have agreed on actions to follow up on the findings of the post review carried out in FY 2006 and FY 2008 which indicated a pattern of collusion among bidders for packages using shopping at the district level. These actions included technical assistance for the Inspectorate General (IG) and a procurement consultant for the CPCU, dissemination of training materials, and sanctions to firms involved in collusive practices.

c. Ex post review in FY 2008 indicated a sharp decrease in the number of suspected cases for collusion. While a wider sampling would be needed to assess the effectiveness of the above measures, it appears that the increased attention from the Bank, IG and CPCU may have helped in raising the awareness and sending the message to interested bidders.

Despite these shortcomings many officials expressed the view that becoming familiar with the Bank’s principles regarding RFPs, procurement, and financial transparency served to strengthen good governance in many districts. 2.5 Post-completion Operation/Next Phase Provinces and many districts expressed a commitment to maintain at least some of the institutional innovations introduced by HWS (DHCs, TRTs, use of HRH master plans to improve efficiency on a province-wide basis, new licensing and certification arrangements for health professionals, etc.). While adequate funding is not assured in all

7 Appropriate training in these principles and procedures was provided but with rapid turnover of staff at the district level this was not always sufficient.

10

districts, HWS has contributed to the capacity of DHOs to develop coherent evidence-based plans and to use these to advocate more effectively to district heads and DPRD. 3. Assessment of Outcomes 3.1 Relevance of Objectives, Design and Implementation The three PDOs were of high overall relevance to the country’s development priorities at the time of inception and they remain so today; they are also still highly relevant to the Bank’s country and sector assistance strategies. 3.2 Achievement of Project Development Objectives The ICR assessed the extent to which the project achieved each of the three PDOs in turn, especially in terms of the PIs listed in the PAD and DCA.8 Data from a number of sources was used, including the Final Evaluation of HWS conducted for the GOI by independent consultants9 (henceforth designated Independent Evaluation); the ISR conducted by the CPCU at MOH10 (MOH ISR); and the Aide Memoire to the Bank’s final review of HWS11 (WB Final Review). None of these reports focuses explicitly on the level of achievement of each PDO, however. The Independent Evaluation introduces “revised key performance indicators” without making it sufficiently clear where these came from,12 the MOH ISR uses a variety of indicators and the Bank’s FR relies on the 7 indicators from the DCA. (None of the three reflects on whether the PIs it uses are sufficient to produce a fair and objective assessment of project outcomes.) First PDO/District Level: Improve financing and delivery of essential health services in the provinces of Jambi, East Kalimantan, West Kalimantan, and West Sumatera to enhance access to care, quality of care and health outcomes at the district level. The PAD envisioned that support for improved service delivery at the district level would be based on a health development master plan and an annual health plan, both developed by the district with support from the project. During project preparation it was clear districts found it extremely difficult to prepare satisfactory plans and the Bank fielded a consultancy in June 2003 to assess the situation. “The overall finding was that the current district proposals were weak particularly in linking proposed activities to the situational assessment. Regardless of the health needs presented, the districts tended to split the

8 Strictly speaking HWS (as described in the PAD) had only one PDO, specified in terms of three “main objectives.” The exposition seems clearer if each main objective is treated as a separate PDO, and this is consistent with the design of PHP I and II as well. 9 PT Bahana Mitra Buana and PT Emendra, 2008, Final Evaluation for the Health Workforce and Services Project (Integrated Final Report). 10 MOH, 2008, Implementation Status Results and Report: HWS Project 2004-2008. 11 WB HWS Project Review Mission 1-15 December 2008, “Aide Memoire.” 12 Or why they were introduced; the main stimulus appears to have been the November 2007 consultant’s report on M and E.

11

budget between existing programs. … It was clear that the districts had limited capacity and that significant technical assistance would be required to strengthen the district proposals.” Over the course of the project districts became far more skilled in producing health plans (with associated detailed budgets) of appropriate focus and quality. To have helped districts develop this capacity is regarded by many connected with the project as one of its major achievements. This capacity certainly needs to be strengthened further. Despite of the progress made so far the activity failed to address the stated objectives, hence this outcome is assessed as moderately unsatisfactory. PI-2: Experience with the provincial- and district- level institutional arrangements introduced by the project (the DHCs, JHCs, and TRTs) was mixed. The JHCs evolved into province-wide fora for discussion of health policy issues and less into policy development institutions (as originally envisioned). DHCs worked well in some districts and not in others. The TRTs were generally a success and the provinces said they will endeavor to keep them beyond the life of the project. Performance is considered moderately satisfactory on this indicator.

Table 2. Achievement of key PIs for the First PDO, by province

Key PI West Sumatera

Jambi West Kalimantan

East Kalimantan

1. Preparation and use of a Health Development Master Plan for all capital investment in human, financial and physical resources (PAD)

MU

MU

MU

MU

2. Design and implementation of new provincial and district level institutional mechanisms (DHCs, JHCs, TRTs) for increased pluralism, particularly with respect to NGO and other civil society organizations, and client improvement in policy making, planning and system management (PAD)

MS

MS

MS

MS

3. Design and implementation of financial mechanisms (block funding) for increased efficiency in resource allocation (PAD)

U

U

U

U

4. The share of local government health spending to reach 15 percent of total local government spending, except in E. Kalimantan where it is to reach 8 percent, or at least Rp 51,000 per capita in 2002 prices (PAD & DCA)

MU

MU

MU

MS

5. Under-5 mortality rate to decrease to 34 per 1,000 live births so as to stay on course with Millennium Development Goals (PAD & DCA)

n.r

n.r

n.r

n.r

12

6. The proportion of births attended by skilled health personnel to increase to 90 percent (PAD & DCA)

MU

MU

MU

MU

7. The proportion of women receiving at least 4 ante-natal visits to increase to 90 percent (PAD & DCA)

MU

MU

MU

MU

8. The lowest income quintile of population to have increased their service utilization by 50 percent, measured as the number of out-patient visits per capita relative to baseline data (PAD & DCA)

MU

MU

S

MS

PI-3: HWS demonstrated that block grants to districts can, under the right conditions, be an effective instrument for improving efficiency in resource allocation. However many districts tended to use the additional funds for mostly routine activities, and project management, despite its best efforts, was never able to fully counteract this tendency and ensure the DFA were used for improvements in financing. Moreover the on-granting mechanisms used by the MOF meant the funds always arrived far too late for them to be used with maximum efficiency and effectiveness. Performance for the design and implementation of financial mechanisms is considered unsatisfactory. PI-4: None of the 3 provinces achieved the target of 15 percent of APBD devoted to health, nor did East Kalimantan achieve its target of 8 percent (Table 3). The Rp 51,000 per capita target appears to have been reached in East Kalimantan.13 Performance on this indicator is moderately satisfactory for East Kalimantan and moderately unsatisfactory for the other 3 provinces.

Table 3. Local health spending, by province, 2003, 2005 and 2007 Share of local APBD spent on health

Percent APBD per capita spending on health

Rp 2003 2005 2007 2003 2005 2007 W Sum 7.2 7.7 5.6 29,235 36,186 49,442 Jambi 6.5 7.7 7.7 18,097 25,669 33,580 W Kal 6.7 8.5 6.9 18,803 31,166 42,662 E Kal 4.3 3.8 4.5 50,436 36,186 49,442 Source: Data reported in WB Final Review, sourced from SIKD-MOF (Regional Financial Information System Ministry of Finance) and Bureau of Planning-MOH. Note: 2007 data report allocation data only, not actual spending. PI-5: Reliable data on the under-5 mortality rate (U5MR) by province for 2007 or 2008 are not available. The 2007 Indonesia Demographic and Health Survey (IDHS) estimates of U5MR by province are averages for the previous 10-year period. (The All Indonesia

13 The latest available data are for 2007; moreover the data used here for 2006 and 2007 are for health allocation, not actual spending.

13

estimate is the average for the previous 5 years.) The mid-point for the reference period reported in the 2007 survey therefore actually coincides with the beginning of HWS, not its closure.14 Comparing 2007 with 2002/03 data suggests the U5MR in East Kalimantan has probably declined to the target of 34 deaths per 1,000 live births by 2008 (regardless of any improvements in child mortality brought about by HWS interventions). Indirect estimation techniques applied to 2010 population census data when available may be able to show whether there has been an acceleration in the decline of child mortality in the HWS provinces. In the absence of data and clear causality between implemented actions and observable indicators the ICR was not able to assess this PI.

Table 4. Under-5 mortality rate, by province, 2002/03 and 2007 IDHS data Region 2002/03 data

(1994-2003) 2007 data

(1998-2007) West Sumatera 59 62 Jambi 51 47 West Kalimantan 63 59 East Kalimantan 50 38 (1998-2002) (2003-2007) All Indonesia 58 44

Source: IDHS. PI-6: Using IDHS data the WB Final Review found no significant improvement in the proportion of births attended by skilled health personnel in the HWS provinces, and no province was within 10 percentage points of the target of 90 percent by 2007 (Table 5). The performance rating is moderately unsatisfactory. PI-7: By 2007 there was some modest improvement in 3 provinces in the proportion of women receiving at least 4 ante-natal visits, according to MOH data,15 but none had reached the 90 percent target. Performance rating is moderately unsatisfactory for all 4 provinces. PI-8: According to the WB Final Review, a comparison of Susenas data for 2003 and 2007 shows that (i) utilization of health facilities by the lowest income quintile has increased in the HWS provinces, but (ii) only one (West Kalimantan) has achieved the targeted 50 percent increase. The Bank report also notes this increase in utilization follows a national trend and undoubtedly has more to do with the introduction of the national Askeskin program (free services for the poor) than with HWS interventions. The ICR rates performance satisfactory in West Kalimantan, moderately satisfactory in East Kalimantan, and moderately unsatisfactory in West Sumatera and Jambi. The overall assessment of the extent to which the project achieved the first PDO is moderately unsatisfactory.

14 The WB Final Review treats these data as if the reference date is the same as the survey data. 15 Cited in WB Final Review. The ICR was not able to confirm the reliability of these data.

14

Second PDO/Province Level: Strengthen health work force policy, management and development in a decentralized context in order to improve allocational efficiencies and equity in the distribution and use of health resources in the districts. The project was designed (according to the PAD) to “support the PHO’s new responsibilities under decentralization to provide technical and administrative support to the Districts for: (a) strengthening health workforce management and training capacities; and (b) improving the coordination, planning, and management of health sector resources.” Only three of the key PIs listed in section 1.2 above pertain directly to the second PDO (see Table 5) so this ICR also used several associated output indicators (taken from the Project Design Summary in the PAD) to substantiate its assessment.16 PI-9: No surveys were conducted to collect data on provider and/or client satisfaction. Performance on this PI was not rated (n.r.). PI-10: Competency-based standards for doctors, dentists and midwives were established at the national level during the course of HWS and each of the four provinces made considerable progress in implementing new standards for licensing, certification, and registration of health professionals in ways reflecting the new emphasis on demonstrated competence. The necessary decrees and regulations were already enacted in East Kalimantan by December 2008 and were at various stages of preparation in the other 3 provinces.17 All 4 provinces developed an HRH Master Plan. The project also supported fellowships and training for health workforce development, including the improvement of in-service training at the province and district levels. Developments in accreditation of health facilities were slight by comparison. Overall rating on this PI is satisfactory. Table 5. Achievement of key PIs for the Second PDO, by province

Key PI West Sumatera

Jambi West Kalimantan

East Kalimantan

9. Increased provider and client satisfaction, as measured by surveys (PAD)

n.r.

n.r.

n.r.

n.r.

10. Development and implementation of new standards for licensing, certification, and registration of health professionals and accreditation of health facilities (PAD)

S

S

S

S

11. Adoption of policies and financial and non-financial incentives by PHOs and DHOs to increase productivity, performance and motivation of health professionals (PAD)

MS

MS

MS

MS

16 Aside from the 10 key indicators listed in section 1.2 above the PAD lists an additional 49 “output indicators” in the Project Design Summary. Most are quite vague and open to various interpretations (e.g. “Percentage of providers satisfied with reformed health care system” and “Formative monitoring and supervision and technical assistance.”) 17 See WB Final Review for details.

15

PI-11: Law 32/2004 makes it clear that the central government retains control over the main parameters of personnel policy for civil servants. Nonetheless regional governments have considerable leeway in introducing reforms to increase productivity, performance and motivation, but a constraint during HWS was that regional governments were still waiting for clarification from the central government regarding the limits of their devolved authorities. The project was able to help districts and provinces develop HRH master plans and use these for more efficient assignment of staff (including to remote areas) but the gains so far have been modest. There were also improvements in the use of job descriptions, regular “mini-conferences” (staff self-assessments of their performance in work units), and in the use of income generated at service delivery points to enhance incentives for good performance (and to retain the services of experienced staff). There was significant movement in the right direction, but it was slow and piecemeal in the absence of clear and consistent guidelines from the central government. The rating for this PI is moderately satisfactory Table 6. HWS fellowship program, by province

Province D3 (Diploma)

S1 (Bachelor’s

Degree)

S2 (Master’s Degree)

Total

West Sumatera 14 236 250 500 Jambi 244 130 132 506 West Kalimantan 320 144 84 548 East Kalimantan 109 37 38 184 Total 687 547 504 1,738 Source: HWS Project Review Mission 19-31 May 2008, Draft Aide Memoire. Provinces and districts report that the main contribution of HWS to workforce policy, management and development was in the form of fellowships and training. Table 6 shows the allocation of fellowships by province (as of May 2008). There has been no attempt to assess systematically the contribution of these fellowships and training to the development of the workforce and improvement in performance. The ICR overall assessment of the extent to which the project achieved its second PDO is moderately satisfactory. Third PDO/National Level: Empower the Ministry of Health (MOH), the Ministry of National Education (MONE), and the Indonesian Medical Association (IMA), the three key stakeholders in the sector, through assistance to redefine their roles and responsibilities vis-à-vis health work force policy, planning and management, and through building their institutional capacity for effective stewardship in fulfilling the functions of policy making, legislation, regulation, quality assurance/control and technical assistance to provinces and districts.

As with the previous PDO, the ICR used data on associated output indicators as well as data on relevant key performance indicators in order to give an objective assessment of

16

performance on this PDO. This PDO had national scope and the organizational changes envisaged are not easy to identify and measure objectively. PI-12: The project was not able to strengthen the IMA as a key self-regulating professional association in the health sector, partly because the channeling of funds to IMA as an NGO was problematic.18 Nor was it able to help IMA develop a capacity for in-service training. HWS was able to support the association’s role in designing the internship program for primary care physicians (under component C2), and its College for Primary Care Physicians was also involved in the development of the new competency-based medical education curriculum. Performance on this PI is rated moderately unsatisfactory. PI-13: A major achievement of the project was to work with MONE on the development and implementation of a new competency-based medical education curriculum, using problem-based learning. Medical schools at Gadjah Mada University and the University of Indonesia were already moving in this direction before the project began, but it was a major accomplishment to get a new curriculum written and approved and the deans of all 52 medical schools as well as the medical associations to agree on this. It is too early to assess the overall impact of this achievement or how well the new curriculum is being followed in different medical schools. MONE and the World Bank are currently preparing a new project to further improve health professional education, including medical education using the competency-based approach. According to the Independent Evaluation a survey was conducted of first-year medical students at Mulawarman (East Kalimantan) and Andalas (West Sumatera) Universities after the introduction of the new curriculum and more than 80 percent expressed satisfaction. Performance on this PI is rated satisfactory. PI-14: Progress on redefining MOH’s roles and responsibilities regarding workforce policy, planning and management has been slow. An information system for all public health sector personnel is now available at the MOH, but it is still a very simple system which does not allow analysis of the rate of staff turnover, for example, or of how the existing workforce compares with anticipated changes in need brought about by the epidemiological transition; and it does not include the private sector. Performance on this PI is rated moderately unsatisfactory. PI-15: The MOH has made none of the structural changes described in this PI. The PAD was not specific on how the MOH might define and exercise its new stewardship functions regarding policy development, legislation development to empower health professions, and quality assurance and technical assistance to districts and provinces in a decentralized setting. During implementation the MOH showed little interest in

18 The most important recent development concerning the IMA’s professional right to self-regulation was the passing of Law 29/2004 on medical practice (including dentists), but this development took place independently of HWS.

17

reorganization and the project provided little to stimulate such interest. Performance on this PI is rated moderately unsatisfactory. Table 7. Achievement of key PIs for the Third PDO

Key PI Key central-level institution involved

12. IMA to have established an institutional structure and instruments for a national medical examination and certification, for formal, graduate and continuing medical education (DCA and PAD)

IMA

MU

13. The proportion of medical students in Mulawarman and Andalas Universities surveyed as satisfied with the curriculum and education provided to be at least 80 percent (DCA)

MONE

S

14. Redefinition of MOH’s roles and responsibilities vis-à-vis health workforce policy, planning and management (PAD)

MOH

MU

15. Strengthening of MOH’s institutional capacity through structural changes in organigramme, and increased allocation of human and financial resources for effective stewardship in fulfilling the functions of policy making, legislation, regulation, quality assurance/control and technical assistance to provinces and districts (PAD)

MOH

MU

The overall ICR rating for the third PDO is moderately unsatisfactory. Summary ratings by PDO and by components Table 8 gives the ICR rating of outcomes for each sub-component as well as overall rating for each PDO. Table 8. Rating of outcomes by PDO and by sub-component

PDO Rating of outcomes First PDO MU Second PDO MS Third PDO MU

Component Component A. District Health Offices and Health Facilities MU

A1: Improved access to and quality of health services MU A2: Project management MS Component B. Provincial Health Offices and Health Facilities MU B1: Health workforce development MU B2: Health system coordination, planning and management MS B3: Project management MS

Component C. Central Level MU C1: Effective health system stewardship (MOH) MU C2: Enhancing the quality of medical education (MONE) S

18

3.3 Efficiency The ICR considers the costs involved in achieving project objectives to be reasonable, but no technical cost-benefit analysis was undertaken at the close of the project. 3.4 Justification of Overall Outcome Rating The ICR rates overall outcome as moderately unsatisfactory. The justification for this rating is as follows: The Project idea was largely appropriate to the challenges brought upon by decentralization in health. However during implementation it became clear that the Project design was too broad and ambitious, particularly with wide range of objectives and the number of organization types involved. The overall outcome rating reflects the combination of all three PDO ratings. The First PDO, represents District component which is the largest component of of the Project, is rated moderately unsatisfactory. This rating reflects that most of the targets were not achieved despite the efforts to focus District activities to be more alligned with PDOs after the MTR. Although there were significant improvements in District’s capacity in planning and budgeting, severe delays in budget availability has reduced the Project’s effective implementation months. There were important achievements and some significant movements in the right direction, at the Provincial and National level, but they were in general slow and piecemeal in the absence of clear and consistent guidelines from the central government. 3.5 Overarching Themes, Other Outcomes and Impacts (a) Poverty Impacts, Gender Aspects, and Social Development Many project activities in the districts were seen by project staff as pro-poor but the activities themselves were generally not well-targeted at improving availability and accessibility of health services for the poor and it is difficult to assess their impact on the poor. Utilization of outpatient care in public facilities increased in 3 of the provinces but this was most likely mostly due to the introduction of the Askeskin (Health Insurance for the Poor) program in 2005. (b) Institutional Change/Strengthening There were no further institutional changes noted by the ICR other than those already discussed.

19

(c) Other Unintended Outcomes and Impacts (positive or negative) The provinces have indicated a follow-up project would be welcome. The MOH has expressed no interest in a follow-up project at this time. 3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops No Beneficiary Survey was conducted. 4. Assessment of Risk to Development Outcomes The ICR rates the risk to development outcomes achieved in West Sumatera and East Kalimantan as generally low, but considerably higher in Jambi and West Kalimantan. The risk to the new competency-based approach to medical education is also substantial; implementation has only recently began in the medical schools, many faculty are not trained in the new approach, and the first cohort of doctors to be trained under the new curriculum will not graduate for several years yet. The new approach needs sustained support if it is to develop deep roots. MONE and the World Bank are currently preparing a project to strengthen quality assurance of health professional education. The new project will enhance the sustainability of, and further progress regarding the HWS Project achievements in introducing the CBC by institutionalizing accreditation of medical schools, certifying the competence of graduates, and improving and assuring the quality of medical education. 5. Assessment of Bank and Borrower Performance 5.1 Bank Performance (a) Bank Performance in Ensuring Quality at Entry The ICR rates QAE as moderately unsatisfactory. HWS was very ambitious in its objectives but was too loosely organized in its design to be able to marshal effectively the available resources to achieve its objectives. The Project aimed to address decentralizing health sector management issues but failed to foresee potential risks that might arise from an ongoing decentralization process. The PDO indicators measured improvement in health outcomes which was unrealistic to attain in a five year project addressing health workforce issues. Those indicators show weak alignment with project components. (b) Quality of Supervision The Bank invested considerable time and effort in supervising HWS, especially in the early years when it reviewed and commented on all district grant proposals. The ISRs and mission aide memoires show the Bank monitored the project’s progress closely and made suggestions as to how to overcome obstacles and speed up implementation when it could.

20

In the Bank’s mission aide memoire January – February 2007, the Bank stated that “the project has made good progress but performance need[s] to be enhanced to achieve the development objectives of the project. This can be achieved, in part by simplifying the project where possible and focusing some of its activities.” What was missing was explicit recognition that not only was the project overly ambitious and complicated but that also some of the basic assumptions on which its design rested were not valid.19 Arguably many of the problems confronting project management were far more deeply rooted than the Bank’s exhortations for enhancing performance suggest. The Bank missed the opportunity to restructure the project (and amend some inappropriate procurement rules and thresholds) after the MTR. However, during and after the MTR, the Bank has consistently engaged the implementing units in re-focusing Project annual plans and activities to be more aligned with the PDO indicators. The task team also conducted a supervision QER after the MTR which focused on monitoring of project performance. Important mitigation steps were also taken to improve procurement management performance which resulted in a sharp decline in collusion practices by the end of the Project. The ICR rates quality of supervision as moderately satisfactory. (c) Justification of Rating for Overall Bank Performance The ICR rates overall Bank performance rating moderately satisfactory. The rating for Quality at Entry is moderately unsatisfactory reflects ambitious Project’s design and objectives, yet weak alignments between performance indicators in the Project’s legal documents and specific project components. During the Project implementation, the task team has taken important steps including re-focusing Project activities to be more aligned with PDOs, and mitigation measures in procurement management which led to its improved performance. 5.2 Borrower Performance (a) Government Performance The Borrower was the central government and the ICR rates its performance as moderately unsatisfactory. The central government’s failure to disburse project funds to the districts in a timely fashion seriously undermined the chances of success at the district level. (b) Implementation Agency or Agencies Performance

19 For example, among “critical assumptions” the PAD lists: Funds allocated to the sector are consistent, sufficient and predictably disbursed; Increased coordination amongst MOH, MONE and IMA allows the center, provinces and districts more flexibility in identifying their workforce needs and in finding mechanisms to hire, retain and compensate adequately their health staff; Agreement of the respective roles and responsibilities pertaining to health workforce management and training issues across all three levels of health systems; Agreement on the respective roles and responsibilities pertaining to health workforce policy issues across all three levels of health systems; Commitment by MOH to strengthen provincial health management and public health units; Commitment by MOH to assume a stewardship and advisory role instead of top down and technocratic mode of system management.

21

The CPCU worked hard to make HWS a success but the fact remains the MOH (the main implementation agency) did not reorganize or downsize in a way consistent with its new roles and responsibilities under decentralization for health workforce policy and other functions. Without reorganization (and modernization of its management structures20) the MOH was not able to coordinate efficiently and effectively a project aimed at facilitating decentralization. Performance was moderately unsatisfactory. (c) Justification of Rating for Overall Borrower Performance The ICR rates Overall Borrower Performance as moderately unsatisfactory. This rating is mainly driven by the government’s inability to disburse project funds to districts, and also weak guidance and coordination at the MOH. 6. Lessons Learned The CPCU, in collaboration with the PCIUs, organized two learning workshops at the province level (in East and West Kalimantan respectively) and one at the central level. A summary of the main points that came out of these workshops follows:21 Project design

In general, province and district stakeholders agreed that the design of the HWS project was overambitious and was not well organized. However, some of the Project features were appropriate for a decentralized setting as it provided equal opportunity for all three administrative levels to build capacity in the new decentralized era

Stakeholders welcomed the opportunity to conduct bottom-up planning under the project and many believed that establishing the capacity to conduct evidence-based planning was a major achievement of HWS

The “services” component channeled through the District Funding Allocations (DFA) became the most challenging component administratively as it involved over 50 implementing units (districts) and received the largest fund allocation

Participants at the central level felt that the MOH components would have been stronger had there been more concerted efforts on behalf of MOH during project preparation

Complex projects like HWS require a lot of learning and adaptation, particularly during the first years of implementation, and a longer implementation period is warranted

Project objectives and indicators

20 A complex project like HWS really needs a Project Director who can function like a CEO, but the management structure imposed by MOH means the PD functions more like a Board Chairman. The structure is not conducive to effective implementation. 21 These points are summarized and discussed in more detail in WB Final Review.

22

Workshop participants at all levels agreed that clear objectives and indicators are important for guiding project implementation and to keep it focused

When asked about project objectives, province participants were most familiar with the objectives to improve the capacity of human resources for health (HRH). Only a few stakeholders were aware of the project objectives as written in the legal documents and the PAD

There needed to be continuous “socialization” of the project given the high staff turnover

Central level participants mentioned that the objectives of the central components were not well defined

There was general agreement that lack of awareness of project indicators and their importance for measuring project performance contributed to lack of focus during implementation

Project components

The unclear role of the province level at the beginning of decentralization contributed to the rather loose description of the provincial component of the project

Participants stated that the largest contribution to HRH at the sub-national level was the fellowship program (especially for midwives and nurses)

Participants confirmed that districts appreciated being invited to participate in designing the district component during project preparation

Participants agreed that performance during implementation was better where the PAD clearly described the scope of project activities

Institutional arrangements

Rather than forming JHC, participants believed it was better to foster cross-district collaboration using existing fora such as “musrenbang” and setting aside time in these fora for discussion specifically on health issues

An important determinant for a well-functioning DHC is good leadership by the Head of the DHO and his or her strong advocacy skills in developing commitments for health among key stakeholders

There was consensus among the provinces and the districts that the TRT is an important project contribution for strengthening the planning process and for improving province-district collaboration

Financial management

Multiple changes in the mechanism to channel funds to the districts resulted each year in long delays in getting money to the districts

Despite the frustration, some considered an unexpected contribution of HWS was to function as a laboratory for MOF, Bappenas, and the line ministries for testing alternative ways to channel foreign loans to the districts

The absence of a clear fund channeling mechanism to non-governmental institutions, such as the IMA, was also considered a constraint.

General comments on advantages and disadvantages of HWS

23

Could be used to fund activities that are difficult to finance under APBD/APBN (e.g. fellowships, pilot studies, comparative studies, TA) (Advantage)

Allowed flexibility to allocate funds according to local needs instead of rigid allocation according to centrally defined interventions (Advantage)

Provided additional resources to implementing units (Advantage) Provided seed money to initiate dialogue for bigger funding from APBD/APBN

(Advantage) Project performance was very much influenced by uncertainties since it was

prepared and implemented in the early years of decentralization when policies and regulations were still evolving (Disadvantage)

Difficult to follow WB procurement guidelines and obtain WB NOL (Disadvantage)

The ICR wishes to add some further general points. To be successful a project needs clear goals, perspicuous organization, and a rigorous mechanism for monitoring progress. HWS was found lacking in all three areas. Its development objectives were extremely broad and its key performance indicators were too arbitrary and were never adequately operationalized or monitored. It was never clear how project management was supposed to coordinate activities across so many components operating simultaneously on so many organizational levels, or integrate their outputs. The result was a proliferation of activities with few major impacts verifiably attributable to those activities. The project had many successes, but most tended to be small and local rather than project- or sector-wide. It may be better to view (and subsequently assess) an omnibus project like HWS as a program more than a project. One important lesson not previously mentioned is the need to strengthen risk assessment in a project like HWS and have a “Plan B” ready for when project activities are seriously threatened by external factors over which the project has no control. The money came late every year, yet those entrusted with implementing the project always acted as if in the next year things would be different. No strategy was developed aimed specifically to help districts implement 12 months’ activities in just 4 or 5 months if the money was only available in August. Experience with HWS also underlines the importance of robust scientific monitoring and evaluation. All three PHP experienced similar problems. Many of these problems might have been avoided if when the PAD was drafted instead of including inadequately thought through performance indicators an M&E specialist was invited to contribute. The relative scarcity of good data, and the misuse of the good data that do exist, are perennial problems in the Indonesian health sector, and the Bank is wasting a valuable opportunity to alleviate the situation if it launches projects with weak M&E. This project would have accomplished so much more if funds had been disbursed in a more timely fashion. This factor combined with the implementing agency’s decision towards the end of 2008 not to proceed with an extension almost guaranteed many objectives would not be achieved in the provinces and districts.

24

Ratings Summary Performance Rating by ICR Outcomes MU Risk to Development Outcome Substantial Bank Performance MS Borrower Performance MU Detailed Ratings of Bank and Borrower Performance by ICR

Bank Rating Borrower Rating Quality at Entry MU Government MU Q. of supervision MS Implementing

agencies MU

Overall Bank Performance

MS Overall Borrower Performance

MU

7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners Notes : The final ICR text has incorporated some changes in response to the comments provided. (a) Borrower/Implementing Agencies I. Background

ICR HWS Project is an important document and give more information about input, process, output, impact and managerial environment of the project. Data collection by the ICR team / World Bank Consultant, facing many problems such as limitation of time, language capacity of the respondent, and opportunity for some government official attended in FGD’s. To support the ICR team CPCU, PCIU and some Implementing units had delivered many documents and data to the consultant.

The MOH comments as an effort to complete the above information and suggest some revision in order for comprehensiveness ICR HWS. We hope that government and Bank will have the same perception for determine conclusion/rating the result.

II. Comments on Content of ICR

A. Data and presentations suggested improvement

1. There are difference data and presentation on draft ICR e.g.

a. Role of district Bappeda (Component A.2 page 4 “The Director for District Implementing Unit was the Head of District Bappeda and Executive Secretary was the Head of District Health Office),

b. Date and amount disbursement (Page 5, Point 1.7 By 31 Dec 2008 The Project had disbursed USD 55,720,152.13 (52,77 percent from the original

25

project allocation) or 88,15% after cancellation. An extension plan had proposed by the Director of the Project but the Government decision on 11 October 2008 suggest to close as originally planned),

c. Proportions provincial health budget, (PI-4, page 11), source of data ‘Integrated Final Evaluation Report, HWS Project 2008’

Table 3.1 Local Health Budget Proportion 2004-2007 Province

Local Health Budget Proportion (%) 2004 2005 2006 2007

West Sumatera 8.64 6.89 4.86 4.72 Jambi 4.94 4.30 5.97 7.91 West Kalimantan 4.22 9.32 10.13 11.37 East Kalimantan 7.76 8.65 6.62 9.51

d. Per capita health spending, (PI – 4 page 12), source of data ‘Integrated Final Evaluation Report, HWS Project 2008’

Table 3.2 Percapita Health Spending based on 2002 IDR Price By Province,

2004-2007 Province

Percapita Health Spending (2002 IDR Price)

2004 2005 2006 2007

West Sumatera 40,685 49,056 47,369 48,840 Jambi 46,953 40,076 51,169 62,115 West Kalimantan 48,532 52,029 64,081 73,590 East Kalimantan 137,071 110,890 161,247 164,883

e. births attended by health personnel there are trends to increase from 2003 - 2008, etc.

2. Table 4.1 Births Attended by Health Personnel in HWS Provinces 2003 - 2008

Provinces 2003 2004 2005 2006 2007 2008 Trend West Sumatera 80.12 81.81 80.15 75.60 82.87 82.90 Increased Jambi 67.45 66.63 82.14 78.05 75.94 83.94 Increased West Kalimantan 65.56 68.94 68.38 66.08 73.48 75.61 Increased East Kalimantan 75.51 65.31 73.26 72.89 75.34 75.78 Increased

Source : Directorate of Maternal Health, DG Community Health and Province ISR 2008

3. Adding statement at some paragraphs for clarification of role, clear perception, completing information, etc.

4. Improvement rating status

B. Specific comment :

1. The attempt to assess the contribution of fellowship and training.

Assessment of fellowship and training was not conducted because:

26

a. We certainly sure that there will be no any significant result. Around 50% of fellows were graduated on year 2006, the rest gradually completed their education on 2007- 2009 (around 10% were financed by APBD and APBN in 2009). It meant that they contributed for one year only to health service/ program, since the final evaluation was based on 2007 data. In fact all graduates still has to adapt to their task or function, and need to improve their knowledge, attitude ,skill ,and increase experiences to meet the competence as needed. Behavior changes need many years to process, in our opinion it’s more effective if assessment conducted after at least 2-3 years of practice.

b. To assess that contribution will need Consultant to carry out, which will cost a sum of fund much higher than the very low threshold of procurement, and lead to 9 months long process of QCBS method; it will be unlikely to complete till the closing of the project.

c. Amid the late fund release, in 2008 CPCU has already had a very heavy load of consultants procurement.

We wish that The ICR would like to look at the fellowship mater in a positive way so that be able to acknowledge the real need of so many young and very young Districts to improve their health workforces quality.

C. Assessment of Performance Indicator of PI -14 and PI-15.

1. It was true that this PI implementation was generally slow, but - refer to Team statement on page 19 on the bottom- it was not true that HWS did not produced National Policies!

a. CBC developed by MONE is a National Policy.

b. Even though the process was very slow, Specialist Medical Training also raised as one of priority National Policy.

c. Internship Program, prepared by IMA, now was taken over by MOH due the long last and complicated implementation on the field.

From positive perspective, it can be said that HWS Project has left behind historic mile stones to HHR Development, which are now financed by APBN, high committed and sustainability has been secured. It’s a very good remark, isn’t it?

Information System business was a very long story of bitterness in MOH, we can not imagine how this kind of weakness of one IU can significantly influence or even “ignore” other central IUs performances

2. Concerning Reorganization Issue

On the very beginning of HWS, the Issue of reorganization as written on PAD, was meant to change the status of Health Academics belong to MOH to a kind of State Owned Company Schools. Several preparation activities had been conducted, such as consultation, data collection, meetings and workshops; but it was found that kind form of school will never be able to be established due to the lack of laws to refer to.

27

That finding had been informed and discussed in a meeting with The Bank Supervision Team that time , and since then the issue was never been raised, mentioned or questioned any more.

BDEHHR did reorganization – partial reorganization- twice during HWS life time, but it focused to adjust to its tasks and functions.

We wonder if The ICR in the end of the Project still raised that issue as a negative finding.

D. Cost Benefit Analysis

Since in 2008 all activities still in progress, we thought that kind of analysis should be conducted after everything were completed. The condition as stated on number I.2 and 3 above was also taken as a consideration. However if The Bank Team felt that mater was important, it can be done by external Consultant like Procurement’s

E. The possibility of inconsistency in rating.

1. In Table 8, under C component, it was written CI rating as MU, C2 as S, while the conclusion was MU.

Was it because the intensity of MU of C component so bad when IMA also included in? This kind of unclearness might be happen if qualitative approach was used.

2. The last two tables tended to mislead the readers. In case of Bank Performance, it can be perceived that lower table was as explanation of the upper, but it seemed not match (MS versus MU). Or, the MU as stated on upper table was re-graded more detail and became MS.

F. Proposition for Bank Performance changing.

On the process of planning, MOH/ Borrower was closely assisted and directed by Bank experts, until agreement was achieved, and the PAD officiated.

We thought that the relationship between MOH/Borrower and the Bank was as “two faces of a coin”.

Learning from the way The Bank assessed C component, we are inspired to make some comparison as follows:

Refer to Statement written on page 6, alinea 4--��

“These limitations in quality of entry…………but there was insufficient time for the Bank to fully harmonize all these inputs in PAD (and fill in the remaining gaps in planning)”

And

Page 16 on the bottom….> “HWS was very ambitious in its objectives was too loosely organized in its design to be able to marshal effectively the available resources to achieve its objectives.” (Haven’t all parties known and agreed?)

Page 20 alinea 2-�” Experience with HWS…………….Many of the problems might have been avoided if when the PAD instead of including so many “ off the

28

top one’s head” potential performance indicators an M& E Specialist was invited to contribute.”

Even though The Bank International Experts together with Government Officials and Indonesian experts had extensively prepared, “but still no one could predict with confidence exactly how the definition and practical allocation of responsibilities for health across the main levels of government would evolve."

G. Finding of Final Evaluation by Independent Consultant note that: "Considering the variety of problems faced by the project within the limited time frame, the implementation of the HWS Project could be categorized as quite satisfactory, particularly, the HWS Project is the first project which includes the anti corruption plan as one of its project activity. " (Integrated Final Report, Final Evaluation HWS Project 2008)

Recommendation ICR Draft Revision: We propose the rating of Government quality of entry should be changed into MS. (b) Co-financiers N/A (c) Other Partners and Stakeholders N/A

29

ANNEXES

Annex 1. Project Costs and Financing

(a) Project Cost by Component (in USD Million equivalent)

Components

Appraisal Estimate

(USD millions)

Actual/Latest Estimate

(USD millions)

Percentage of Appraisal

(%)

A. District Health Offices and Health Facilities * 309.25

39.06

12.63

B. Provincial Health Offices and Health Facilities

B1. Health Workforce Development 19.05 2.25 11.81

B2. Health System Coordination, Planning and Management

3.38

1.33 39.34

B3. Project Management 1.74 1.95 112.07 C. Central

C1. Effective Health System Stewardship (MOH)

9.66 4.33 44.82

C2. Enhancing The Quality of Medical Education (MONE)

12.616.01

47.66

Total Project Costs 429.37

57.49

13.39

Total Financing Required 429.37

* Inclusive of Sub Component A1. Improved access to and quality of health services, and A2. Project Management

(b) Financing

Source of Funds Type of

Cofinancing

Appraisal Estimate

(USD millions)

Actual/Latest Estimate

(USD millions)

Percentage of Appraisal

(%)

International Development Association (IDA)

74.50 54.78 73.53

30

Annex 2. Outputs by Component

District Component

Component West Sumatera Province

Jambi Province West Kalimantan Province

East Kalimantan Province

A. District Improved access to and quality of health services

Improved in planning and budgeting skills and knowledge shown with significant improvement of DFA proposal over time

2003 : The utilization of health facilities by the poor 36.48%, in 2007 only slightly increased by 6% to 38%

2006: Ratio MD to 100,000 population increased to 22.2 from 20.8, Ratio of Midwives to 100 K population has almost doubled to 59.1

The family doctor and medical billing system is piloted in Tanah Datar district

2003 Skilled birth attendance 79.8% and relatively stagnant in 2007 with 80.5%


2003 : The utilization of health facilities by the poor 27.34%, in 2007: increased only 20% to 32.66%

2006: Ratio MD to 100,000 population increased to 19.8 from 12.4, Ratio of Midwives to 100 K population has more than doubled from 21.7 to 46.8

2003 Skilled birth attendance 70.5% and decreased slightly to 69.8% in 2007

2003 Pregnant women receiving at least four ante natal visit 78.5% and


2003 : The utilization of health facilities by the poor 19.67% in 2007 surpassed the target with 67% to 33%

2006: Ratio MD to 100,K population relatively has not changed from 12.4 to 12.2, while ratio of Midwives to 100 K population has almost tripled to 33.6

2003 Skilled birth attendance remain unchanged 63% in 2003 and 62% in 2007

2003 Pregnant women receiving at least four


2003 : The utilization of health facilities by the poor 26.63%, in 2007 increased 42% to 37.35%

2006: Ratio MD to 100K population has been stagnant at 22.2, Ratio of Midwives to 100 K population has doubled from 15.7 to 39.1

The family doctor concept is piloted at the City of Bontang

2003 Skilled birth attendance decreased from 79% in 2003 to 75.5% in 2007

31




2003 Pregnant women receiving at least four ante natal visit 79.8% and increased to 84.8% in 2007

increased to 82.4% in 2007

Development and establishment of Maternal Waiting Home for remote areas in two districts 2006-2008

Health services for vulnerable indigenous population in Districts Batanghari 2005 - 2008

ante natal visit 65.7.% and increased to 78.3% in 2007

2003 Pregnant women receiving at least four ante natal visit 78.5% and increased to 82.4% in 2007

Project Management B. Province Health Workforce Establishment of a

Quality Council or MTKP Recruitment and training of midwives assessor and competence testing for midwives in 2007 - 2008 Fellowship for D3 program 667, bachelor degree (S1) is 218, and for post graduate Master program 216, and 29 for Specialist training. Total

HRH Master Plan 04-08 is available Establishment of regulation and certification for Midwives, 2007 – 2008, the guidelines will be in Governor’s decree Fellowship for D3 program 366, bachelor degree (S1) is 128, and for post graduate Master program 134, and 23 for Specialist program, the

Develop HRH Master plan in 2008 Piloted the SIMPEG Establishment of a Quality council for health workforce, or MKTP Conducted competence testing for midwives starting 2008 Fellowship for D3 program 1,050, and Bachelor degree (S1) 175, and for Post graduate Master

Conducted various studies and trainings on health workforce issues and regulations in collaboration with Gadjah Mada University in 2004 – 2008 and produce HRH Master plan for 04-08 and is being prepared the 09-13 Establish Health Professionals Forum and then the Province Quality Council (MTKP) piloted 2007,

32




fellowship is 1,130. Establish Functional Training Unit in Bukittinggi to assess training need and accredit in-service training unit 2007

total the number of fellowship is 651 Strengthen in-service training by accrediting Bapelkes from 2004 – 2007

program 89, and 22 MDs for the Hospital based Specialist training

implemented in 2008 Develop local regulations for accreditation and licensing for health practitioners Fellowship for D3 program 117, bachelor degree (S1) is 37, and for post graduate Master program 37, in total the number of fellowship is 191

Health System Coordination

Development of Provincial Health System document is to be finalized in 2009

2007 conducted a cohort study on maternal and infant mortality at the Province level

2004 – 2007 Develop health promotion strategy and establish a Mini Studio to produce health promotion material

Development of Provincial Health System Document in 2006

2006-2007 Conducted Local Health Household Survey in collaboration with the NIHRD

An integrated HIS system covers 7 out of 10 districts 2008, an integrated surveillance is also developed

Development of Provincial Health System document in 2008

Governor decree for Minimum Service Standards in 2008

2007 Establishment of West Kalimantan Health Information System, with an integrated online system with the Central covers 7 districts

2006, 2007 Mapping of Midwives; A database

Development of Provincial Health System Document in 2008

Develop local regulations for user fees, accreditation and licensing for health providers including private facilities in 2007

Develop quality assurance framework, first for public providers and facilities, and then expanded to private providers in

33




of midwives

Local Health Household Survey (Surkesda) in 2007

2008

Establish Province Health Office structure refer to the Government regulation that fits local specific needs in 2008

Establish Provincial Health Account system to provide inputs for health policy making, 2007 - 2008

Project Management Five rounds of HWS Project Province proposal

Establishment of JHC and TRT

Five rounds of HWS Project Province proposal






34

C. Central Components C.1: Effective health system stewardship (MOH) 2005 - Medium term human resource for health (HRH) strategy has developed 2006-2007 - Distance learning module development 2007 - Pilot of competence testing for midwives 2008 - Grand design of a competence based training of specialist doctors (PDSBK) and six PDSBK modules (for internal medicine, pediatrics, ob-gyn, surgery, anesthesiology, and clinical pathology) 2008 - Design for certification of midwives including standard of competencies, competence testing, and the concept of Indonesia Health Workforce Board (MTKI) 2008 - Design of an in-service training unit at the district level Strengthen institutional capacity of IMA 2005 - The academic papers on i) Concept of Family Doctor, ii) The concept of Internship program for primary care physician, iii) Improvement in Organizational structure and arrangement of IMA 2007 - Piloting of family doctors in two districts in East Kalimantan and West Sumatera C.2: Enhancing the Quality of Medical Education (MONE) 2004 - Establish National Medical Education Development and Research Center 2004 - Develop and completed the competence based curriculum (CBC) for medical education which is then validated by the Indonesian Medical Council (KKI) as the new curriculum known as KIPDI III. In 2006 the Standards of Medical Education and Standards of Doctor’s Competence were also developed based on the draft curriculum 2005 - Start the implementation of the new KIPDI III/CBC curriculum in 5 medical schools 2007–2008 - Completion of civil works of PBL Facilities at University of Andalas, West Sumatera, and Skills Laboratory at University of Mulawarman in East Kalimantan 2008 - Guidelines and manuals for competency-based assessment, quality assurance in medical education, and development of admission test

Annex 3. Economic and Financial Analysis

Not Applicable

35

Annex 4. Bank Lending and Implementation Support/Supervision Processes

(a) Task Team Members

Names Title Unit Lending Samuel S. Liberman Lead Economist EASHD Anthony Toft Chief Counsel LEGES Put Marzoeki Senior Health Specialist EASHD William Hardi Consultant EAPCO Yogana Prasta Operation Adviser EACIF Naseer Ahmad Rana Adviser SARSQ Novira Kusdarti Asra Financial Management Specialist EAPCO Benedicta R. Sembodo Program Assistant EACIF

Supervision Puti Marzoeki Senior Health Specialist EASHD Novira Kusdarti Asra Financial Management Specialist EAPCO Jed Friedman Economist DECRG William Hardi Consultant EAPCO Pandu Harimurti Health Specialist EASHD Adrian Hayes Consultant EASHD Peter F. Heywood Consultant EASHD Edhie Santosa Rahmat Consultant EASHD Claudia Rokx Lead Health Specialist EASHD Imad Saleh Lead Procurement Specialist EAPCO Agus Sasmito Consultant EASHD Anita Kentjanawati Tuwo Procurement Specialist EAPCO Ryma F.J. Aguw Team Assistant EACIF

(b) Staff Time and Cost

Stage of Project Cycle Staff Time and Cost (Bank Budget Only)

No. of staff weeks USD Thousands (including travel and consultant costs)

Lending FY97 7.80 FY98 33.64 FY99 93.58 FY00 57 102.48 FY01 0.00 FY02 0.09 FY03 0.00 FY04 0.00

36

Stage of Project Cycle Staff Time and Cost (Bank Budget Only)

No. of staff weeks USD Thousands (including travel and consultant costs)

Lending FY05 3 1.89

Total: 60 239.48 Supervision/ICR

FY01 22 32.46 FY02 32 74.34 FY03 14 57.82 FY04 30 126.69 FY05 22 74.22 FY06 21 46.91 FY07 19 53.71 FY08 20 45.03

Total: 180 511.18

37

Annex 5. Beneficiary Survey Results

Not conducted

38

Annex 6. Stakeholder Workshop Report and Results

Not Conducted

39

Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR

40

Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders

Not Available

41

Annex 9. List of Supporting Documents

Project Appraisal Document Credit Agreement Implementation Supervision Reports (ISRs) Aide Memoires Project baseline, midterm, and final evaluation reports Tracking study reports

42

MAP

IBRD 37030