Document of The World Bank

Report No:ICR000012

IMPLEMENTATION COMPLETION AND RESULTS REPORT (IDA-N0410)

ON A

CREDIT

IN THE AMOUNT OF XDR 56.8 MILLION (US$ 76.4 MILLION EQUIVALENT)

TO

India

FOR THE

ORISSA HEALTH SYSTEMS DEVELOPMENT PROJECT

September 26, 2006

Human Development Unit South Asia Region

This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not otherwise be disclosed without World Bank authorization.

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

CURRENCY EQUIVALENTS (Exchange Rate Effective 01/01/2006)

Currency Unit = Rupee Rupee 1.00 = US$ 0.02

US$ 1.00 = Rupee 45.12

Fiscal Year April 1 - March 31

ABBREVIATIONS AND ACRONYMS

CAS Country Assistance Strategy CHC Community Health Center CMC Construction Management Consultant DEA Department of Economic Affairs DH District Hospital DHS Directorate of Health Services DfID Department for International Development FMR Financial Management Report FMS Financial Management System FRU First Referral Unit GOI Government of India GOO Government of Orissa HCWMP Health Care Waste Management Plan HMIS Health Management Information System IDA International Development Association IDCO Industrial Development Corporation of Orissa IEC Information, education and communication IEG Independent Evaluations Group ISP Informal Service Provider ITF Interim Trust Fund M and E Monitoring and Evaluation MDG Millennium Development Goal MHU Mobile Health Unit MTR Mid-Term Review NGO Non-government Organization NRHM National Rural Health Mission NSSO National Statistical Survey Organization O and M Operations and Maintenance OHSDP Orissa Health Systems Development Project

OPD Out-patient department OPV Oral Polio Vaccine PAD Project Appraisal Document PD Project Director PDO Project Development Objective PIP Project Implementation Plan PLA Personal Ledger Account PMC Project Management Cell PSPU Policy Support and Planning Unit PSR Project Status Report QA Quality Assurance QAG Quality Assurance Group QER Quality Enhancement Review RCH Reproductive and Child Health RKS Rogi Kalyan Samiti (Patient Welfare Committee) SC Scheduled Caste SHSP State Health Systems Projects ST Scheduled Tribe WDR World Development Report WHO World Health Organization ZSS Zilla Swasthya Samiti

Vice President: Praful C. Patel

Country Director: Fayez S. Omar Sector Manager: Anabela Abreu

Project Team Leader: Preeti Kudesia

INDIA ORISSA HEALTH SYSTEMS DEVELOPMENT PROJECT

CONTENTS

1. Basic Information............................................................................................................ 12. Key Dates........................................................................................................................ 13. Ratings Summary............................................................................................................ 14. Sector and Theme Codes ................................................................................................ 25. Bank Staff ....................................................................................................................... 26. Project Context, Development Objectives and Design................................................... 37. Key Factors Affecting Implementation and Outcomes .................................................. 78. Assessment of Outcomes .............................................................................................. 159. Assessment of Risk to Development Outcome............................................................. 2210. Assessment of Bank and Borrower Performance ....................................................... 2311. Lessons Learned.......................................................................................................... 2712. Comments on Issues Raised by Borrower/Implementing Agencies/Partners............. 29Annex 1. Results Framework Analysis............................................................................. 31Annex 2. Restructuring (if any) ........................................................................................ 36Annex 3. Project Costs and Financing.............................................................................. 37Annex 4. Outputs by Component...................................................................................... 39Annex 5. Economic and Financial Analysis (including assumptions in the analysis)...... 48Annex 6. Bank Lending and Implementation Support/Supervision Processes................. 51Annex 7. Detailed Ratings of Bank and Borrower Performance...................................... 54Annex 8. Beneficiary Survey Results (if any) .................................................................. 55Annex 9. Stakeholder Workshop Report and Results (if any).......................................... 56Annex 10. Summary of Borrower's ICR and/or Comments on Draft ICR ....................... 57Annex 11. Comments of Cofinanciers and Other Partners/Stakeholders ......................... 70Annex 12. List of Supporting Documents ........................................................................ 71Map ................................................................................................................................... 73

1

1. Basic Information

Country: India Project Name: ORISSA HEALTH SYS

Project ID: P010496 L/C/TF Number(s): IDA-N0410 ICR Date: 09/29/2006 ICR Type: Core ICR Lending Instrument: SIL Borrower: GOI Original Total Commitment:

XDR 56.8M Disbursed Amount: XDR 56.8M

Environmental Category:B Implementing Agencies

Project Management Cell, OHSDP Cofinanciers and Other External Partners 2. Key Dates

Process Date Process Original Date Revised / Actual Date(s)

Concept Review: 02/24/1997 Effectiveness: 09/08/1998 09/08/1998 Appraisal: 01/20/1998 Restructuring(s): Approval: 06/29/1998 Mid-term Review: 04/15/2002 Closing: 03/31/2004 03/31/2006 3. Ratings Summary 3.1 Performance Rating by ICR Outcomes: Moderately Satisfactory Risk to Development Outcome: Moderate Bank Performance: Moderately Satisfactory Borrower Performance: Moderately Satisfactory 3.2 Quality at Entry and Implementation Performance Indicators Implementation Performance Indicators QAG Assessments (if any) Rating:

Potential Problem Project at any time (Yes/No):

No Quality at Entry (QEA): None

Problem Project at any time (Yes/No):

Yes Quality of Supervision (QSA): Highly Satisfactory

DO rating before Closing/Inactive status:

Satisfactory

2

4. Sector and Theme Codes Original Actual

Sector Code (as % of total Bank financing) Sub-national government administration 9 9 Health 91 91

Original Priority Actual Priority Theme Code (Primary/Secondary) Participation and civic engagement Secondary Secondary Indigenous peoples Secondary Secondary Health system performance Primary Primary Population and reproductive health Primary Primary 5. Bank Staff

Positions At ICR At Approval Vice President: Praful C. Patel Mieko Nishimizu Country Director: Fayez S. Omar Edwin R. Lim Sector Manager: Anabela Abreu Richard Lee Skolnik Project Team Leader: Preeti Kudesia Tawhid Nawaz ICR Team Leader: Vikram Sundara Rajan ICR Primary Author: Vikram Sundara Rajan

3

6. Project Context, Development Objectives and Design (this section is descriptive, taken from other documents, e.g., PAD/ISR, not evaluative) 6.1 Context at Appraisal (brief summary of country macroeconomic and structural/sector background, rationale for Bank assistance) The Orissa Health Systems Development Project (OHSDP) was approved by the Bank's Board on June 29, 1998 and became Effective on September 8, 1998. At that time, India was in the process of continuing economic reforms initiated in the early 90s and the increased economic growth was leading to a reduction in poverty. Central and state governments at the time were initiating increase of private sector participation in areas like infrastructure but the openness for Public Private Partnerships (PPPs) in areas like health was limited. Orissa was one of the most backward states in India with more than 44% of its population below the poverty line and with poor health infrastructure and outcomes (Infant Mortality rate of 77 per 1000). The main health sector issues identified at the time of appraisal at the state level were as follows: Financing Issues: In the public sector, provision and financing of health has been a shared responsibility of the central and state governments with close to three-fourths of the financial burden being borne by the states. However: (i) the overall allocation to health was low due to the deteriorating fiscal situation in most states, including Orissa; (ii) within the health sector, resource allocation in the public sector was skewed in favor of tertiary care services relative to the needs at the primary and secondary levels; and (iii) much of the resources were absorbed by the salary costs, and the recurrent budgets for operations and maintenance were chronically under-funded. Performance of Health Systems: Response of most states to the high rates of communicable and non-communicable diseases was hampered by: (i) Over-reliance on population-size based norms rather than specific health needs at the community level; and (ii) inadequate attention to workforce issues including insufficient incentives for staff, and limited in-service training. Capacity for Management, Planning and Coordination: There had been progress in recent years in the availability, quality and use of information on health financing and management capacity at the central and state levels. However, overall capacity for management, planning and coordination remained limited in the government sector. Government and private sector roles: About 80% of health spending was private and largely from out-of-pocket sources. The private sector played a dominant role in the provision of individual curative care through ambulatory health services, whereas the

4

public sector provided most hospitalization services and preventive and promotive health services. Despite the comparative advantage of each of the two sectors, public-private partnerships were not fully exploited. The government's capacity to deliver both essential public health and curative care services needed to be strengthened since government facilities were accessed by the poor for curative care, especially in states like Orissa as outlined in section 8.5(a). Government Strategy: Government of India (GOI) had declared health as one of the six priority areas identified in the Ninth Plan (1997-2002). The Ninth Plan emphasized successful preventive and promotive activities, better control of communicable and non-communicable diseases, improved surveillance and improved systemic efficiency. In addition, the Central Council of Health and Family Welfare noted the importance of linking preventive and promotive care with selective aspects of curative care provided at first referral hospitals. Since the states were largely responsible for the financing and implementation of health programs, especially upgradation of healthcare infrastructure, these issues needed to be addressed at the state level. Rationale for Bank Assistance: The project was in keeping with the Country Assistance Strategy (CAS; Report No. 17241 -IN; dated December 19, 1997) which recommended a focus for Bank Group-financed investments on states which were undertaking economic restructuring programs and supporting sectoral policy reform. Government of Orissa (GOO) had undertaken a detailed study of public expenditures jointly with the Bank, and had initiated a process of fiscal reform. In the health sector, IDA's specific objective was to work with public, private and voluntary sectors towards (a) improving primary care and nutrition policies; (b) consolidating disease control efforts; and (c) upgrading state health systems. Since most of the disease control and family welfare programs were targeting investments at the primary care level, the upgrading of state health systems focused on the First Referral Unit (FRU) level to complement these interventions, especially for critical programs such as safe motherhood and child survival. The project supported these objectives by strengthening institutional capacity and upgrading effectiveness and quality of services at first referral levels of health care. Finally, the project was to strengthen IDA's strategy of poverty reduction in India through its focus on underprivileged people, especially women and SC/STs. 6.2 Original Project Development Objectives (PDO) and Key Indicators (as approved) The objectives of the project were to assist GOO to: (i) improve efficiency in the allocation and use of health resources through policy and institutional development; and (ii) improve the performance of the health system through improvements in the quality, effectiveness and coverage of health services at the secondary level and selective coverage at the community level, so as to improve the health status of the people, specially the poor, by reducing mortality, morbidity and disability.

5

6.3 Revised PDO and Key Indicators (as approved by original approving authority), and reasons/justification The development objective was not revised during project implementation. However, there was a reallocation of US$25 million (XDR 17.02 million) to assist GOI in eradicating poliomyelitis, which came from the net savings of the project. This was communicated to the Department of Economic Affairs (DEA) by IDA on December 23, 2004 and had been made effective from April 1, 2004. This had supplemented the then ongoing Immunization Strengthening Project (Cr. 3340-IN) by financing "polio eradication activities" that included Oral Polio Vaccines (OPV), training, orientation, honorarium, consultant services, mobility support and Information, Education, Communication (IEC) material. The ICR report for the Immunization Strengthening Project would assess the achievement of this objective. 6.4 Main Beneficiaries, original and revised (briefly describe the "primary target group" identified in the PAD and as captured in the PDO, as well as any other individuals and organizations expected to benefit from the project) The main project beneficiaries were identified as follows: (i) the state's population as a whole, particularly the poor, would be benefited through the establishment of an effective and sustainable health system in Orissa. Studies had shown that the majority of those utilizing proposed project facilities belonged to the poorest 40% of the population; (ii) those using the secondary care system would be major beneficiaries as quality of services provided was to be improved across the board and the establishment of service norms would allow more cost-effective service delivery, allowing the state to provide a larger array of services to a greater number of people; (iii) in rural and underserved areas, the strengthening of the primary level of care and linkages with the secondary level would increase access to essential primary and clinical services to patients in these areas; and (iv) tribal populations were to benefit under the Tribal Development Plan, through improved services at health facilities in tribal areas as well as community-based outreach services. 6.5 Original Components (as approved) The project comprised the following components: Component 1: Management Development and Institutional Strengthening (US$ 10.1 million; 11% of total project costs): The component included (a) strengthening planning, management and implementation capacity at the state, district and facility levels; and (b) strengthening the HMIS and capacity for surveillance of major communicable diseases.

6

Component 2: Improving Service Quality, Access and Effectiveness at the Secondary Level (US$ 52.5 million; 58% of total project costs): This component included (a) upgrading area, sub-divisional and district hospitals; and (b) upgrading the effectiveness of clinical, managerial and support services at the facility level. Component 3: Improving Access to Basic Health Services at the Community Level (US$ 28.1 million; 31% of total project costs): The component included (a) selective upgrading of community health centers; (b) promoting health services in tribal areas and for disadvantaged groups; and (c) improving referral mechanisms and strengthening linkages between different tiers of the health care system. Under all components, the project supported civil works, goods, consultants and services, incremental salaries, and operations and maintenance costs. In addition, the project supported GOO in implementing a key set of policy initiatives, contained in a Letter of Health Sector Development Policy, the important points of which were linked to the Remedies section of the ITF Development Credit Agreement. The policy issues included: (i) increasing financing and improving resource allocation for the health sector; (ii) strengthening capacity for management, planning and coordination; (iii) enhancing the role of the private and voluntary sectors; (iv) implementing a user charge policy; (v) providing incentives for the workforce; and (vi) redressing regional and other imbalances. 6.6 Revised Components Not Applicable. 6.7 Other significant changes (in design, scope and scale, implementation arrangements and schedule, and funding allocations) The project was originally scheduled to close on March 31, 2004 and was extended twice for a period of 12 months each. The final closing date for the project was March 31, 2006. The original Credit was for US$ 76.4 million equivalent (XDR 56.8 million); and as of September 7, 2006, US$ 84.6 million equivalent (XDR 56.76 million), i.e., 99.93% has been disbursed. There was a reallocation of US$25 million equivalent or XDR 17.02 million to assist GOI in eradicating poliomyelitis, which came from the net savings of the project. The Health Department Building (Swasthya Bhawan) was not constructed as per the decision of the GOO and the PMC. There was no other significant change in design, scope or implementation arrangements. For details of the funding reallocation and extensions, please see section 7.2.

7

7. Key Factors Affecting Implementation and Outcomes 7.1 Project Preparation, Design and Quality at Entry (including whether lessons of earlier operations were taken into account, risks and their mitigations identified, and adequacy of participatory processes, as applicable) The project has been consistent with the CAS (Report no. 17241-IN, Dec. 1997) goals of focusing on reforming states, support for policy reforms and acceleration of human development and poverty alleviation, specially through the upgradation of state health systems. The project was also consistent with the strategic objectives of improving access and quality of health services to achieve these goals. The priority in upgradation of state health systems were mostly focused at the secondary level (lower level hospitals and FRUs), which was consistent with the sector strategies as reflected in the sector work: India: New Directions in Health Sector Development at the State Level: An Operational Perspective, February 1997. The main reasons for focusing on the secondary level were: (i) to complement investments made at the primary level by other IDA programs; (ii) treatment of the same conditions cost two-thirds of that at the tertiary level; (iii) inability to extend these services to the Primary Health Centre (PHC) level due to inadequate investment resources, low absorptive capacity, staffing issues and difficulty in providing emergency obstetric care at the primary level; (iv) increased credibility and hence increased utilization at the PHC level due to improved FRUs and referral systems; and (v) in-patient treatment at public hospitals was more equitable and that the poor were more likely to access government facilities for hospitalization. Also as discussed in section 8.5(a), government health institutions were the predominant providers of health care in the state, especially the poor and disadvantaged groups. A formal Quality at Entry Assessment by the Quality Assurance Group was not carried out during the time of project initiation for the Project. The project was designed in consultation with various stakeholders and there were adequate peer review assessments. The project design was consistent with the safeguard policies and necessary clearances were obtained. Key sector issues were identified and the project addressed many of these issues in its design. These issues were related to financing, performance of health systems, management, planning and coordination, access by vulnerable groups to health services, technical and quality issues. However, the project did not address to a similar extent some other key issues outlined in sector work, such as private sector involvement, incentives for the workforce, decentralization of administration, capacity building of Panchayati Raj Institutions (PRIs), community financing and health insurance. Few of these issues were also raised by the peer reviews but were not incorporated into the design as it may have increased the complexity of the project with relation to the capacities in the state. Lessons learned from previous State Health System project operations were taken into account and important issues were identified as potential problem areas, including funds

8

flow, empowerment and continuity of the Project Management Cell, preparation of plans in advance, etc. Some of these were addressed through legal covenants and project design, but few of these continued to be problems during implementation. There have also been key lessons identified and used from international experience (WDR 1993) which were a limited package of public health and essential clinical interventions are a top priority for government finance. The focus of the project was on essential clinical interventions given the poor state of clinical service delivery infrastructure in the state. Key risks were identified prior to the project and ratings were realistic in most cases. The mitigation measures identified were suitable for most of the risks but risks related to project management staffing and delayed implementation due to lack of capacity were not addressed as well. There had been good involvement of the beneficiaries, government, NGOs and Department for International Development (DfID) in the preparation of the project. DfiD has been supporting the health sector in Orissa over the past two decades. There were several studies conducted by the Government to facilitate the preparation of the project. 7.2 Implementation (including any project changes/restructuring, mid-term review, Project at Risk status, and actions taken, as applicable) Factors not under Government Control: There was a super-cyclone in October 1999 which disrupted 14 out of the 30 districts and left over 10,000 people dead. The Bank team was pro-active in its assistance during this period to the Government of Orissa. State elections held in February 2000 delayed the procurement process post the super-cyclone. There were incessant rains and flooding in July 2001 which carried on till September 2001 and affected the ongoing civil works. These natural disasters significantly affected the government machinery and caused a considerable shift of resources towards managing these disasters. This in turn caused implementation delays in project activities, most of which were beyond the control of the PMC, GOO and the Bank team during the first two years. Factors under Government Control: (i) Funds flow to the project was a problem especially in the first two years. This was later resolved by the opening of a Personal Ledger Account (PLA) after which funds flow considerably eased out; (ii) continuity of project leadership and staffing of the PMC was not maintained and project leadership changed three times during the course of the project; (iii) the decision not to hire a management agency which affected its capacity to handle key activities of the project; (iv) the Principal Secretary Health, GOO and the Director of Health Services also changed several times during the life of the project; (v) inadequate provision of manpower by the Directorate of Health Services (DHS) to the project facilities affected the staffing and utilization of these facilities during early years of the project; and (vi) availability of critical manpower like anesthetists, pathologists and obstetricians has been

9

a problem in the state itself. Factors subject to implementing agency control: The project team put in effort to implement project activities despite many factors outside its control affecting the project. Some of key factors that affected implementation under its control were: (i) the hiring of Industrial Development Corporation of Orissa (IDCO) as the initial Construction Management Consultant (CMC) which delayed the progress of civil works due to the poor performance of IDCO; (ii) the inability to put an effective monitoring system (HMIS) in place to ensure timely completion of project activities and monitor project outputs; (iii) lack of co-ordination of project inputs and under-utilization of the same; and (iv) not having developed a Financial Management System (FMS) for timely preparation of Financial Management Reports (FMR). Project Extensions: The project has been extended twice for a period of 12 months each on request by the GOO and the DEA and the project closed on March 31, 2006. The justification of the first extension was slow start up due to force majeure conditions, financial condition of the state, poor state of its health systems, the need to stay engaged in Orissa and a somewhat increasing state ownership towards the project. The justification of the second extension was the increased commitment shown by GOO during the previous year to improve project performance, to consolidate recent gains and to focus on specific areas like operationalization of equipment, staffing, implementation of systems for Operation and Maintenance (O&M), to make referral systems functional and to expand tribal district activities. Addressing these areas was important to maximize utilization and sustainability of project inputs and also to ensure these services were extended to the poor and disadvantaged groups. Project Reallocation/savings: In June 2003, with 9 months remaining for the project to close as per its original date, it was estimated that US$ 16.7 million would be the net savings from the project. These were due mainly due to savings from staff salaries, consultant services, operational and maintenance expenses, cancellation of the Health Department Building (Swasthya Bhawan), savings on equipment procurement and some exchange rate savings. In December 2004, $25 million equivalent (XDR 17.02 million) was diverted on request of the DEA to Polio eradication activities in India. Besides this reallocation, there were no other cancellations from the Interim Trust Fund Credit. The implementation of the project since the beginning had been rated satisfactory, though implementation had been considerably slower than expected, mainly due to factors beyond project control. The disbursement at Mid-Term review (MTR) was only 16% of the total amount and US$7 million was identified as net savings for the project. At the MTR, the Bank team focused on two key issues: (i) increasing disbursements by expediting activities related to civil works and goods; and (ii) addressing key staffing issues to improve service delivery and also increase capacities within the PMC to manage project activities. Project components were reviewed in detail and actions to

10

address the same were identified and agreed upon with the Bank team, which included many technical consultants. The MTR also identified the need for increased focus on software issues, involvement of Informal Service Providers (ISPs) and Non-Government Organizations (NGOs), involvement of district level bodies (Zilla Swasthya Samities) and the need to adequately staff the facilities. In October 2003, the implementation progress of the project was rated unsatisfactory and it was identified as a "problem project". Five risk flags were identified which were financial management, financial performance, slow disbursement (26.5% disbursed after 5 years), management problems and procurement performance. These issues were discussed with the GOO, including the Finance Secretary, and benchmarks to achieve most of these were agreed upon. Subsequently, a Bank team visited Orissa in December 2003 to determine the extent of compliance with the critical benchmarks agreed to during the October 2003 mission, and project implementation continued to be unsatisfactory. Discussions regarding the same were also held during the portfolio review meeting with the Country Management Unit (CMU), DEA, the Bank team and the project team. These frequent reviews at the national and state level combined with regular supervision helped increase the GOO and PMC's commitment to bring the project back to satisfactory status. In May 2004, the IDA mission reassessed project performance and agreed to upgrade three risk flags. In September 2004, all risk flags were removed due to satisfactory procurement performance and initiation of other agreed activities. 7.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization Design of M&E: The design of M&E data capture systems for collecting information and management decision making was found to adequate and consisted of monitoring tools like the Health Management Information Systems (HMIS), Disease Surveillance Systems and Financial Management Systems (FMS). It also included evaluation mechanisms in the form of baseline, mid-term and endline surveys. The monitoring systems were designed to function in an integrated manner and provide information ranging from the policy level to the facility level with the PMC holding the primary responsibility for M&E. Some of the project indicators were not well designed to measure accurately PDOs and project components: (i) some of the project indicators had a very broad scope and were not designed to accurately measure the activity intended (for e.g., increased number of institutions under the project utilizing Management Information Systems); (ii) some indicators did not clearly indicate the denominator to be used for measurement; and (iii) some of the indicators could not be measured due to a lack of data (for e.g., data on SC/STs was not captured at facility levels in Orissa and most other states due to concerns of sensitivity of the population in recording such data). However, other indicators can be used to measure project achievements in some of these cases.

11

Implementation of M&E: The implementation of the M&E systems was found to be inadequate. Though the HMIS was designed and developed by a reputed private contractor (Tata Consultancy Services), it never became truly operational due to the unavailability of the "source code" of the software developed. However, regular monitoring data was collected manually or through e-mail on the formats developed for the HMIS. There was no Joint Director (Statistics) in position as envisaged during project design to integrate information for effective decision making. The FMS for the project was also not developed and the project continued to submit manually prepared Financial Management Reports (FMRs). Baseline surveys were conducted during project preparation but the project did not conduct any endline surveys. However, endline impact assessment studies supported by the Bank included an equipment survey, healthcare waste management survey, Informal Service Providers (ISPs) survey and tribal strategy surveys. The surveillance system for major communicable diseases proposed under the project was designed to focus on seven major communicable diseases in Orissa, which was developed and was being used. Post the cyclone, World Health Organization (WHO) developed a more sophisticated Disease Surveillance System, which was supported by the Project. Utilization of M&E: A manual HMIS has been utilized by the project to monitor hospital indicators during the project but its sustainability after the life of the project appears to be highly unlikely. The Disease Surveillance System was utilized well during the floods that followed the cyclone and was found to be highly effective. It is very likely that this system will continue to be used after the project. 7.4 Safeguard and Fiduciary Compliance (focusing on issues and their resolution, as applicable) Tribal Strategy: Studies conducted during preparation indicated that the utilization of secondary and primary health services in tribal areas was low due to problems with physical, economic and social access of the tribals (22% of state population) to health services. The project developed a Tribal Strategy aimed at increasing the demand for primary health services in tribal areas by improving the quality of services and providing effective IEC to better inform tribal populations of the benefits of using government health services. Main findings of an implementation review of tribal strategy by the Bank in 2006 were: (i) the focus was more on upgrading the physical infrastructure and IEC activities were limited; (ii) additional money offered to doctors as an incentive to work in tribal areas proved to be inadequate; and (iii) nine mobile health clinics and about 5,000 annual health camps were organized to increase access to healthcare services. Since no endline survey was conducted, it is not possible to measure the impact of these interventions, including impact in reducing out-of-pockets expenditure for the poor in these areas. Later during the project, a program of training of Informal Service Providers (ISPs) in early referral of emergency obstetric/pediatric cases, management of fever and diarrhea has been undertaken in tribal districts. The main objective of this

12

program was to involve the ISPs, who were first points of contact with their communities, to provide linkages with the formal government health system and IEC to their communities. The review concluded that better health indicators and health seeking patterns amongst tribals could be achieved through this program, if it is supported and monitored by the local health administration. Environmental Issues: The project was rated as Category "B", due to the risks associated with the final disposal of health care waste and 85% of all waste was non-infectious and non-hazardous. Taking into account GOI's Bio-medical Waste Management Rules (1998), the project developed a Health Care Waste Management Plan (HCWMP) that included actions to be taken within the hospital premises and actions to be taken with regard to ultimate disposal outside hospital premises. A review of the HCWMP conducted in February 2006 by the Bank found that implementation was unsatisfactory. Though activities for the implementation of the plan at project facilities had been initiated, many activities that were required to achieve full compliance of the Rules were yet to be completed. These included: (i) poor segregation practices; (ii) deep burial pits were being dumped with mixed waste; (ii) poor supply and condition of consumables; and (iii) training was done only once knowledge among most hospital staff was found to be poor. Management and monitoring of the implementation of the HCWMP had also not been properly planned. It is understood that GOO has earmarked Rs. 10 million in its annual budget to complete these activities to achieve full compliance. Procurement: Procurement Arrangements: The PMC managed procurement for the entire project which involved civil works, goods and equipment and consultant services The project staff dealing with procurement were trained at ASCI and National Institute of Financial Management (NIFM) and staff were competent to handle procurement. However, the PMC was understaffed to manage procurement in areas like civil works. Goods and Equipment: Goods and equipment valued at Rs. 666.47 million were procured in three phases over the life of the project and was handled by ELMARC, a GOO undertaking. The initial procurement plan tried to synchronize the completion of civil works with the supply of goods and equipment, but this did not happen resulting in the non-commissioning of most of the equipment procured in the first phase. To avoid this situation, subsequent procurements of equipment were delayed until the last two years of the project. As a result of this and the delayed completion of the civil works in the larger facilities, a large amount of equipment could not be commissioned even till project closure. Frequent complaints and court cases during the procurement process also contributed to procurement delays although no complaints could be substantiated. Drugs and Consumables: In the initial phases, the project procured drugs and consumables following GOO's certification criteria. From the year 2003 onwards, the

13

project procured all drugs following WHO-GMP certification criteria, due to efforts from the PMC. This switching over to WHO-GMP certified drugs was completed despite resistance from various quarters of GOO. Civil works: The total value of civil works procured under the project, new construction as well as repair and renovation of existing buildings, was about Rs. 1,685 million and was mainly procured following NCB procedure. IDCO was selected on a single source basis for the management of the first 22 works to avoid initial delays, but they did not deliver these on time and this delayed the entire civil works component of the Project. Subsequently, seven design consultants and five Construction Management Consultant (CMC) firms were appointed under the project for the remaining 135 works for the design of facilities, inviting and evaluating bids and signing of contracts. Standard designs were not possible for the hospitals as most works involved the renovation of existing buildings, making the design process complex. Also, the PMC was understaffed and was unable to provide the kind of oversight necessary for the design and contract management tasks, leading to significant delays in completion. The issues that impacted on procurement under the project resulting in delays were: (i) Financial Management; (ii) Capacity of contractors to execute multiple contracts; (iii) repair and renovation jobs which involved demolition, were delayed to minimize disruption of hospital services; and (iv) delays in procuring other site requirements like electricity, water and sewage, telephones, etc. Financial Management/Disbursement: The overall financial management arrangements have been moderately satisfactory during the implementation of the project. To address the funds flow issues, a Personal Ledger Account (PLA) with check signing powers was given to the Project Director. However, large payments still required clearance from the Finance Department, especially in the initial years. The project was unable to successfully implement the FMS due to lack of perceived benefits and lack of skilled manpower to handle the FMS. However, manual books of account were maintained as required by the State's normal financial rules and monthly financial reports by project activity were regularly submitted, which served the purpose of financial monitoring reports. The project was also timely in submission of SOE claims especially in the last two years of the project when the level of expenditure picked up. Delay in submission of audit reports on a timely basis has been a problem in the project leading to suspension of Statement of Expenditure (SOE) based disbursement for FY04 and FY05. These suspensions were lifted soon after satisfactory submission of the respective audit reports. The delay in submission of audit reports was largely attributable to lack of adequate staff and agreed terms of reference for staff to deal with special purpose audit of all Externally Aided Projects. The funds drawn and deposited in the PLA were audited separately by the AG, resulting in a backlog of audit of project expenditure incurred from the PLA. The project has to furnish audit reports for the year

14

2005-06 as well as the audit for expenditures incurred from the PLA account since April 2004. The audit report is expected to be submitted after completion in October 2006. 7.5 Post-completion Operation/Next Phase (including transition arrangement to post-completion operation of investments financed by present operation, Operation & Maintenance arrangements, sustaining reforms and institutional capacity, and next phase/follow-up operation, if applicable) The project inputs have been brought under the Directorate of Health Services (DHS) which is a part of the Department of Health and Family Welfare (DOHFW). Allocation of financial resources to the primary and secondary sector and drug budget per bed show an upward trend which seems likely to be sustained. User fees collected at District Hospitals have increased and with the formation of facility based patient welfare committees called Rogi Kalyan Samitis (RKS) under the National Rural Health Mission (NRHM), greater increase can be expected as user fees are expected to be collected at lower level facilities also. Increased funding from central government sources through the Twelfth Finance Commission may also support maintenance of healthcare facilities. The Zilla Swasthya Samiti (ZSS) has been active in collecting user fees in the districts and some districts have also been fairly successful in out-sourcing non-clinical activities like cleaning and ambulance services. The Finance Department of GOO has agreed to sustain drug budgets and provide funds for annual maintenance of civil works and equipment provided under the project post the completion of the project. The GOO has also agreed to sustain the manpower, both contractual staff and additional posts, which had been added during the project and to maintain HCWM practices and innovative pilots like training of ISPs initiated under the project. All facilities, except for two under litigation have been physically completed, five of which have been completed after project closure with GOO funds. Completion of electricity connections and water supply in a few facilities remain and would be indicative of GOO's commitment towards the project. A full time Project Director has been retained to complete these and other activities. While funding for O&M of project inputs and activities is likely to be sustained by GOO, the actual sustainability of these project inputs and activities continues to be an area of concern. This is primarily due to the low motivation of the managerial, clinical and para-medical staff of DHS to sustain project inputs, activities and service improvements. This is evident from the fact that during the project: (i) only 40% of the total value of equipment provided was made operational; (ii) there was poor implementation of critical software activities, including HCWM practices; and (iii) no Operation and Maintenance (O&M) manual was presented to the Bank Team, though it was mentioned by the DHS that such a document was prepared. These issues have been raised strongly by the Bank team, especially in last one year of supervision, through regular follow-up and also by supporting a detailed equipment quality and utilization survey to document poor use of

15

these inputs. The Bank team has left specific recommendations with the PMC and Principal Secretary Health to address the same. The GOO has assured the Bank of its commitment to address this issue and have issued directives to the DHS for the same. The recommended time for any further impact evaluation would be after 2 years of the close of the project. This would help assess whether the project inputs and activities have been sustained and also whether the DHS has developed sufficient institutional capacities to carry on project activities. The manual MIS system is not expected to last beyond the project and hence it would be difficult to monitor the same using present project indicators. The Bank has kept engaged with Orissa through a state level policy dialogue and is discussing a follow-up operation to reduce child and maternal mortality with a multi-sectoral approach, for which the Bank is expecting a formal request from DEA. 8. Assessment of Outcomes 8.1 Relevance of Objectives, Design and Implementation (to current country and global priorities, and Bank assistance strategy) As discussed in section 7.1, the project has been consistent with the then CAS and the sector strategy. The project inputs and activities were focused at the lower hospital level and the FRU level, which primarily may have positively impacted maternal mortality and neo-natal mortality outcomes as discussed in section 8.2. Though there has been an increase in OPD utilization rates in children under five in project facilities, there is no data to support any significant linkages between infant and child mortality indicators and project outputs. Given the current GOI priorities as outlined in the NRHM and Reproductive and Child Health II (RCH II) programs and the Bank's current emphasis on supporting GOI in achieving the Millennium Development Goals (MDGs), the relevance of this project's design is weak in certain aspects. These aspects include decentralized planning, building of decentralized management capacities, greater linkages with PRIs, greater focus on multi-sectoral approaches, Public Private Partnerships (PPPs), monitoring outcomes and performance based funding. The project objectives did not emphasize on population based interventions to complement facility based interventions. With context to the present CAS (Report no. 29374, Sep 2004), this project's design would need to consider both the private sector and the public sector towards achieving better health outcomes and consider mechanisms to improve risk protection, especially for the poor. A Quality Enhancement Review (QER) was carried out on July 31, 2002 to review the Bank's experience with the India State Health System Projects (SHSPs) noted that in new project designs: (i) there was greater need to monitor outcomes; (ii) have greater focus on targeting the poor; (iii) more state and situation specific project designs; and (iv) increased involvement of the private sector in new projects. Analysis of the National

16

Statistical Survey Organization's (NSSO) population based survey data indicates significant increase in hospitalization rates in both public and private facilities in Orissa within a period of ten years (1995-96 to 2005-06). The increase seen in the private sector shows that new project designs should incorporate more strongly the private sector in delivering healthcare to the population of Orissa (please refer to Annex 4 for more details). As outlined in section 7.3, the project indicators were not well designed to measure project objectives. The M&E design would also need to be improved to reflect HNP outcomes as goals, with specific provision for their measurements, including for the poor. The objectives of the project may still be relevant to achieving outcomes but the indicators used to measure the same needs to be more focused on the population and the end-user rather than facility based process and output measures. 8.2 Achievement of Project Development Objectives (including brief discussion of causal linkages between outputs and outcomes, with details on outputs in Annex 4) The achievement of project development objectives is rated as moderately satisfactory. This rating is based on the progress measured on key performance indicators as outlined in Annex 1 and 4, NSSO survey data (a national population based survey) and analysis based on RCH survey data. Even though some of the indicators may not have been well designed to accurately measure the activity intended, other indicators can be used to measure achievement of the same. Details of achievements of PDOs and components are given in Annex 4. Change in selected health outcome indicators in Orissa, based on the two rounds of district level RCH surveys, is as follows: (i) a reduction of Infant Mortality Rate (IMR) by 13/1000 from 77/1000 in 1998-99 to 64/1000 in 2002-04; (ii) a reduction of neo-natal morality rate by 13/1000 from 57.3/1000 in 1998-99 to 44.3/1000 in 2002-04; and (iii) an increase in institutional deliveries by 12% from 23% in 1998-99 to 35% in 2002-04. The project outputs may have positively impacted maternal mortality and neo-natal mortality outcomes by contributing to an overall increase in institutional deliveries in the state. Though there has been an increase in OPD utilization rates in children under five in project facilities, there is no data to support any significant linkages between infant and child mortality indicators and project outputs. DO 1: Improve efficiency in the allocation and use of health resources through policy and institutional development: Achievement of this DO is moderately satisfactory. (i) Increased share of resources for the primary and secondary levels of health care in the total resources (plan and non-plan) allocated to the health sector, until year 2002: Achievement in this regard is satisfactory. The project did track the proportion of the

17

health budget (Plan and non-Plan) allocated to the primary and secondary levels till 2003-03 and it rose from 83% in 1998-99 to about 85% in 2002-03, indicating a 2% increase in share and a 70% increase in actual allocations (in real terms). Combined allocations in real terms to the primary, secondary and tertiary health sector (Plan and Non-Plan) without project funds have increased substantially by 64.3% between 1998-99 and 2005-06. This indicates that the increase in Health and Family Welfare (H&FW) budget is likely to be sustained even without OHSDP funds after project closure. (ii) Increased resources for drugs, essential supplies and consumables in accordance with agreed norms per inpatient per bed per year (Rs. 3.500 at baseline): Achievement in this regard is moderately satisfactory. The drug budget per in-patient per bed per year has increased from Rs.3,500 at baseline to Rs. 7,479 in 2004-05 - an 84.3% increase in real terms. However, actual allocations towards drugs and consumables by the state government under all programs in real terms increased by 14.15% between 1998-99 and 2005-06. During the same period OHSDP saw a 219.1% increase in real terms and therefore the effective increase in drug and consumables budget of the state through its own resources was only 5.29% increase. This shows that although there has been an increase in drug and consumables budget per bed in the state, there has not been a significant increase in the entire state budget for drugs and consumables without project funds. (iii) Implementation of a user charge policy: Achievement in this regard is satisfactory. User fee collections went up from Rs. 3.8 million in FY 99 to Rs. 30.4 million (Rs. 26.3 million in real terms) in FY 06, an increase of 601.6% in real terms. Given the high levels of poverty in the state, it was decided to levy user charges only at the District Hospital (DH) level. Inspite of this, there has been a significant increase in collection of user fees in the state. This increased collection was applied towards contracting out cleaning services, procuring ambulance services and purchasing additional drugs and consumables, all intended to improve quality of care. Overall, there has been an increase in the state budget allocation to primary and secondary levels, an increase in the per bed drugs and consumables expenditure (although mainly financed by the project) and successful implementation of a user fee policy. Based on these findings overall this DO is rated moderately satisfactory. DO 2: Improved quality, effectiveness and coverage of health services: Achievement of this DO is moderately satisfactory. (i) Improved quality: Increase in percentage of institutions staffed in accordance with agreed norms, particularly in tribal districts, from 25 % at baseline; to 75% at end of project. This indicator measured the number of institutions that had the required staffing norm out of the total number of institutions identified for this staffing norm. Achievement in this regard is satisfactory. About 75% of project facilities were staffed

18

with doctors according to norms, and 80% with paramedical workers according to norms at the end of the project. GOO has committed to continuing 854 contractual staff beyond the project period and an additional 223 posts of doctors and nurses that have been created and filled up under the project and will be continued. However, there are still vacancies in key areas such as anesthesia and pathology; this is due to an absolute shortage in the availability of such skills in the market. There appears to be an overall equitable distribution of institutions staffed according to norms among tribal and non-tribal populations. The other indicators used to measure this were also found to be satisfactory. Total number of laboratory tests conducted increased by 52.5% for in-patients during the project period and 61.5% for out-patients during the project period. (ii) Improved effectiveness: Increase in number of admissions to institutions under the project due to pregnancy-related complications from 20% at baseline, by 5 percentage points by mid-term and 15 percentage points by end of project: This indicator is not very specific on the denominator used to calculate the figures and the denominator does not seem to be based on either IP admissions or pregnancy related admissions. However, admissions due to pregnancy related complications have increased in aggregate, from 45,241 across all hospitals in 1997-98 to 72,123 in 2003-04 - an increase of 60%; and all categories of hospital have reported large increases over the baseline. There is also an increase in the number of pregnancy related admissions from 99,160 in 1998-99 to 178,693 in 2005-06; an increase of 80.2%. Hence, achievement in this regard can be rated as moderately satisfactory. Increase in the number of institutional deliveries in institutions under the project from 16% at baseline by 5 percentage points at mid-term and 15 percentage points at endline: This indicator too does not specify the denominator to be used for calculation. However, using other measures outlined below this indicator is rated as satisfactory. There has been an 81% increase over baseline in the number of institutional deliveries at project facilities between 1997-98 and 2005-06. In 1999 about 860,000 births occurred in Orissa; NFHS II (1999) reported that about 22.9% of these were institutional deliveries. About 98885 deliveries took place in project hospitals in 1998-99 indicating that about 50% of all institutional deliveries in Orissa took place at project hospitals. Hence, the increase in institutional deliveries at project facilities may have contributed to the overall increase in institutional delivery in the state by 12% between 1998 and 2005, as indicated in the RCH data. The other indicators used to measure this were found to be satisfactory, including a 51% increase over baseline in the number of surgeries at project facilities between 1997-98 and 2004-05 and an increase in the number of under-5 OPD admissions by 104% over the baseline. NSSO data indicates an increase in hospital admissions in both rural and urban areas by 85% and 159% respectively between 1995-96 and 2005-06. In rural areas, admissions in government facilities grew by 61.8% and in private facilities grew by 312% during the same period. The data indicates an overall increase in access to hospitals in rural and

19

urban areas, with significant increases in both private and public sector. (iii) Increase outpatient attendance in total, among women and STs, from 30% for women and 10% for STs at baseline: The project has not been able to disaggregate numbers by gender and tribal status; it is therefore not possible to report on increases in OPD attendance for those groups. However, social assessments at baseline show that these facilities were being used by the poor. Total OPD attendance increased substantially from 19,485 OPD per day in 1997 to 33,616 OPD per day in 2005 - a 72.5% increase during the project period, despite disruption of services at all hospitals for various periods of time due to on-going construction and renovation. This may have resulted in increases in out-patient attendance for women and STs as well.

There has been an increased utilization of these facilities as indicated by facility based and population survey based data. Key services including OPD for children, pregnancy related admissions, institutional deliveries, in-patient admissions and laboratory tests have shown significant increases. However, there is no evidence to indicate whether this has benefited the poor, though social assessments at baseline show that these facilities were being used by the poor. Quality of services has shown improvement in the facilities though the large amount of equipment (about 60% of the value) not utilized at the close of the project indicates that more can be achieved in this area. Hence, this achievement of this DO is moderately satisfactory. 8.3 Efficiency (Net Present Value/Economic Rate of Return, cost effectiveness, e.g., unit rate norms, least cost, and comparisons; and Financial Rate of Return) The project did not calculate any traditional measures of efficiency like Net Present Value (NPV) and Economic Rate of Return (ERR) and Financial Rate of Return during the time of appraisal. This has not been calculated for most sectoral projects prepared at the time. However, during the appraisal, there has been a review of the fiscal situation in the state, budgetary allocations of the health sector, discussion of some cost-effectiveness assumptions and project financial sustainability. Some measures of efficiency have been calculated as below with details in annex 5. As discussed in previous sections, the project focused on improving resource allocations within the health sector and improving performance of the health system. In terms of improving efficiency in resource allocations, the project managed to satisfactorily increase budgetary allocations to the primary secondary sectors, budget for drugs and consumables per bed and collection of user fees. As discussed in section 8.2 and annex 4, the H&FW budget for primary and secondary levels has increased by 70% in real terms during the life of the project. Also, H&FW Budget without project funds increased by 64.3% in real terms. The incremental increase of the total H&FW budget over the lifeof the project was 4.79 times the entire OHSDP budget in real terms and the incremental

20

increase of the H&FW budget for the primary and secondary levels over the life of the project was 4.14 times the entire OHSDP budget in real terms (Please refer to Annex 4 for details of calculation). This shows that the project investments may have been efficient in improving resource allocations. In terms of improving health system performance, the main assumption for cost effectiveness given at the time of appraisal was that cost of providing services at the secondary and FRU level of care was 25-40% cheaper than providing the same at tertiary levels. Using that basis it is estimated that cost savings of Rs. 160 million have taken place during the project with an increasing trend. Given the total project inputs of Rs. 2.54 Billion for these interventions and assuming a 25 year life of the project, it is likely that the benefit from the costs savings would equal 60-70% of the initial project inputs. This calculation does not take into consideration that this project may have actually decreased cost per unit of bed day in secondary facilities due to increased utilization and hence is a conservative estimate. It also does not factor in increased costs to the patient in accessing a tertiary facility, improvement of quality of services and other services the population would benefit from, including critical areas like handling of obstetric emergencies and institutional deliveries. 8.4 Justification of Overall Outcome Rating (combining relevance, achievement of PDOs, and efficiency) Rating: Moderately Satisfactory The project has achieved its objectives related to improved resource allocation at the state level and increased utilization of facilities by the population, including increase in institutional deliveries. It also seems to have been efficient in achieving improved resource allocation and has also achieved some cost savings by increased utilization at the secondary level of care. It has managed to improve the quality of services to a lesser extent but significant amounts of equipment have not been utilized till project closure. Given the present context, the project design would have to address some key issues such as private sector participation, performance based funding, decentralized planning and management and have stronger links to outcomes in its design to make it more relevant. Also, some project indicators were not well designed to measure accurately PDOs and the activities intended, and M&E design would need to be improved. Hence the project outcome is rated as moderately satisfactory. 8.5 Overarching Themes, Other Outcomes and Impacts (if any, where not previously covered or to amplify discussion above) (a) Poverty Impacts, Gender Aspects, and Social Development A social analysis during project preparation indicated that: (i) about 44% of the state's population lived below the poverty line; (ii) almost 40% of the state's population was comprised of SC/STs; (iii) the burden of communicable disease in Orissa, at 62% was 10

21

percentage points higher than the average for India; (iv) an analysis of hospital activity indicators showed that government health institutions were the predominant providers of health care in the state, particularly in rural and tribal areas and for the poor (of those using the facilities to be included in the project, 32% were below the poverty line, 69% were either SC/ST or from the Backward Castes and 39% were women); and (v) several indicators of the availability and accessibility of health facilities were much lower in Orissa than in other Indian states (only 54% had a government or private health facility within 5 kms. from their residence, about 20% of them had to travel more than 20 kms. to reach any medical facility and only 30% had access to affordable public transport). These statistics suggest that a project that successfully improved access and quality in lower level hospitals and first referral facilities could potentially have a significant benefit for the poor. At the end of the project, a survey conducted by the Bank indicated that 75% of the facilities were at a referral distance of less than two hours to the next level of care. The improvements in availability of staff, including doctors, at these facilities, increased utilization rates and improvements in supplies and equipment are likely to have improved health service delivery to the poor, SC/ST and women accessing these facilities. In addition, in selected areas the project supported involving ISPs to refer patients to facilities, which may have benefited populations in these tribal areas. Even though this was started as a pilot initiative, it managed to garner support from the political leadership, which may be an indication of continued GOO support for this effort. Since an end-line survey has not been conducted, it is not possible to make exact comparisons on changes in health seeking behavior, or physical and economic access of health care facilities at end of project. (b) Institutional Change/Strengthening (particularly with reference to impacts on longer-term capacity and institutional development) The main contribution to institutional development/strengthening by the project were: (i) Civil Works: Due to the extensive and relatively complex nature of the civil works undertaken in the project, the capacity of the engineering division of Public Works Department (PWD) of GOO is likely to have substantially increased. At the beginning of the project, there were limited capacities in the PWD to design and plan for civil works in the project facilities. Seven external design consultants were assigned to the project who worked with the PWD engineers thus increasing the technical capacity of the engineering division to ensure good quality civil works in the state. (ii) WHO-GMP certification for drugs: As outlined in section 7.4, the project procured all drugs following WHO-GMP certification criteria from the year 2003 onwards and it is to the credit of the PMC that this switchover was successfully accomplished. Drugs are now procured in the state using WHO-GMP certification, which is likely to result in

22

better quality of drugs being procured. (iii) Procurement of goods and equipment: Procurement of goods and equipment was handled by ELMARC, a GOO undertaking for the PMC. A study was conducted by the Bank near project closure and it was found that only about 4.4% of entire cost of "major" equipment procured was found to be of quality levels below the acceptable range. The extensive range and large amounts of procurement done during the project is likely to have built better capacities within ELMARC, and hence GOO, to undertake such procurement. (iv) Disease Surveillance System: The disease surveillance system developed by WHO and supported by the project functioned well. It was found to be effective post the floods that followed the "super-cyclone" and is still functional. It is highly likely that this system will continue to be effectively used. (v) Drug distribution system: The project engaged in terms of dialogue with GOO for the establishment of a central drug distribution system, an activity that was supported by DfID. The system is now well established and working well at the district level, resulting in improved availability of drugs at all levels of the health system. (c) Other Unintended Outcomes and Impacts (positive or negative, if any) Shift to government OPD facilities in urban areas: As discussed in Annex 4, NSSO data indicates a 13% fall in share of population accessing private OPD facilities in urban areas compared to a 12% increase in share of population accessing government OPD facilities between 1995-96 and 2005-06. This maybe due to the population preferring the improved government facilities under the project in the urban areas compared to private facilities. It is possible that the non-poor increased utilization of these improved government facilities but there is no data available to indicate which income categories saw increases to these facilities. 8.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops (optional for Core ICR, required for ILI, details in annexes) N/A 9. Assessment of Risk to Development Outcome Rating: Moderate Risk to outcome is rated as moderate. The reasons for the rating is discussed below: (i) Financial: The allocation of resources for primary and secondary levels of care, drug budget per bed and implementation of a user charge policy are likely to be sustained and

23

pose low risk to outcomes. The sixth pay commission that is being constituted presently may affect the fiscal situation of the state due to the expected increases in disbursement of salaries, which could affect future health budget increases. (ii) Commitment of GOO: As outlined in section 7.5, the financing for maintenance of project inputs and facilities is at low risk. The GOO has also agreed to sustain the manpower that has been added through the project. The commitment of GOO has also been shown in the physical completion of five out of the remaining seven facilities at project closure using its own funds (the remaining two are under litigation). (iii) Institutional: The commitment of the DHS towards the project has been low. The risk to sustaining project activities such as HCWM and HMIS is substantial, despite its stated commitment. Operationalization of equipment continued to be a problem at project closure with only 40% of the value of the major equipment being operationalized at project closure. Maintenance of equipment too is at substantial risk with no O&M guidelines being put into place during the project and low ownership of the project inputs by the end users - doctors, nurses and technicians. However, these issues have been taken up by the Bank team with GOO and the GOO is committed to address them. Utilization rates of the facilities have increased during the project despite the non-operationalization of large amounts equipment. Hence, utilization of these facilities by the population is at moderate risk despite the substantial risk of equipment not being maintained by the DHS. (iv) Political: The political leadership at the state level seems to be stable at project closure and shows an increasing commitment to improving health outcomes. There is increased naxalite activities in the tribal districts may affect provision of health services in affected areas. In balance, political risk is rated as low. 10. Assessment of Bank and Borrower Performance (relating to design, implementation and outcome issues) 10.1 Bank (a) Bank Performance in Ensuring Quality at Entry (i.e., performance through lending phase) Rating: Moderately Satisfactory A formal Quality at Entry Assessment by the Quality Assurance Group was not carried out during the time of project initiation for the Project. The Bank's performance in ensuring Quality at Entry is rated as moderately satisfactory. The project design was consistent with the CAS and the sector strategy at that time, which was also reflective of GOI's priority. The selection of states for intervention were based on several criterion including commitment from the Government, poverty levels, GOI priority, implementation capacity, status of project preparation, etc. The project was designed in

24

consultation with various stakeholders and several technical, environmental, social, financial and economic studies were used for the preparation of the project. It was consistent with the safeguard policies and necessary clearances were obtained. An Investment Lending instrument was used for the project and choice of instrument was suitable. In terms of skills, there was a good choice of consultants and Bank staff used in project preparation. Lessons learnt from previous and ongoing projects were taken into account, especially in the preparation of implementation and financial arrangements, which were made Conditions to Negotiations. The general sector issues and how they were to be addressed by the project, including the strategic choices, were well identified. However, the design of the project was similar to other SHSPs despite the differences between these states, including the high level of poverty and tribal population in Orissa compared to those states. The project does not incorporate as well into the design some key issues such as private sector involvement, incentives for the workforce, decentralization of administration, community financing and health insurance even though they have been identified as key interventions for not addressing sector issues. Though some of these were reflected in the State Policy Matrix and reasons for not addressing some of the same were given, it could have been better integrated into the design. Though the design of the date capture systems for M&E were well designed, some of the project indicators were not well designed to measure accurately PDOs and project components. (b) Quality of Supervision (including of fiduciary and safeguards policies) Rating: Moderately Satisfactory The Bank's performance in ensuring Quality of Supervision is rated as moderately satisfactory. There was change of TTL only once in early years (1999) of the project and a single TTL for the rest of the project gave it stability from the Bank side. There was a Quality of Supervision Assessment carried out in 1999 which rated the quality of supervision as satisfactory and commended the supervision team for (i) use of experience from other projects in India; and (ii) being well led, with a professional approach towards problem solving. There was another Quality of Supervision of Risky Project undertaken by QAG for this project in 2000-01 and rated the quality of supervision as highly satisfactory. It commends many aspects of the supervision strategy, especially in difficult environments like Orissa, like (i) field based supervision by highly qualified staff; (ii) regular supervision visits, almost once in every 2-3 months; and (iii) use of highly qualified consultants in areas like social development, civil works, equipment and IEC. There was a proactive approach in identifying problem areas, suggesting solutions and agreeing on benchmarks. There were surveys conducted by the Bank Team (equipment survey) to study impact of innovative measures (ISP survey) and the use of project inputs. There were regular supervision missions with procurement, financial

25

management, social development and other technical experts participating in these missions. Quality of reporting in Aide-Memoirés and ISRs were found to be adequate with proactive identification of problem areas and solutions to address the same being given. Risk ratings were realistic most of the time, but there could have been a more realistic assessment of risk in certain areas (for e.g., delay in setting up project Management team in place). Though some of the project indicators were not well designed to measure accurately PDOs and project components, there was no attempt to change these during supervision. As discussed in section 7.2, the first two years saw implementation delays in project activities, most of which were due to natural disasters and beyond the control of the PMC, GOO and the Bank team. There was flexibility and client responsiveness shown during the time of natural disasters where the Bank team was open to restructuring the project and also brought in experience in dealing with the situation. The project still continued facing implementation delays after that and even though only 16% had been disbursed at MTR (March 2002), the project was not rated unsatisfactory until October 2003. $7 million of net savings was identified for possible cancellation during the MTR but this later became a part of the reallocation made to polio eradication activities. Post the unsatisfactory rating, monitoring was intensified by the Bank team through frequent review missions and different forums like the portfolio reviews and the State Health System workshops to improve implementation and resolve the problem project status. This helped increase the GOO and PMC's commitment to bring the project back to satisfactory status. (c) Justification of Rating for Overall Bank Performance Rating: Moderately Satisfactory The Bank team ensured that the project design was consistent with the Bank strategies and the preparation processes at the time. The choice of interventions was to complement other investments in the health sector and the design may have contributed to improving some health outcomes in Orissa. However, project design could have been made more specific for Orissa and there could have been better selection of project indicators in some cases to better measure these outcomes. The implementation delays in the first two years of the project were mostly beyond the control of the Bank team. Though the Bank team rated the project unsatisfactory only in Oct 2003, intense monitoring by the Bank team after that helped in speeding up implementation progress. Hence the overall performance of the Bank is rated as moderately satisfactory. 10.2 Borrower (a) Government Performance Rating: Moderately Satisfactory Performance during preparation: The GOO's performance at entry is rated satisfactory.

26

There were several studies conducted by GOO to facilitate the preparation of the project. These included economic studies, technical studies, beneficiary assessments, social assessments, facility survey and equipment surveys. A PIP was completed and presented to the Bank team during the Appraisal Mission. A Health Sector Development Policy Letter was also presented to the Bank which focused on areas like resource allocation, public-private partnerships, and incentives for the workforce and implementation of a user charge policy. Appropriate action plans for rationalization of services, drug procurement, HMIS and equipment management were completed and presented to the Bank. GOO also fulfilled the necessary fiduciary and safeguard requirements of the Bank and presented the financial management arrangements, procurement arrangements, tribal strategy and environmental action plan to the Bank. There had been good involvement of the beneficiaries, government, NGOs and DfID in the preparation of the project. Performance during Implementation: The GOO's performance during implementation is rated moderately satisfactory. The Government helped the project achieve its DOs related to improved resource allocations at the state level in H&FW for primary and secondary services and drug budgets. The GOO is also committed to sustain funding for all project activities after project closure. However, the DHS has shown low motivation in implementing various project activities, including operationalization of equipment and HCWM. The project also had implementation delays and had to be extended twice for a period of one year each. In the first two years of the project, the reasons for these delays were caused due to several factors mostly out of GOO's control. However, some of the other delays were under GOO's control, which are outlined in section 7.2. The commitment of GOO towards the project increased in the last three years of the project improving completion and sustainability of project activities. Hence GOO performance is rated as moderately satisfactory. Despite the satisfactory performance of GOO during the preparation, the overall performance of GOO is rated as moderately satisfactory given the delays in implementation of the project. (b) Implementing Agency or Agencies Performance Rating: Moderately Satisfactory Implementing Agency

Performance

Project Management Cell, OHSDP

The performance of the PMC is rated as moderately satisfactory. Project activities were affected by natural disasters in the first two years, which caused implementation delays. The PMC were able to complete most project activities, despite several factors beyond its control. Most civil works have been completed satisfactorily, despite the low capacities in the state to undertake good quality civil works in the health sector.

27

However, the decision to use IDCO as a CMC initially for the project significantly affected the civil works activity due to poor performance by IDCO. There was a lack of coordination of project inputs during the early stages of the project leading to expiry of warranties on equipment without their utilization. The project managed to procure most equipment with acceptable quality for the project and introduced drug procurement using WHO-GMP certification in the later stages of the project. However, installation and operationalization of equipment was inadequate and only 40% of the value of equipment procured was operational at project closure. Software activities which were successfully undertaken by the project included the innovative pilot for training of ISPs, equipment training and clinical skills training. However, implementation for other activities like HMIS, HCWM and IEC were found to be inadequate. Many of the operationalization issues and poor implementation were caused due to the poor ownership of the DHS and which was beyond the control of the PMC.

(c) Justification of Rating for Overall Borrower Performance Rating: Moderately Satisfactory The Borrowers performance during preparation was satisfactory with necessary studies and plans being conducted. There was also consistency with the project preparation process with all necessary clearances being obtained and a high level of commitment shown by the Borrower. Most of the factors causing implementation delays were beyond the Borrower's control for the first two years of the project. However, post that there factors that caused delays that were within the Borrower's control. The Borrower showed an increased commitment in the last three years of the project to improve completion and sustainability of project activities. Hence Borrower performance is rated as moderately satisfactory. 11. Lessons Learned (both project-specific and of wide general application) Institutional: (i) Assess and address state and project management capacities: Despite having addressed the issue of project management and funds flow arrangements in the project design, these continued to be a problem in the early stages of the project. The capacity of the PMC to plan, execute and monitor various project activities, including civil works and procurement, was significantly affected due to the decision of the government not to contract a management agency post project effectiveness. There were also delays in submission of audit reports and responses to audit disallowances due to lack of state capacity. It is recommended that the Bank assesses these capacities and agree with the

28

Borrower on a time bound plan to address the same during project preparation itself. (ii) Accountability also important for results: During field visits, it was observed that there were differences in implementation progress across the state despite the same incentives given to project managers. Hence, it is also recommended that issues of accountability and motivation are also studied and addressed during project preparation. (iii) Develop local capacities to use fees collected: The collection of user fees has increased the availability of incremental budgets for non-recurrent expenditure at the facility level, which is likely to positively affect the quality of services provided. However, there is a need to develop local structures like the RKSs at the facility level that should be able to use these funds for the local population and be accountable to them. Operational: (i) Synchronize inputs to prevent delays: It is important to carefully plan phasing and synchronization of project inputs to avoid implementation delays and improve achievement of outputs. It is also recommended that alternative options to avoid such delays be generated through a scenario planning exercise during project preparation itself. Activities such as civil works, which affects implementation of other activities, that are not following schedule should be expedited with fixed timelines or cancelled from the project. (ii) "Software" should not wait for the "hardware": Software activities like IEC should be initiated in the early stages of the project irrespective of the status of the completion of civil works. This gives enough time for the project to implement these activities and generate awareness amongst the population to use these facilities. Innovative pilots too should be initiated in the early stages of the project to give sufficient time period to assess the impact of these interventions and put in place arrangements to scale up these interventions if the impact is positive. (iii) HCWM is a behavioral change: Change in HCWM practices takes time and therefore constant training and monitoring is required by the project and the DHS to inculcate this behavior. This needs to be a continuous process and supply of HCWM consumable material should be continually provided or procured using ZSS funds. (iv) Prompt identification of problem status important for improvement: It was observed that execution of project activities and disbursements significantly improved after it was identified as a problem project in October 2003, despite the delay in identifying it as a problem project. This indicates that proactive identification of problem status and realistic ratings are important for successful project execution.

29

Project Design: (i) M&E design and implementation needs more attention in projects: Project indicators need to be designed to effectively measure DOs and ensure availability of accurate baseline data. Incase, the indicators are found to be ineffective in measuring the intended activity during implementation, changes in such indicators should be considered. Information systems like HMIS and FMS should be made operational early on in the project to enable timely receipt of monitoring reports, including FMRs. (ii) District specific planning maybe more suitable: As the study on child mortality in Orissa notes, there are significant differences in health indicators between districts. Hence, district specific planning would seem to be more appropriate compared to a uniform set of activities being implemented across the state to improve health outcomes. (iii) Technical assistance to improve implementation: Attention needs to be given to the provision of technical assistance (TA) and building of various capacities at the state and district levels. TA should be provided for in the areas of project/program management, organizational restructuring, public health and other specific areas like insurance and PPPs, if required. It is also recommended that strong implementation support is also provided along with funding for TA in weaker states as government effectiveness in using this funding for TA is usually low. (iv) Engage further the private sector: As outlined in section 8.2, given the growth of the private sector in Orissa, it is important to engage more actively with the private sector and go beyond the contracting out of just "hotel" functions. Though the commitment of GOO to strengthen the public sector is warranted given the large number of poor still accessing these facilities in the state, it is also important to engage the private sector in healthcare given its increased presence. The focus of the PPPs should be on improving access to and quality of healthcare in remote and tribal areas where the public sector may have poor reach and contracting in of specialist services to provide comprehensive emergency obstetric and neonatal care. 12. Comments on Issues Raised by Borrower/Implementing Agencies/Partners (a) Borrower/implementing agencies (i) With reference to the Department of Economic Affairs letter dated June 13, 2006, there have been changes made to the comments of the OHSDP authorities. These changes were primarily made to OHSDP comments on: (i) diversion of $25 million towards polio eradication activities; (ii) extension of the project for a further 3 months; and (iii) Bank team relationship with Borrower during supervision missions.

30

(ii) The Borrower has indicated that the diversion of $25 million from the project to Polio eradication activities has affected the completion of project activities as envisaged in the PIP. It is unlikely that diversion of these funds affected completion of the project as most of this amount except for $3 million came from net savings of the project. As discussed in section 7.2, in June 2003, it was estimated that $ 16.7 million would be the net savings from the project. In December 2004, $25 million equivalent (SDR 17.02 million) was diverted on request of the DEA to Polio eradication activities in India. The project was extended twice for a period of 12 months each and the project finally closed on March 31, 2006. At the time of project closure, there was a request from the GOO to extend the project by 3 more months and for reprovision of $3.5 million for completion of project activities. In a letter dated February 9, 2006 sent by the Bank to the Chief Secretary of Orissa, the Bank noted that the credit shortfall of the Bank's share was estimated to be only $3 million. It also noted that: (i) given the small amount; (ii) the project's overall weak performance in terms of operationalization of equipment; and (iii) the Bank's ongoing plans for significant support to Orissa's finances, it found it difficult to supplement the present project. Therefore, GOO was requested to fulfill this small financing gap from its own resources. (iii) Due to the slow implementation progress and potential problem status of the project, the Bank team had to agree on benchmarks, which may have been perceived to be over ambitious by the PMC in retrospect. The aide-memoires indicate delayed timelines and new agreed benchmarks as in other projects. (iv) There were delays in procurement clearances in some cases as reported by the Borrower. The delays in clearance of building design specifications were mainly due to a cost containment exercise that needed to be done following cost escalations. Delays in clearing technical specifications for equipment took place due to the complaints received by the Bank which needed to be addressed adequately before the Borrower could award the contract. A report of the Comptroller and Auditor General of India, 2005 on the project also highlights some of the issues mentioned in this report, including funds flow issues, problem with CMCs, quality and delays in civil works, quality and operationalization of equipment, training and HCWM issues. (b) Cofinanciers N/A (c) Other partners and stakeholders (e.g. NGOs/private sector/civil society) No comments.

31

Annex 1. Results Framework Analysis Project Development Objectives (from Project Appraisal Document) The objectives of the project were to assist GOO to: (i) improve efficiency in the allocation and use of health resources through policy and institutional development; and (ii) improve the performance of the health system through improvements in the quality, effectiveness and coverage of health services at the secondary level and selective coverage at the community level, so as to improve the health status of the people, specially the poor, by reducing mortality, morbidity and disability. Revised Project Development Objectives (as approved by original approving authority) The development objective was not revised during project implementation. However, there was a reallocation of US$25 million (XDR 17.02 million) to assist GOI in eradicating poliomyelitis, which came from the net savings of the project. This was communicated to the Department of Economic Affairs (DEA) by IDA on December 23, 2004 and had been made effective from April 1, 2004. This had supplemented the then ongoing Immunization Strengthening Project (Cr. 3340-IN) by financing "polio eradication activities" that included Oral Polio Vaccines (OPV), training, orientation, honorarium, consultant services, mobility support and Information, Education, Communication (IEC) material. The ICR report for the Immunization Strengthening Project would assess the achievement of this objective. (a) PDO Indicator(s)

Indicator Baseline Value

Original Target Values (from approval

documents)

Formally Revised Target Values

Actual Value Achieved at Completion or Target

Years

Indicator 1 : Increase resources for drugs, essential supplies and consumables in accordance with agreed norms per in-patient per bed per year.

Value (quantitative or Qualitative)

Rs. 3,500 Rs. 5,500 Rs. 7,974

Date achieved 01/01/1998 03/31/2006 03/31/2005 Comments (incl. % achievement)

The drug budget per in-patient per bed per year has increased from Rs.3,500 at baseline to Rs.5,783 in 2000-01 and to Rs.7,974 in 2004-05 (Rs. 6447.2 in real terms) - a 84.2% increase over baseline.

Indicator 2 :

Increase in number of institutions staffed in accordance with norms from 0 at baseline, by 25% at mid-term and 75% at end of project.

32

Value (quantitative or Qualitative)

0 75% (at end of project)

70% of project facilities are staffed with doctors, and 80% with paramedical workers in according to norms

Date achieved 01/01/1998 03/31/2006 11/30/2005 Comments (incl. % achievement)

This indicator measured the number of institutions that had the required staffing norm out of the total number of institutions identified for this staffing norm.

Indicator 3 : Increased share of resources for the primary and secondary levels of health care in the total resources (plan and non-plan) allocated to the health sector, until year 2002

Value (quantitative or Qualitative)

83% Increase upto year 2002 85%

Date achieved 03/31/1999 03/31/2002 03/31/2003 Comments (incl. % achievement)

No target specified. H&FW budget for primary and secondary levels has increased from Rs. 3134.6 million in 1997-98 to Rs. 7441.6 million in 2002-03 -- a 70% increase in real terms.

Indicator 4 : Increase in number of admissions to institutions under the project due to pregnancy-related complications from 20% at baseline by 15 percentage points at the end of project.

Value (quantitative or Qualitative)

20% 35% over baseline 9.8%

Date achieved 03/31/1998 03/31/2006 03/31/2004 Comments (incl. % achievement)

This indicator is not specific on the denominator used to calculate the figures. If new in-patients are used as the denominator, then baseline was 12% and 9.8% was target achievement in 2003-04.

Indicator 5 : Increase outpatient attendance in total, among women and STs, from 30% for women and 10% for STs at baseline

Value (quantitative or Qualitative)

30% for women, 10% for STs

10 and 5 percentage points increase over baseline for women and STs respectively

Date achieved 03/31/1998 03/31/2006 Comments (incl. %

Data for this has not been routinely collected. For the MTR (2002) a survey indicated that there was a 12% increase for women; 13 for STs in OPD attendance

33

achievement) compared to baseline figures (b) Intermediate Outcome Indicator(s)

Indicator Baseline Value

Original Target Values (from approval

documents)

Formally Revised

Target Values

Actual Value Achieved at Completion or Target

Years

Indicator 1 : Increase number of institutions under project utilizing management information systems from 0 at baseline.

Value (quantitative or Qualitative)

0% 100% 100%

Date achieved 01/01/1998 03/31/2006 03/31/2006 Comments (incl. % achievement)

MIS continues to be manual, as the computerized system could not be implemented. However, information from the manual MIS is being used regularly for monitoring purposes.

Indicator 2 : Percentage of inpatients and outpatients in institutions under the project receiving diagnostic tests from 10% at baseline

Value (quantitative or Qualitative)

10% 30% 71.3% of all in-patients and 12.3% of all out-patients

Date achieved 01/01/1998 03/31/2006 03/31/2006 Comments (incl. % achievement)

OPD admission has increased by 72.3% during the same period due to which tests as a percentage has not gone up significantly

Indicator 3 : Increase number of medical staff trained (clinical, management, IEC, MIS, maintenance, referral. waste management) and number of training courses held under the project according to plan

Value (quantitative or Qualitative)

0 100% 100%

Date achieved 01/01/1998 03/31/2006 03/31/2006 Comments (incl. % achievement)

Training had a slow start. At MTR, the training strategy was revised, and appropriate institutions were actively involved in providing necessary training which helped in completing the Training Plan.

Indicator 4 : Increase number of institutions under project to introduce waste management practices based on agreed plan from 0 at baseline

Value (quantitative or Qualitative)

0%

80%

Short-term and medium-term measures introduced at all project facilities

34

Date achieved 01/01/1998 03/31/2006 03/31/2006 Comments (incl. % achievement)

While supplies required for waste management have been procured and distributed, training provided and containment areas built, waste management is not being practiced by hospital staff.

Indicator 5 : Increase number of districts utilizing community-level data from 0 at baseline

Value (quantitative or Qualitative)

0% 100% of districts

Community-based disease surveillance system operational throughout the state

Date achieved 01/01/1998 03/31/2006 03/31/2006 Comments (incl. % achievement)

The indicator is not specific on defining utilization and also does not specify whether it is MIS data or disease surveillance data

Indicator 6 : Total number of civil works grounded according to plan Value (quantitative or Qualitative)

0% 100% All civil works grounded except for two, which are under litigation

Date achieved 09/30/1998 03/31/2006 03/31/2006 Comments (incl. % achievement)

Indicator 7 : Increase in in-patient days Value (quantitative or Qualitative)

1740654 Increase in number of in-patients

3178785

Date achieved 03/31/1998 03/31/2006 03/31/2006 Comments (incl. % achievement)

No target set. In-patient days increased by 83.5% over baseline.

Indicator 8 : Increase in number of OPD cases per day Value (quantitative or Qualitative)

19485 Increase in number of OPD cases per day

33616

Date achieved 03/31/1998 03/31/2006 03/31/2006 Comments (incl. % achievement)

No target set. OPD cases increased 72.5% over baseline.

Indicator 9 : Increase in number of patients under five getting treatment in OPD Value (quantitative or

898062 Increase in number of patients under five

1827969

35

Qualitative) getting treatment in OPD

Date achieved 03/31/1998 03/31/2006 03/31/2006 Comments (incl. % achievement)

No target set. Under five OPD cases increased 104% over baseline.

36

Annex 2. Restructuring (if any)

ISR Ratings at Restructuring Restructuring

Date(s)

Board Approved

PDO Change DO IP

Amount Disbursed at Restructuring in

USD M

Reason for Restructuring & Key

Changes Made N/A

37

Annex 3. Project Costs and Financing (a) Project Cost by Component (in USD Million equivalent)

Components Appraisal

Estimate (USD M)

Actual/Latest Estimate (USD M)

Percentage of Appraisal

MANAGEMENT DEVELOPMENT AND INSTITUTIONAL STRENGTHENING

9.10 4.30 47.25

IMPROVING SERVICE QUALITY AND EFFECTIVENESS AT DISTRICT, SUB-DIVISIONAL AND AREA HOSPITALS

46.30 44.80 96.76

IMPROVING ACCESS TO BASIC HEALTH SERVICES

24.90 23.10 92.77

POLIO ERADICATION 0.00 24.10 Total Baseline Cost 80.30 96.30

Physical Contingencies 7.30 Price Contingencies 3.10

Total Project Costs 90.70 Project Preparation Facility (PPF) 0.00 0.00 0.00 Front-end fee IBRD 0.00 0.00 0.00

Total Financing Required 90.70 96.30

(b) Financing

Source of Funds Type of Cofinancing

Appraisal Estimate (USD

M)

Actual/Latest Estimate (USD M)

Percentage of Appraisal

INTERNATIONAL DEVELOPMENT ASSOCIATION

76.40 84.50 110.60

38

(c) Disbursement Profile

Note: The graph is in currency of the loan/credit/grant.

39

Annex 4. Outputs by Component The project comprised the following components: Component 1: Management Development and Institutional Strengthening (US$ 10.1 million; 11% of total project costs): (a) Strengthening planning, management and implementation capacity at the state, district and facility levels: GOO had set up a three-tier structure for the management of OHSDP: at the top were the Project Review Committee, Steering Committee and Project Board that provided general oversight and clearance of proposals; actual implementation was the responsibility of the Project Management Cell headed by the Project Director (PD). District level management was entrusted to a Project Management Unit consisting of the ZSS, Chief District Medical Officer (CDMO), additional CDMO and their team. Powers were delegated to the PD, Project Board, Steering Committee and Project Review Committee to take important decisions. However, there was insufficient delegation of financial powers to PD in the initial years. This along with frequent turnover of project management, particularly of the Health Secretary and PD contributed to delays in implementation. The DfID-supported Policy, Support and Planning Unit (PSPU) had been assigned the task of providing strategic advice to the PMC. They undertook several studies on behalf of the project, including the patient satisfaction survey; however, the capacity within GOO for strategic thinking in the health sector continued to be weak during the life of the project. This component of the project was to strengthen sectoral capacity at the state and district levels for planning, management and implementation. This component was to help achieve the first DO in terms of improving resource allocation at the state level, which is discussed in section 8.2 and further detailed below. (i) Increased share of resources for the primary and secondary levels of health care in the total resources (plan and non-plan) allocated to the health sector, until year 2002: The project did track the proportion of the health budget (Plan and non-Plan) allocated to the primary and secondary levels till 2003-03 and it rose from 83% 1n 1998-99 to about 85% in 2002-03. The H&FW budget for primary and secondary levels has increased from Rs. 3134.6 million in 1997-98 to Rs. 7441.6 million in 2002-03 (Rs. 2481.8 million to Rs. 4833.3 million in real terms) -- a 70% increase in real terms. As seen in the figure below, combined allocations to the primary, secondary and tertiary health sector (Plan and Non-Plan) have increased substantially from Rs. 3,269.8 million in FY98 to Rs. 7,441.6 million (Rs. 5,830.1 million in real terms) in 2005-06 - a 78.3% increase in real terms. The increase in H&FW Budget without project funds increased from Rs. 3700.6 million in 1998-99 to Rs. 7035 million in 2005-06 (Rs. 3354.9 million to Rs. 5511.6 million in real terms) -- a 64.3% increase in real terms. H&FW budget as a percentage of state budget increased from 2.68% in FY 98 to 2.8% in FY 06.

40

(ii) Increased resources for drugs, essential supplies and consumables in accordance with agreed norms per inpatient per bed per year (Rs. 3.500 at baseline) : The drug budget per in-patient per bed per year has increased from Rs.3,500 at baseline to Rs.5,783 in 2000-01 and to Rs.7,974 in 2004-05 (Rs. 6447.2 in real terms) - a 84.2% increase over baseline. Comparative numbers for other states are as follows: in Maharashtra it was Rs. 9,445 in 2004-05 and in Andhra Pradesh it was Rs. 8,000 in 2002. Total allocations towards drugs and consumables by the state government under all programs increased from Rs. 811.1 million in FY98 to Rs. 1181.8 million (Rs.925.9 million in real terms) in FY06 - a 14.15% increase in real terms. Of this, the share of OHSDP was Rs. 22.5 million in FY99, and Rs. 83.1 million (Rs.71.8 million in real terms) in FY06 - a 219.1% increase in real terms, as a result of increased procurement of drugs and consumables under the project. This makes the effective increase in drug and consumables budget of the state through its own resources from Rs. 811.1 million in FY98 to Rs. 1098.7 million (Rs.854.1 million in real terms) in FY06 -a 5.29% increase. This shows that although there has been an increase in drug and consumables budget per bed in the state, there has not been a significant increase in the entire state budget for drugs and consumables without project funds. (iii) Implementation of a user charge policy: Additional funding for non-salary expenditures at the facility level was obtained through the implementation of user charges.

Year User Fees collected (Rs. million)

User Fees collected (Rs. million in real terms)

98-99 3.8 3.8 99-00 10.6 9.7 00-01 13.2 12.5 01-02 15.1 14.6 02-03 16.2 15.7 03-04 26.2 24.1 04-05 26.0 23.2 05-06 30.4 26.3

41

As the table indicates, collections went up from Rs. 3.8 million in FY 99 to Rs. 30.4 million (Rs. 26.3 million in real terms) in FY 06, an increase of 601. 6% in real terms. Given the high levels of poverty in the state, it was decided to levy user charges only at the District Hospital (DH) level. Inspite of this, there has been a significant increase in collection of user fees in the state. This increased collection was applied towards contracting out cleaning services, procuring ambulance services and purchasing additional drugs and consumables. (b) Strengthening the HMIS and capacity for surveillance of major communicable diseases: Computerized HMIS covering 7 project activities - civil works, drug management, equipment status, health camps, mobile health units, hospital activity and disease surveillance - had been planned for the project; however, despite several efforts, the computerization was never completed, and was finally abandoned. A paper-based system was maintained, however; and this system has been implemented in all 156 project facilities, and regular reports have been received through this system. The project component indicator was: Increase number of institutions under project utilizing management information systems from 0 at baseline. This however was not specific enough to measure the intended outcome and hence is rated as 100% achievement despite only a manual MIS being used to collect information. Thirty Disease Surveillance Units were formed, staffed with an Assistant Surgeon and support staff and provided with a vehicle. Block level disease surveillance reports have been collected at district headquarters and transmitted through internet/fax to State Disease Surveillance Cell for analysis, monitoring and follow up action. Post the cyclone, World Health Organization (WHO) developed a more sophisticated Disease Surveillance System, which was supported by the Project and continues to be used. Component 2: Improving Service Quality, Access and Effectiveness at the Secondary Level (US$52.5 million; 58% of total project costs): (a) Upgrading area, sub-divisional and district hospitals: The project has upgraded 32 district hospitals (DH), 20 sub-divisional hospitals (SDH) and 19 area hospitals (AH) including infrastructure, manpower, equipments and diagnostic services. All facilities, except for two under litigation have been physically completed, five of which have been completed after project closure with GOO funds. Completion of electricity connections and water supply in a few facilities remain and a full time Project Director has been retained to complete these and other activities. Compared to a planned increase of 3007 beds planned in PIP, OHSDP has actually provided 20% more bed space. New Labor Rooms and Operation Theatres conforming to space norms have been constructed in 140 hospitals and all others have been renovated. Staff quarters were also constructed at several hospital sites. The project has established special dedicated transformers in all project hospitals to provide uninterrupted power supply. After completion of the civil works 68 buildings have been transferred to the concerned line

42

departments, who will be responsible for the maintenance of the facilities after project closure. The Finance Department, GOO has committed and made provision for the annual maintenance of all 157 project institutions as well as for high-value equipment after project Closure. A review conducted by the Bank indicated that about 10% of the works were of below acceptable condition (acceptable was defined by a superficial assessment of the status of floors, walls, fabric, roofing, woodwork, windows, paintwork etc.). (b) Upgrading the effectiveness of clinical, managerial and support services at the facility level: Various inputs were provided under the project to upgrade the quality and effectiveness of services provided at project hospitals. Ultrasound, Semi Autoanalyzer, X- Ray 300 mA, 60 mA and Dental X-ray were supplied to all 32 DH. X-Ray 100 mA were supplied to 20 SDHs, 19 AH, and 37 CHC-I. Laparoscopic Surgery Units were supplied to seven identified DH. QBC machine for malaria parasite detection have been provided to 22 DH and 15 SDH. Essential audiological and orthopedic equipments were provided to 32 DH. Cardiological equipments have been provided to five selected DH. Besides different baby care equipments & pathological equipments were provided to all identified project hospitals. 99 Ambulances have been supplied to needy project institutions and drivers have been appointed through District ZSS on contractual basis for efficient use of emergency patients. All the districts have been provided with vehicle, telephone, internet, fax and other supports to effective implementation of services. In addition, instruments, hospital furniture, office furniture and public utility furniture were provided to all project hospitals. Provision was also made for drugs and medical consumables based on certain norms agreed during preparation. A review of the status of supply, quality, installation and utilization of project equipment undertaken by the Bank in collaboration with the PMU indicated that: (i) equipment that was operational amounted only to 40% of the total economic value of the equipment; (ii) 15% of the major equipment amounting to 4.4% of entire cost was found to be of quality levels below the acceptable range; (iii) 77% of materials by value were supplied to district hospitals and above; and (iv) warranties had expired for about 22% of the items and 3-year warranties were applicable for most of the remaining items. The review suggested several measures for improving the supervision and monitoring of the 'commissioning' of equipment. The project adopted a Quality Assurance Program, and constituted Clinical Quality Care (CQC) & Quality Improvement Group (QIG) committees at all project facilities for on-going monitoring, evaluation and regular follow up actions. In order to ensure better quality of care at project facilities, the project created 142 new posts of doctors; hired 712 paramedics on contractual basis; and 158 Specialists and 65 senior paramedics posts on regular basis. 492 posts of senior medical officers were also upgraded. Due to these efforts initiated by GOO, the indicator on facilities staffed according to norms was largely met: at project closure, 70% of all facilities were staffed with doctors and 80% were staffed with paramedics according to norms; compared to only 30% and 45% respectively at project mid-term. The Principal Secretary, Finance had provided a commitment to maintaining contractual staff recruited under the project. However, during field visits, the Bank team was informed that no notification for

43

continuation of contractual staff has so far been made. Continuation of service provision at the health facilities depends entirely on the maintenance of staff, and the DHS needs to continuously monitor doctor absenteeism and skill mix. A total of 1494 Doctors and 921 paramedical staff have been trained in clinical and managerial aspects during the project period. The Training Plan designed for the project was completed and all staff received some training. Two rounds of sensitization on health care waste management have been undertaken for all categories of staff. Appropriate doctors and paramedical staff utilizing equipments provided by OHSDP have been trained in proper handling of equipments. After closure of the project, GOO and DHS have already agreed to continue these posts to continue the health delivery services in project hospitals. In-service training, though initiated late in the project, was provided through premier institutions like ASCI, NIFM, Kalinga Hospital and SIHFW. Project management also recently undertook equipment-based training, however this activity would have to be repeated at regular intervals in order to maintain quality utilization of equipment. Health Care Waste Management System has been initiated in all project hospitals, through the provision of equipments, training support and construction of containment areas for proper segregation, transportation and disposal of biomedical wastes respectively. However, the implementation of health care waste management is unsatisfactory. Although waste segregation materials have been provided and deep burial pits and K-wells for sharps have been constructed, waste management practices continue to be unacceptable. The end-of-project goal was for diagnostic tests for in-patients and out-patients to increase from 10% at baseline to 30%. At endline, this stood at 18%, partly because of the increase in out-patient and in-patient admissions by 72.5% and 83.5% respectively during the project period. Total number of tests conducted for in-patients increased during the project period from 410891 to 626578 (a 52. 5% increase) and for out-patients from 931047 to 1503869 (a 61.5% increase). Overall, laboratory investigations increased from 1003887 at baseline to 2130474 at end-line: a 112% increase.

44

The indicator for increase in outpatient attendance in total was exceeded: total OPD attendance increased substantially during the project period, despite disruption of services at all hospitals for various periods of time due to on-going construction and renovation. Total attendance increased from 19,485 OPD per day in 1997 to 33616 OPD per day in 2006 - a 72. 5% increase. However, the target for increased OP among women and STs could not be tracked. Unfortunately, despite repeated follow-up, the state has not been able to disaggregate numbers by gender and tribal status. Selected service delivery indicators have registered substantial gains: number of surgeries at project hospitals increased from 136,575 at baseline to 207,479 at end of project - a 52% increase. This subsequently dropped to 141,588 in 2005-06, largely due to operating theaters being under construction/renovation at district hospitals, causing a disruption in service delivery.

45

Number of under-5 treated as out-patients increased from 898,062 at baseline to 1,827,969 at the end of the project -- about 104% increase. This component was to improve quality and effectiveness of health systems as outlined in the second DO which is discussed in section 8.2 and further detailed below: (i) Improved quality: Increase in percentage of institutions staffed in accordance with agreed norms, particularly in tribal districts, from 25 % at baseline; to 75% at end of project: This indicator measured the number of institutions that had the required staffing norm. About 30% of project facilities were staffed with doctors and 45% of project facilities with paramedics according to norms at mid-term; this has increased to about 75% of project facilities staffed with doctor, and 80% with paramedical workers in according to norms at the end of the project. GOO has committed to continuing 854 contractual staff beyond the project period and an additional 223 posts of doctors and nurses have been created and filled up under the project and will be continued. An additional 492 positions of doctors have been upgraded. However, there are still vacancies in key areas such as anesthesia and pathology; this is due to an absolute shortage in the availability of such skills in the market. There appears to be an overall equitable distribution of institutions staffed according to norms among tribal and non-tribal populations. No patient satisfaction survey was conducted by the project at endline: A patient satisfaction survey conducted by ASCI for the Project in 3 tribal districts during mid-term indicated that about 75% of respondents within the catchment facility of a DH, Sub-Divisional Hospital (SDH) and Area Hospital (AH) were satisfied with the services provided in those facilities. (ii) Improved effectiveness: Increase in number of admissions to institutions under the project due to pregnancy-related complications from 20% at baseline, by 5 percentage points by mid-term and 15 percentage points by end of project. This indicator does not specifically define the denominator used to calculate the figures since the denominator does not seem to be based on either IP admissions or pregnancy related admissions. If the indicator uses new In-Patient (IP) admissions as a denominator, then at baseline about 12% of all admissions were due to pregnancy-related complications (and not 20% as reported in the PAD) and if the indicator uses all pregnancy related admissions as a denominator, then in 1998-99 about 58% of all admissions were due to pregnancy-related complications. Hence, it is not possible to measure this indicator. Cases of pregnancy related complications have only been measured till 2003-04 and do not seem to be measured routinely by the project. However, admissions due to pregnancy related complications have increased in aggregate, from 45,241 across all hospitals in 1997-98 to 72,123 in 2003-04 - an increase of 60%; and all categories of hospital have reported large increases over the baseline. There is also an increase in the number of pregnancy related admissions from 99,160 in 1998-99 to 178,693 in 2005-06; an increase of 80.2%.

46

Increase in the number of institutional deliveries in institutions under the project from 16% at baseline by 5 percentage points at mid-term and 15 percentage points at endline: This indicator too does not specify the denominator to be used for calculation. However, there has been an 81% increase over baseline in the number of institutional deliveries at project facilities, from 78,157 in 1997-98 to 141,168 in 2005-06. In 1999 about 775,000 births occurred in Orissa; NFHS II (1999) reported that about 22.9% of these were institutional deliveries. About 98885 deliveries took place in project hospitals in 1998-99 indicating that about 50% of all institutional deliveries in Orissa took place at project hospitals. Hence, the increase in institutional deliveries at project facilities may have contributed to the overall increase in institutional delivery in the state as indicated in the RCH data in section 8.5(a). NSSO data indicates that in rural areas there has been an 85% increase in total hospital admissions from 383,200 in 1995-96 to 710,200 in 2005-06. Admissions in government facilities during the same period grew by 61.8% from 347,200 to 561,800. Admissions in private facilities during the same period grew by 312% from 36,000 to 148,400. In urban areas, there has been a 159% increase in total hospital admissions from 83,000 in 1995-96 to 214,900 in 2005-06. Admissions in government facilities during the same period grew by 133.8% from 67,200 to 157,100. Admissions in private facilities during the same period grew by 674% from 7,500 to 57,800. This indicates an overall increase in access to hospitals in rural and urban areas, with significant increases in both private and public sector. It is important to note the increased growth of private sector in both rural and urban areas. As indicated in the table below showing OPD access, the share of private sector has increased by 6% in the rural areas, whereas the government sector has more or less remained the same. In urban areas, there seems to a significant increase in the share of the government sector by 12%, whereas the private sector has seen a drop of 13%. This maybe due to the population preferring the improved government facilities under the project in the urban areas compared to private facilities. It is possible that the non-poor increased utilization of these improved government facilities but there is no data available to indicate which income categories saw increases to these facilities. OPD Rural Urban NSSO Rounds

62nd

52nd

42nd

62nd

52nd

42nd

% accessing Government Facility (OPD)

39

38

37

46

34

43

% accessing Private Facility (OPD)

37

31

40

53

% of Total access (OPD)

76

69

86

87

47

Component 3: Improving Access to Basic Health Services at the Community Level (US$ 28.1 million; 31% of total project costs): (a) Selective upgrading of community health centers: 37 CHCs were strengthened under the project, including renovation of facilities, provision of additional manpower and provision of equipment, drugs and other supplies. These CHCs were provided with a labor room, OT, diagnostic block, water supply, sewerage and a solid waste management system. A total of 624 beds were added to these CHCs. An additional 48 Block PHCs were be upgraded and called CHC-II. The extension program upgraded these PHCs to 16-bedded CHCs, with rooms for specialists, continuous water supply, meeting hall, MIS/IEC room, store, dispensing room, immunization room and space for administrative work.. A total of 480 beds were added in CHC-IIs. (b) Promoting health services in tribal areas and for disadvantaged groups: Several activities were undertaken as part of the Tribal Strategy to improve access to basic services in tribal areas as outlined in section 7.4. (c) Improving referral mechanisms and strengthening linkages between different tiers of the health care system: The project introduced Referral System to strengthen the linkage between different tiers of health facilities. However, the project did not track the effectiveness of the referral system in sending patients to appropriate levels of healthcare facilities as per patient needs. A total of 2,346 Informal Service Providers (ISPs) were provided training in early referral (for obstetric and neonatal emergencies and cerebral malaria) and treatment of fever with chloroquin tablets and management of acute gastroenteritis with oral rehydration solution, in six predominantly tribal districts. This component was designed to help achieve improved coverage of health systems as outlined in the second DO and is discussed in section 8.2.

48

Annex 5. Economic and Financial Analysis (including assumptions in the analysis) The project did not calculate any traditional measures of efficiency like Net Present Value (NPV) and Economic Rate of Return (ERR) during the time of appraisal. This has not been calculated for most sectoral projects prepared at the time. However, during the appraisal, there has been a review of the fiscal situation in the state, budgetary allocations of the health sector, discussion of some cost-effectiveness assumptions and project financial sustainability. There has also been no calculation of the Financial Rate of Return. As discussed in previous sections, the project focused on improving resource allocations within the health sector and improving performance of the health system. In terms of improving efficiency in resource allocations, the project managed to satisfactorily increase budgetary allocations to the primary secondary sectors, budget for drugs and consumables per bed and collection of user fees. As discussed in section 8.2 and annex 4, the H&FW budget for primary and secondary levels has increased from Rs. 3134.6 million in 1997-98 to Rs. 7441.6 million in 2002-03 (Rs. 2481.8 million to Rs. 4833.3 million in real terms) - a 70% increase in real terms. Also, H&FW Budget without project funds increased from Rs. 3700.6 million in 1998-99 to Rs. 7035 million in 2005-06 (Rs. 3354.9 million to Rs. 5511.6 million in real terms) a 64.3% increase in real terms. The incremental increase of the total H&FW budget over the life of the project was 4.79 times the entire OHSDP budget in real terms and the incremental increase of the H&FW budget for the primary and secondary levels over the life of the project was 4. 14 times the entire OHSDP budget in real terms (This assumes that allocations to the primary and secondary levels continued to stay at 85% as in 2002-03 as shown in the table below). This shows that the project investments may have been efficient in improving resource allocations.

(All figures in Rs. million) 1998-99 1999-00 2000-01 2001-02 2002-03 2003-04 2004-05 2005-06 2005-06

H&FW Budget (Real Terms) 3269.8 3423.8 4260.9 4763.2 5122.9 5686.2 5075.0 5708.1 5830.1

Incremental H&FW Budget over baseline (Real Terms) 154.0 991.1 1493.4 1853.1 2416.4 1805.2 2438.3 2560.3 Primary and Secondary Budget (Real Terms) 2714.0 2841.8 3408.7 4001.1 4354.5 4833.3 4313.8 4851.9 4955.6

Incremental H&FW Budget for primary and secondary over baseline (Real Terms) 127.8 694.8 1287.2 1640.5 2119.3 1599.8 2137.9 2241.6

OHSDP Budget (Real terms) NA 68.9 173.4 309.0 487.3 461.0 620.9 420.7 318.5

Total incremental H&FW Budget over baseline (Real Terms) 13711.7

Total incremental H&FW Budget for primary and secondary over baseline (Real Terms) 11849.0

Total OHSDP Budget (Real terms) 2859.8 In terms of improving health system performance, the main assumption for cost effectiveness given at

49

the time of appraisal was that cost of providing services at the secondary and FRU level of care was 25-40% cheaper than providing the same at tertiary levels. A comparison of costs for specific interventions at the primary, secondary and tertiary levels in Andhra Pradesh showed the following (Administrative Staff College of India. "Andhra Pradesh Burden of Disease and Cost-Effectiveness of Health Interventions". Report Volume II, 1966. Center for Social Services, Hyderabad):

Though these costs were calculated for Andhra Pradesh, they were indicative of costs in other states as well. Using this as a basis, cost savings from the project can be calculated using the following assumptions: (i) costs are in current prices; (ii) all incremental increases in facility utilization may have accessed tertiary facilities instead; (iii) The project costs as inputs are civil works, goods and equipment and incremental operating costs; (iv) the expected life of these inputs are 25 years after project closure; and (v) the difference in costs between secondary and tertiary facilities do not increase. This calculation does not take into consideration that this project may have actually decreased cost per unit of bed day in secondary facilities due to increased utilization and hence is a conservative estimate. It also does not factor in the increased cost and possible loss of income for a patient who accesses a tertiary facility instead of a secondary one. The total project inputs are Rs. 2.54 billion and the incremental increases for utilization and costs saved for the project life are given in the tables below. In these seven years, cost savings of Rs. 160 Million have taken place using this method of calculation and annual savings show an increasing trend. Assuming a 25 year remaining life of the project facilities, it is likely that the benefit from the costs savings would equal 60-70% of the initial project inputs.

50

Costs saved

1999-00 2000-01 2001-02 2002-03 2003-04 2004-05 2005-06 Bed Days -408765

-1622769 -5194095 -5522110 -8627721

-10591425

-24251418

OPD -509508 915660 12526320 22354704 38949420 39789972 55451280 Lab tests 1306530 4356945 3216930 -394200 6839340 6833235 12149880 X-Ray 357780 774700 1309020 1431520 1694940 1851580 1933700 Surgeries -275190 -800800 -259350 569380 2409540 3568880 -1043490 Total -529153 3623736 11598825 18439294 41265519 41452242 44239952

This does not factor into the improvement of quality of services and other services the population would benefit from the project inputs, including critical areas like handling of obstetric emergencies and institutional deliveries.

51

Annex 6. Bank Lending and Implementation Support/Supervision Processes (a) Task Team members

Names Title Unit Responsibility/SpecialtyLending Sanjeev Aggarwal Consultant ECSSD HCWM Nina Anand Program Assistant SASHD Program Assistance Mam Chand Sr Procurement Spec. SARPS Procurement Jean-Jacques Dethier Research Manager DECRS Peer Reviewer

Salim J. Habayeb Lead Public Health Specialist LCSHH Institutional Development

Hiroko Imamura Sr Counsel LEGMS Legal Eva Jarawan Sector Manager AFTH2 Peer Reviewer Raj Kumar Public Health Spec. SASHP Training plan Samuel S. Lieberman Lead Economist EASHD Peer Reviewer Julie Mittman Consultant LCSES Social Development Tawhid Nawaz Operations Adviser HDNOP TTL Indra Pathmanathan Consultant EACIF Reproductive Health Cheryl M. A. Powell-Ambassa

Program Assistant SASHD Program Assistance

Shreelata Rao-Seshadri Consultant SASHD Social Development

Sanjay N. Vani Sr Financial Management Specia

ECSPS Financial Management

Maj-Lis A. Voss Consultant HDNSP Project costing Supervision/ICR Nina Anand Program Assistant SASHD Program Assistance Sushil Kumar Bahl Sr Procurement Spec. SARPS Procurement Peter A. Berman Lead Economist, Health SASHD Lead Specialist Debabrata Chakraborti Sr Procurement Spec. SARPS Procurement, ICR inputsMam Chand Sr Procurement Spec. SARPS Procurement

Mohan Gopalakrishnan Sr Financial Management Specia

SARFM Financial Management

Michele Gragnolati Senior Economist ECSHD ICR Peer Reviewer

Manoj Jain Sr Financial Management Specia

SARFM Financial Management

Preeti Kudesia Sr Public Health Spec. SASHD TTL

52

Rajat Narula Senior Finance Officer LOAG2 Financial Management Tawhid Nawaz Operations Adviser HDNOP TTL Snehashish Rai Chowdhury

Operations Officer SASHD Operations officer

Vikram Sundara Rajan Health Specialist SASHD ICR report TTL Gandham N.V. Ramana

Sr Public Health Spec. SASHD Public Health

Shreelata Rao-Seshadri Consultant SASHD Social Development Rakesh Sahni Consultant SASHD Architect (b) Ratings of Project Performance in ISRs

No. Date ISR Archived IP DO Actual Disbursements

(USD M) 1 11/10/1998 Satisfactory Satisfactory 0.00 2 03/18/1999 Satisfactory Satisfactory 3.60 3 06/23/1999 Satisfactory Satisfactory 3.60 4 12/27/1999 Satisfactory Satisfactory 3.74 5 03/06/2000 Satisfactory Satisfactory 4.23 6 06/13/2000 Satisfactory Satisfactory 4.23 7 10/27/2000 Satisfactory Satisfactory 5.13 8 03/14/2001 Satisfactory Satisfactory 6.03 9 08/29/2001 Satisfactory Satisfactory 7.59 10 12/20/2001 Satisfactory Satisfactory 10.26 11 05/24/2002 Satisfactory Satisfactory 11.91 12 11/25/2002 Satisfactory Satisfactory 14.85 13 06/26/2003 Satisfactory Satisfactory 20.01 14 10/06/2003 Satisfactory Unsatisfactory 20.01 15 11/25/2003 Satisfactory Unsatisfactory 22.87 16 06/07/2004 Satisfactory Unsatisfactory 25.43 17 11/30/2004 Satisfactory Satisfactory 31.92 18 05/07/2005 Satisfactory Satisfactory 59.78

19 11/04/2005 Satisfactory Moderately Unsatisfactory

66.69

20 03/31/2006 Moderately Satisfactory

Moderately Satisfactory

72.58

53

(c) Staff Time and Cost

Staff Time and Cost (Bank Budget Only)

Stage of Project Cycle No. of staff weeks

USD Thousands (including travel and

consultant costs) Lending

FY96 1.00 FY97 92.00 FY98 204.00 FY99 5.00 FY00 8.48 FY01 0.00 FY02 0.00 FY03 0.00 FY04 0.00 FY05 0.00 FY06 0.00 FY07 0.00

Total: 310.48 Supervision/ICR

FY99 50.00 FY00 20 62.25 FY01 22 88.20 FY02 23 91.37 FY03 12 54.42 FY04 18 50.96 FY05 16 44.10 FY06 24 123.90 FY07 2 4.00

Total: 137 569.20

54

Annex 7. Detailed Ratings of Bank and Borrower Performance

Bank Ratings Borrower Ratings Ensuring Quality at Entry:

Moderately Satisfactory Government: Moderately Satisfactory

Quality of Supervision: Moderately Satisfactory Implementing Agency/Agencies:

Moderately Satisfactory

Overall Bank Performance:

Moderately Satisfactory Overall Borrower Performance:

Moderately Satisfactory

55

Annex 8. Beneficiary Survey Results (if any) Not Applicable.

56

Annex 9. Stakeholder Workshop Report and Results (if any) Not Applicable.

57

Annex 10. Summary of Borrower's ICR and/or Comments on Draft ICR With reference to The Department of Economic Affairs letter dated June 13, 2006, there have been changes made to the comments of the OHSDP authorities. Orissa Health Systems Development Project Credit No- 041-IN

Project Details Original Project cost Revised cost/ closing date after diversion of US $ to polio eradication programme

Board & DC Agreement Date 13th August 1998 Project Commencement Date 18th September 1998 Project Period 5 ½ years 7 ½ years Project Closing Date 31st March 2004 31st March 2006 Credit Amount Rs. 415.58 Cr. Rs. 307.98 cr. Financial Statement Enclosed at Annexure - I

The Orissa Health Systems Development Project was originally sanctioned cost of Rs.415.58 Crs from International Development Association. But unfortunately as the progress of work during the first three years could not pick up due to "Super Cyclone" of 1999 and subsequent floods in 2000-01 which affected all works in the State. Thereafter US $ 25 million (Rs.107.60 Crs) was diverted from this project to Polio Eradication Programme. This resulted in non-completion of many agreed project activities as per the approved Project Implementation Plan of OHSDP. Borrower/ implementing Agency: Govt. of India/ Govt. of Orissa Government of Orissa had received a credit through Govt. of India from International Development Association (IDA) to strengthen secondary and primary health care facilities in rural, urban and semi urban areas throughout the State. Though the project period was estimated to end by 31st March 2004, due to some unavoidable natural calamities like Super cyclone of 1999 and severe flood of 2001 which affected the State in 1999 and 2001, on the request of the State Govt. the project period has been extended up to 31st March 2006. After diversion of the funds to polio eradication programme, lot of agreed project activities like construction of civil works i.e. attendant rest shed, boundary walls of hospitals and water supply to some hospitals and staff quarters, maintenance of buildings, drainage system in project hospitals, repair and renovation of existing hospitals and staff quarters, annual maintenance of equipments and training of Informal Services Providers in many tribal pockets could not be taken up. During the last joint review meeting held on January 2006 at DEA, it was specifically requested to extend the project period for another 3 months to complete the 12 ongoing civil works in every respect. However, this was not agreed to and it has seriously affected the completion of the agreed project activities of this project.

58

Assessment of Development Objective, Design and quality at Entry Objective: The broad objectives of Orissa Health Systems Development Project were:-

1. To improve efficiency in the allocation and use of health resources through policy and institutional development and;

2. To improve the performance of health care systems through improvements in the quality, effectiveness and coverage of health services at secondary level and selective coverage at community level, so as to improve the health status of the people, especially the poor, by reducing mortality, morbidity and disability.

Design: To achieve the project objectives, the Project had designed the following strategies.

1. Provision of additional 3007 beds in 156 identified project hospitals to extend the basic health care facilities and accommodate the increasing patients both in rural and urban areas.

2. Provide Specialist services in the project hospitals by creating new specialist and paramedics posts.

3. Improve Patient Referral System in Secondary level of health facilities by introducing Referral Card (RC) and arrangement of transport facility for transporting emergency patients to nearby referral hospitals.

4. Launching of an ambitious civil work programme towards renovation, extension, onetime repair and new construction of 156 identified project hospitals ensuring upgraded and integrated water supply and power supply.

5. Provide sophisticated and high-end medical equipment to its project hospitals in order to strengthen diagnostic and specialized services.

6. Supply of quality drugs and medical consumables to its project hospitals for distribution among the patients free of cost to benefit of the poor and disadvantage group of people.

7. Implementation of Hospital Waste Management System for proper segregation, transportation and disposal of Bio Medical Wastes in order to provide hazardous free treatment to the patients.

Quality at Entry Level At the beginning of project implementation in 1998, where there were acute shortage of health resources in terms of infrastructure, equipment, sufficient drugs & medical Consumables and manpower, at the end these components are in position. Project Components: The project had to undertake three major components (1) Management Development and Institutional strengthening (11%) by (a) strengthening planning, management and implementation capacity at the State, district and facility levels and (b) strengthening the HMIS and the capacity of surveillance of major communicable diseases; (2) Improve service quality, access and effectiveness at the secondary level (58%) by (a) upgrading sub divisional and district hospitals and (b) upgrading effectiveness of clinical, managerial and support services at the facility level and (3) Improving Access to Basic Health Services at the community level (31%) through (a) selective upgrading of

59

CHCs (b) promoting health services in tribal areas and (c) improving referral mechanism.

1. Management Development and Institutional Strengthening: (11%) The project aimed at developing and strengthening institutional framework for policy development like policy issues on public private health care, burden of disease, and cost-effectiveness of public health interventions, medical manpower, cost recovery mechanisms and sectoral resource allocation patterns by involving the Zilla Swasthya Samities.

2. Improving Service Quality, Access and Effectiveness at the Secondary Level (58%)

• The OHSDP has constructed new hospital buildings, renovated existing buildings and made one time repair the old hospital buildings in order to accommodate increasing number of patients (both in-patients & out patients) in project hospitals and to provide better service of Labor Room, OT and also constructed essential and minimum staff quarters to accommodate medical health providers. Out of 156 hospital buildings and one State Institute of Health & Family Welfare (SIHFW) building, 145 buildings have been completed fully and another 10 has been partially completed without providing 3-phase power connection and water supply by 31.3.2006 which will be completed by June 2006. The remaining 2 hospital buildings like Chandballi,CHC-I and Nilgiri,SDH can not be completed by the end of the Project period as these two are locked up in litigation in the Hon'ble High Court. But these two hospital buildings could have been completed after vacation of stay order by Hon'ble High Court, if World Bank would have agreed for extension of another 4 months (i.e. within the grace period) for completion of the 12 incomplete buildings. This has resulted wastage of money in half construction of the two buildings and also it will severely affect the health delivery system in the concerned hospital area of which SDH Nilagiri is in a tribal pocket of the State. However the project has completed the following facilities in the project hospitals.

• Provided water supply to all its project hospitals with adequate sewerage line excepting the 12 hospitals mentioned in paragraph above.

• Established special dedicated transformers in all project hospitals to provide uninterrupted power supply excepting the above mentioned hospitals out of which 10 will be completed within grace period.

• To improve the effectiveness of clinical, management and support services at district, sub divisional, area hospital and community level hospitals OHSDP procured and supplied equipments, drugs and medical consumables by adopting staffing and equipment norms.

• Adopted Quality Assurance Programme to ensure quality of health services by constituting Clinical Quality Care (CQC) & Quality Improvement Group (QIG) committees at all project facilities and regular monitoring, evaluation and regular follow up actions.

• The project has provided in-service and equipment based service to the medical professionals to enhance their clinical and managerial skill and knowledge at premier institutions like ASCI, ISHA, AHA, NIFM, JIPMER, NIMS, Kalinga Hospital and

60

SIHFW. • Improved Health Care Waste Management System in all project hospitals (except two)

by providing adequate equipments, training support and constructed Containment areas for proper segregation, transportation and disposal of biomedical wastes respectively. The project has completed 155 containment areas and the remaining one at Pallahara SDH could not be completed for non- availability of site. Training of doctors and para medicals has already been conducted twice but as it has been rightly pointed out in the World Bank review Mission, the attitude of concerned doctors and para medicals could have been changed if the project period would have been extended for another 1 year to sustain all the project activities.

3. Improving Access to Basic Health Services at the Community level and in inaccessible areas (31%)

• In order to provide better health care service at doorstep this project has provided nine mobile health clinics in Mayurbhanj, Sundargarh, Keonjhar, Gajapati and Kandhamal the most inaccessible non-KBK tribal districts.

• The project had conducted 5000 Gram Panchayat level health camps in 118 tribal blocks for first three years of project implementation to create awareness amongst the poor and disadvantage group of people.

• Incentive was given to the Doctors of single doctor's institution in tribal areas @ Rs. 400/- p.m to encourage them to stay & treatment in tribal areas. But the system was no longer useful due to small amount of incentive.

• Introduced Referral System in project hospitals to strengthening linkage between different tiers of health facilities.

Achievement of Objective and Output: The achievement of the project objectives of this project is rated as satisfactory by successive World Bank Missions as most of the targets have been achieved except in some areas which was not possible due to diversion of project fund to the Polio Eradication Programme and lack of optimum time for consolidation of the project activities after the completion of civil works and installation of equipments. For successful implementation of the project, Govt. of Orissa had set up three tier management structure of OHSDP. At top level there were Project Review Committee, Steering Committee, Project Board and at middle level Project Management Cell (PMC) headed by Project Director with his four implementing units of Engineering, Finance, Health & Administration. In district level Project Management Unit comprised of Zilla Swasthya Samiti, CDMO, Additional CDMO with their office team available for successful implementation of the project. As committed, Govt. of Orissa has increased the allocation to the health sector every year i.e. from financial year 1998-99 Rs. 3269.8 million to FY 2004-05 Rs. 7061.1 millions. The share of resources allocated for primary and secondary levels of health care increased from 83% in financial

61

year 1998-99 to 85% in FY 2004-05. Annual drug allocation per bed of the State has increased from Rs. 3500 in 1997-98 to Rs. 7974 in 2004-05

User charges which were initially collected from District headquarter hospitals only are now collected up to SDH level with exemption provided to the poor & vulnerable section. Funds are retained by the collecting institutions for utilizing in operation and maintenance of the equipments and services in hospital. Total collection increased from Rs. 3.7 million in FY 1998-99 to Rs. 26 million in FY 2004-05 that is a remarkable increase of sevenfold. External Evaluation: The studies and survey conducted by external agencies and departmental team regarding utilization of facilities and satisfaction by patients have revealed that the project has significantly contributed to improve the health status of the community. ASCI's satisfaction study conducted during 2002 revealed that major patients were satisfied with the treatment provided by the facility and those were unsatisfied due to inadequate diagnosis service, inadequate drug supply and non availability of doctors. After operationalisation of OHSDP inputs like equipments and sanction of surplus staff and as like improvement of skill of doctors by rendering training, this problem could be solved. Recently the comparative study between OHSDP supported and non-OHSDP institutions conducted by PSPU reveals that the patients have better faith on OHSDP institutions than others. The end-line evaluation of the project could not be taken up due to paucity of time and diversion of funds.

62

Comparison of Important monthly indicators (average) in project and non-project Hospitals done by PSPU Hospitals OP IP BOR Delivery Doctors Staff Nurse OHSDP assisted project hospitals (Betanoti, Soro & Basudevpur)

4450 416 87% 75 8 6

Non Project Hospitals (Jatni, Balakati & Kishantandi)

3203 131 27% 33 5 3

The project has achieved its target in health indicators as regards project institutions. Per day Outpatient attendance rose to 30040 in 2004-05 from 19485 in 1997-98, number of institutional delivery increased from 78157 in 97-98 to 129532 in 2004-05 and Surgery increased from 136575 in 97-98 to 207479 in 2004-05. Similarly laboratory investigations rose from 10 lakhs in 97-98 to 21 lakhs in 2004-05.

Besides, OHSDP has achievement towards involvement of tribal traditional healers into mainstream of Health services by identification, imparting training and creating awareness about the modern health care facilities. The project has introduced this programme as Informal Service Providers Training (ISP) and so far 2455 traditional healers like Gunia, Deshari, and Quack community of people are brought into the mainstream of health delivery system from 24 tribal blocks. These ISPs are now working as basic health service providers and become good informant about the health service in their own inaccessible tribal areas. They are referring emergency patients to the nearest health facility without detention at their end unlike practiced before ISP training. The project has emphasized upon tribal health service and provides doorstep health services in inaccessible 5 non- KBK tribal district through functioning of nine mobile health clinics. At the first

63

three years of project, 5000 health camp were conducted in each GP of 118 tribal blocks in order to provide free health services to disadvantage group of people and to create awareness amongst poor, women & SC/ST community of people about the modern health care facility. Important Project Outputs: 1. Management Development and Institutional Strengthening The Govt. of Orissa addressed the management development and institutional strengthening

• By delegating the powers to the Project Director, Project Board, Steering Committee and Project Review Committee to take important decisions for speedy implementation of the Project activities. The delegation to project in civil works for sanction of deviation, extension of time etc. was given in 2003-04.

• To improve the private public partnership in tribal areas, the training of ISPs have been taken up in 7 tribal districts and it has helped not only to create a bridge between the health delivery system and tribal people at large but also it has helped to increase the institutional deliveries resulting decrease in IMR and MMR as per the Millennium Development Goal (MDG).

• After completion of the civil works all the buildings have been transferred to the concerned line departments like, Public Works Department, Public Health Department., RWSS and electrical distribution companies for post maintenance of buildings after the closure of the Project.

• Finance Department, Government of Orissa has agreed to provide funds for annual maintenance of the civil works in 157 project institution and also for maintenance of high-end equipments supplied through OHSDP to different project hospitals after the warranty period is over.

(i) Strengthening management and implementation Capacity: The project has been implemented and managed by posting an officer in the rank of Additional Secretary to Govt., as Project Director supported by experienced Doctors, Engineers, Finance and Accounts Experts, Procurement specialist, Administrative Officer with experienced supporting staff. They have been trained in their respective discipline through various premier training institutions, workshops, seminars in national and state level, to built their capacity and equip them to effectively implement the project. Most of the senior Health Administrators have been trained on hospital management system for better utilization of services. The project has established an efficient system to manage quality of service by setting up of a CQC & QIG committees in all project hospitals, Hospital Waste Management Team, District Task Forces and Disease Surveillance System throughout the State. Project has introduced Referral Card in green color to link referral chain with incentives for referral patients in treatment at tertiary level.

64

(ii) Developing Surveillance Capacity for Major Communicable Diseases: Disease Surveillance Unit in all 30 districts have been formed with one Vehicle, fuel, driver, Assistant Surgeon for new districts as well as technical supports to surveillance nine major communicable diseases. Block level disease surveillance reports are collected at District headquarters and transmitted through internet/fax to State Disease Surveillance Cell for analysis, monitoring and follow up action. District Task Force comprising one Doctor, one Pharmacist, one Staff nurse and attendant with a vehicle, and necessary laboratory investigation instruments for early detection and outbreak of communicable diseases. 2. Improving Service Quality, Access and Effectiveness New Construction, renovation, and onetime repair of secondary level facilities: The project has undertaken 32 district cadre hospitals, 20 sub divisional hospitals and 19 Area hospitals for upgrading its infrastructure, manpower, equipments and diagnostic services. 145 project works have been completed with water supply and 126 connectivity to 3 phase service connection to improve operational efficiency and reduce fatigue factor. Against 3007 beds planned in PIP, OHSDP has actually provided 20% more bed space to (a) conform to space norms of bed layout for congested old wards, and (b) enable reallocation of functional of old wards for operational efficiency. New Labor room and Operation Theatre conforming to space norms and support units have been constructed in 140 hospitals and old OTs and LRs have been renovated Supply of Hospital Equipment, Instruments and Furniture Ultrasound, Semi Autoanalyzer, X- Ray 300 mA, 60 mA and Dental X-ray have been supplied to all 32 District cadre hospitals. X-Ray 100 mA have been supplied to 20 SDHs, 19 Ahs, and 37 CHC-I. Laparoscopic Surgery Units have been supplied to seven identified district hospitals. QBC machine for malaria parasite detection have been provided to 22 DHHs and 15 SDHs. Essential Audio logical and Orthopedic equipments have been provided to 32 district cadre hospitals. Cardio logical equipments have been provided to five selective district cadre hospitals. Besides different baby care equipments & pathological equipments have been provided to all identified project hospitals. These major equipments provided by the project have improved the effectiveness. Apart from essential medical equipments, instruments, hospital furniture, office furniture and public utility furniture have been provided to all project hospitals as part of modernization. This has not only strengthened the health delivery service in project hospitals, but also has helped increasing the collection of user charges, which can be spent for urgent maintenance of equipments. Improve Quality: The project has created new posts of 142 Doctors and 712 paramedics on contractual basis and 158 Specialists & 65 senior paramedics posts on regular basis. 492 posts of senior medical officers have been upgraded under this project for improving quality of health delivery services. A total of 1494 Doctors and 921 paramedical staff have been trained in clinical and managerial aspects during the project period. All categories of staff have been sensitized in implementing

65

hospitals waste management system 2 rounds under this project. Concerned doctors and paramedical staff utilizing equipments provided by OHSDP have been trained in proper handling of equipments. After closure of the project Govt. of Orissa and DHS, Orissa have already agreed to continue these posts to continue the health delivery services in project hospitals. Access of Service Quality: Patient satisfaction study by ASCI, Hyderabad, OP attendant study by PSPU, Bhubaneswar have been conducted to access service quality and patient satisfaction provided by project hospitals. Hospital quality care committee and improvement group committee in all project hospitals have been formed to access the quality of service and improve through necessary feedback. Improving Effectiveness: Supply of high end equipments like Ultrasound, Semi Autoanalyzer, X- Ray 300 mA, 100mA, 60 mA and Dental X-ray. Laparoscopic Surgery Units QBC machines for malaria parasite detection, essential Audio logical and Orthopedic equipments, Cardio logical equipments, different baby care equipments & pathological equipments to project hospitals has improved the effectiveness of project hospitals to cater to the needs of the patients, which has increased in every project hospitals.99 Ambulances have been supplied to needy project institutions and drivers have been appointed through District ZSS on contractual basis for efficient use of emergency patients. All the districts have been provide by Vehicle, telephone, internet, fax and other supports to effective implementation of services. 3. Improve Access to Basic Health services in tribal and inaccessible areas. The project had conducted 5000 health camps in all GP of all 118 tribal blocks during first three years of project to provide free health services to poor, disadvantage and vulnerable group of people in inaccessible tribal areas. Nine Mobile Health clinics in five non KBK tribal districts like Mayurbhanj, Sundargarh, Kandhamal, Keonjhar and Gajapati are functioning to provide doorstep health care facilities in inaccessible tribal areas. Financial Achievement: The project is being implemented with the financial assistance of IDA (World Bank credit No. 041-IN) with March 31, 2006 as credit closing date. The original project cost of Rs. 415.58 crores has been revised to Rs. 307.98 crores after diversion of Rs. 107.60 crores towards Polio Eradication Programme. The project adopted an efficient financial management system with a smooth decision in finalizing the project activities.

66

Sustainability: Government of Orissa is fully committed to sustain the project activities created by Orissa Health Systems Development Project. The following high level policy decisions have been taken to address the issues of sustainable in the project hospitals of this project.

• Government in Finance Department has agreed to provide funds for post annual maintenance of the civil works constructed in project hospitals by this project.

• Government in Finance Department has also agreed to provide funds for annual maintenance contract of the high end equipments like cystscope, Ultrasound, X-ray machines, Laparoscopic etc. by providing additional funds in the budget of H & FW Dept. for the purpose.

• Government has also agreed to sustain the additional manpower created through this project to sustain the project activities.

• Optimum collection of user charges, with the exception of poor people (Below Poverty Line Patients)

• Ensuring continuation of correct practices in Hospital Waste Management Systems and compliance to legal issues

• Providing adequate budget provision for drugs • Exploring and enhancing the scope for private and public partnership by training of

ISP's in tribal areas.

Lessons Learnt: The project period of five years should be earmarked for different specific activities like first 2 years be earmarked to complete all civil works, 3rd year for procurement and installation of equipments and 4th year should be earmarked for residual civil works, waste management, quality assurance, installation of equipments, equipment based training of doctors and para medicals, IEC activities etc. The last year should be earmarked for supervision and ensure proper implementation and sustenance of the Project activities to achieve the overall objectives of the Project. In OHSDP, the delegation of financial power to Project authorities was completed in December 2003 which resulted in delay in completion of civil works, procurement of equipments etc. So all delegation of powers to project authorities should be made by the State Govt. in the first year of the Project. This should be a pre-condition of loan. As per the advice of the World Bank separate Design Consultant (DC) and Construction Management Consultant (CMC) were appointed for construction of civil works in the Project hospitals, which is a defective procedure. During implementation, it was found that designs prepared by Design Consultants was far from the actual requirement of the project hospitals for which there was huge deviations during construction of the buildings. To ensure cost control in the civil works i.e. 40% of the total project cost, certain important items like, doors, windows, and roof

67

of certain buildings were eliminated from the design and BOQ. This has caused wastage of time during the construction of the buildings by the CMCs and finally it resulted in delay in completion of civil works and also in completion of other project activities like, installation of equipments and training etc. So it is suggested that the CMC should be appointed as Design Consultant in stead of separate Design Consultants. The total cost of civil works as per the Project Implementation Plan (PIP) is Rs.168.50 Crs and to supervise the work of CMCs the Engineering Unit in the PMC is too small to cope up with the actual work in the project. So it is suggested that the Engineering Unit in the PMC should be headed by an officer of the rank of Chief Engineer. Besides, Superintendent Engineer, Executive Engineer, Assistant Engineers, including Civil, Electrical and PH persons be engaged so that the supervision of civil works under taken by CMCs could be made by the PMC to ensure timely completion of the works. The Consultancy fees of CMCs were fixed on the basis of man-month remuneration basis. In the first three years without making any physical progress and financial expenditure in civil works, the Design Consultants and CMCs have taken huge amount as consultancy a fees which was revised in 2003-04 on parentage basis of actual expenditure done and actual work made by the CMCs. This has resulted in better expenditure and better physical progress of the civil works. So it is suggested that the Consultancy fees should be fixed on the basis of physical progress as well as financial expenditure made by CMCs. There should be also a provision for penalty for non achievement of targets by the contractors by the CMCs. In OHSDP, some Govt. Corporations were appointed as CMCs and contractors. For example IDCO, OCCL (Govt. of Orissa Corporation), MECON (Govt. of India Corporation) were appointed as CMCs and other Govt. Corporations like OSIC, OBCC, (Govt. of Orissa), Andrew Yule (GOWB-Kolkata), Richards Cruddas (GOM, Mumbai) were appointed as NCB contractors for Civil Works. On a review of the performance of CMCs and contractors, it was found that the performance of the Govt. Corporations who are appointed either as CMCs or as Contractors was highly unsatisfactory during the project period in comparison to the private NCB contractors and private CMCs. So it is suggested that Govt. Corporations should not be appointed as NCB contractors or as CMCs as in most of the cases they are sub letting the works to other petty contractors for which the quality of work has suffered in some cases. It was observed during the implementation of OHSDP works, that the big super class contractors, who were awarded with contracts of more than one civil works worth more than one crore have defaulted in completing the works in time. For example three DHHs (Cuttack,Puri and Khurda) was entrusted to one contractor with the cost of above Rs.12.00 crores, and the contractor took almost more than 3 years to complete the work. So it is suggested that no contractor should be allowed to participate in the bidding process of more than one work whose value is either one crore or more. Procurement of equipments made in the first 2 years of Project were not put to use due to non availability of space as civil work construction was not completed. So the procurement of equipments should be made only after completion of civil works i.e. from 3rd year of the project.

68

Equipment based training should be given to doctors and paramedics to ensure proper utilization of equipment supplied by the project. This will not only help the poor patients but also will increase the collection of user charges in project hospitals to make them self sufficient. Provisions should be made in the PIP for Zonewise equipment maintenance training/centers in the State for better operationalization of project inputs. Provision should be made in the PIP for maintenance of buildings and high end equipments during the project period and State Govt. should also provide funds for building maintenance and equipments after the project period. The maintenance grant for buildings should be given to the line departments who will finally take over the buildings for maintenance. Comments on the performance of the World Bank Review Missions in implementation of the project activities: In hindsight, it can be suggested that the World Bank Mission team should be a friend and guide to the Project officials for better and timely implementation of the project activities. The "Aide Memories" of World Bank Review Missions in the first four years would show that hardly any constructive suggestions have been given for better and timely completion of the project activities. Rather they were fixing unachievable milestones and rating the performance of the Project as "Unsatisfactory" and asking questions as to why the milestones have not been achieved. World Bank took a long time for approval of plans and designs of the project hospital buildings and specifications of the equipments to be procured for the Project. This resulted slow progress of civil works in the first four years of the Project.

69

Annexure-I Statement Showing the PIP Provision, Expenditure incurred and reimbursement amount received from World Bank during the period from 1998-99 to 2005-06.

70

Annex 11. Comments of Cofinanciers and Other Partners/Stakeholders Comments from DfID: These comments were made during an interview with Sabina Barnes from DfID and were made on a superficial assessment of the project in the last two years.

The infrastructure developed by the project was satisfactory along with the equipment procured. However, it was observed that the equipment was not utilized as much. Staffing at these facilities was observed to be a key issue and sustainability of contractual staff after the project was an area of concern. User fees were being collected though there were no clear guidelines on how it was to be used. There was outsourcing of non-clinical services and cleaning services, which was satisfactory. Electricity and water supply in many facilities were not satisfactory and affected utilization and quality of services at these facilities. Training needs assessment was not done, which affected quality of training. HCWM and HMIS were not being implemented satisfactorily. Systemic changes and institutional changes were not addressed very well by the project.

71

Annex 12. List of Supporting Documents Report No. 17653-IN; Project Appraisal Document. Orissa Health Systems Development Project. SASHP, World Bank; 1998. Report No. 13042-IN; Policy and Finance Strategies for Strengthening Primary Health Care Services; World Bank, Washington DC; 1995 Report No. 13042-IN; Policy and Finance Strategies for Strengthening Primary Health Care Services; World Bank, Washington DC; 1995 Minutes of PCD Review Meeting; February 1997. India: Orissa Health Systems Development Project: Summary of Negotiations, April 22, 1998. India: Orissa Health Systems Development Project: Interim Fund Development Credit Agreement, August13, 1998. India: Orissa Health Systems Development Project: Project Agreement, August 13, 1998. Orissa Health Systems Development Project: Project Implementation Plan, April 1, 1998. Quality of Supervision Assessment, July 1999 Quality of Supervision of “Risky” projects (QSR), October 2001 Quality Enhancement Review Panel Report, July 2002 World Development Report, 1993 Country Assistance Strategy (CAS; Report No. 17241 -IN; dated December 1997) Country Assistance Strategy (CAS; Report No. 29374; dated September 2004) Aide Memoire Supervision Mission, March 2006. Aide Memoire Supervision Mission, December 2005 Aide Memoire Supervision Mission, July 2005. Aide Memoire Supervision Mission, December 2004. Aide Memoire Supervision Mission, August 2004. Aide Memoire Supervision Mission, May 2004. Aide Memoire Supervision Mission, October 2003. Aide Memoire Supervision Mission, April 2003. Aide Memoire Supervision Mission, September, 2002. Aide Memoire Supervision Mission, February-May, 2002 (Mid-Term Review Mission). Aide Memoire Supervision Mission, July 2001 Aide Memoire Supervision Mission, January 2001 Aide Memoire Supervision Mission, September 2000 Aide Memoire Supervision Mission, April 2000 Aide Memoire Supervision Mission, January 2000 Aide Memoire Supervision Mission, November 1999 Aide Memoire Supervision Mission, May 1999 Aide Memoire Supervision Mission, February 1999 Aide Memoire Appraisal Mission, January 1998 Aide Memoire Preparation Mission, November 1997 Aide Memoire Preparation Mission, July 1997

72

Aide Memoire Preparation Mission, March 1997 Orissa Health Systems Development Project Status Reports Orissa Health Systems Development Project: User Charges in Government Hospitals in Orissa: September 1999 Orissa Health Systems Development Project: An Evaluation of the Impact of Hospital Management Training: 2000-2001. Orissa Health Systems Development Project: Manual on Health Camps: February 2001 Orissa Health Systems Development Project: Report on OPD Attendance Study: May 2002 Orissa Health Systems Development Project: Proposed Operational Framework for the MOU-based Financing of Health Sector Reform and Development: June 2002 Social Assessment Study of Health Care in Orissa. Communications Management Foundation, New Delhi; 1997 Patient Satisfaction Survey; Administrative Staff College of India; 2002. Joint Survey of Status of Equipment Supplied under OHSDP. David Porter, Srinivas Taman; 2006 ISP Training under OHSDP in Tribal Districts. Aruna Bhattacharya; 2006 Report of the Comptroller and Auditor General of India, March 31, 2005, Government of Orissa

73

Map

KORAPUTKORAPUT

GANJAMGANJAM

CUTTACKCUTTACK

BAUDABAUDA

DHENKANALDHENKANAL

KEONJHARKEONJHAR

BALESHWARBALESHWAR

MAYURBHANJMAYURBHANJ

SUNDARGARHSUNDARGARH

SAMBALPURSAMBALPUR

BALANGIRBALANGIR

KALAHANDIKALAHANDI

MALKANGIRIMALKANGIRI

RAYAGADARAYAGADA

NU

APA

RHA

NU

APA

RHA

JHARSUGUDA

JHARSUGUDA

BARGARHBARGARH

PHULABANIPHULABANI

SONAPURSONAPUR ANGULANGUL BHADRAKBHADRAK

JAGATSINGHPURJAGATSINGHPUR

KENDRAPARAKENDRAPARA

JAIPURJAIPUR

KHORDHAKHORDHA

NAYAGARHNAYAGARH

NNAABBAARRAANNGGAAPPUURR

GGAA

JJAAPPAATTII

DEOGARHDEOGARH

PURIPURI

KORAPUT

GANJAM

CUTTACK

BAUDA

DHENKANAL

KEONJHAR

BALESHWAR

MAYURBHANJ

SUNDARGARH

SAMBALPUR

BALANGIR

KALAHANDI

MALKANGIRI

RAYAGADA

NU

APA

RHA

JHARSUGUDA

BARGARH

PHULABANI

SONAPUR ANGUL BHADRAK

JAGATSINGHPUR

KENDRAPARA

JAIPUR

KHORDHA

NAYAGARH

NABARANGAPUR

GA

JAPATI

DEOGARH

PURI

Koraput

Dhenkanal

Phulabani

Sundargarh

Sambalpur

Bhawanipatna

Malkangiri

RayagadaNabarangapur

Nuaparha Balangir

Jharsuguda

Bargarh

Bauda

Parlakimidi

Sonapur Angul

Baripada

Baleshwar

Bhadrak

Jagatsinghpur

Kendrapara

Jaipur

Chhatrapur

Nayagarh

Deogarh

Cuttack

Keonjhar

Rourkela

Bhubaneshwar

Puri

Ganjam

R.

Vansadhara R.

Tel R.

Mahanadi R.

Devi

BrahmaniR.

Brahmani R.

Baitarani R.Mahanadi R.

R.

Indravati

Bay

of

Ben

gal

82° 84° 86°

82° 84° 86°

22°

20°

18°

22°

20°

18°

This map was produced by the Map Design Unit of The World Bank. The boundaries, colors, denominations and any other informationshown on this map do not imply, on the part of The World BankGroup, any judgment on the legal status of any territory, or anyendorsement or acceptance of such boundaries.

INDIA

ORISSA HEALTH SYSTEMSDEVELOPMENT PROJECT

DISTRICT CAPITALS

STATE CAPITAL

DISTRICT BOUNDARIES

STATE BOUNDARY

0

0

20

20

40

40

60 80

60 MILES

100 KILOMETERS

IBRD 35098

OC

TOBER 2006