Document of The World Bank

Report No: 35391

IMPLEMENTATION COMPLETION REPORT(IDA-33400 IDA-33401)

ON A

LOAN/CREDIT/GRANT

IN THE AMOUNT OF SDR 106.5 MILLION and SDR 59.5 MILLION (US$142.6 MILLION and US$83.4 MILLION EQUIVALENT)

TO THE

INDIA

FOR AN

IMMUNIZATION STRENGTHENING PROJECT

JUNE 28, 2006

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

CURRENCY EQUIVALENTS

(Exchange Rate Effective February 1, 2006)

Currency Unit = Rupee Rupee 1 = US$ 0.02299

US$ 1 = Rupees 43.5

FISCAL YEARApril1 1 March 31

ABBREVIATIONS AND ACRONYMS

ANM Auxiliary Nurse MidwifeASHA Accredited Social Health Activist CAS Country Assistance StrategyCFC Organic Component containing carbon, fluorine and chlorine;

used for refrigerationCFMS Computerized Financial Management SystemDANIDA Danish International Development AssistanceDFID Department for International Development (United Kingdom)EC European CommissionFIC Full Immunization CoverageGOI Government of IndiaICR Implementation Completion ReportIDA International Development AssociationIEAG Indian Expert Advisory GroupIEC Information, Education, CommunicationLHW Lady Health Worker MTR Mid Term ReviewMOHFW Ministry of Health and Family Welfare, Government of IndiaNIC National Immunization CellNID National Immunization DaysNGO Non Governmental OrganizationsNRHM National Rural Health MissionNTAGI National Technical Advisory Group on ImmunizationPAD Project Appraisal DocumentPHC Primary Health CentrePIP Project Implementation PlanQER Quality Enhancement ReviewRCH I Reproductive and Child Health Project Phase IRCH II Reproductive and Child Health Project Phase IISDR Special Drawing RightsSIA Supplemental Immunization ActivitiesS-NID Sub-National Immunization DaysUNICEF United Nations Children’s Fund

UPHSDP Uttar Pradesh Health Systems development ProjectUSAID United States Agency for International Development WHO World Health Organization

Vice President: Praful C. PatelCountry Director Michael F. Carter

Sector Director/Sector Manager Julian F. Schweitzer/Anabela Abreu Task Team Leader/Task Manager: Birte Holm Sorensen

INDIAImmunization Strengthening Project

CONTENTS

Page No.1. Project Data 12. Principal Performance Ratings 13. Assessment of Development Objective and Design, and of Quality at Entry 24. Achievement of Objective and Outputs 55. Major Factors Affecting Implementation and Outcome 126. Sustainability 147. Bank and Borrower Performance 148. Lessons Learned 179. Partner Comments 1910. Additional Information 19Annex 1. Key Performance Indicators/Log Frame Matrix 20Annex 2. Project Costs and Financing 22Annex 3. Economic Costs and Benefits 24Annex 4. Bank Inputs 25Annex 5. Ratings for Achievement of Objectives/Outputs of Components 28Annex 6. Ratings of Bank and Borrower Performance 29Annex 7. List of Supporting Documents 30

Project ID: P067330 Project Name: Immunization Strengthening ProjectTeam Leader: Birte Holm Sorensen TL Unit: SASHDICR Type: Core ICR Report Date: June 29, 2006

1. Project DataName: Immunization Strengthening Project L/C/TF Number: IDA-33400; IDA-33401

Country/Department: INDIA Region: South Asia Regional Office

Sector/subsector: Health (91%); Central government administration (9%)Theme: Child health (P); Participation and civic engagement (P); Access to urban services and housing

(S); Rural services and infrastructure (S)

KEY DATES Original Revised/ActualPCD: 07/13/1999 Effective: 08/18/2000 08/18/2000

Appraisal: 12/15/1999 MTR: 11/01/2002Approval: 04/25/2000 Closing: 06/30/2004 12/31/2005

Borrower/Implementing Agency: GOVERNMENT OF INDIA/DEPARTMENT OF FAMILY WELFAREOther Partners: NA

STAFF Current At AppraisalVice President: Praful C. Patel Mieko NishimizuCountry Director: Michael F. Carter Edwin R. LimSector Manager: Anabela Abreu Richard L. SkolnikTeam Leader at ICR: Birte Holm Sorensen Indra PathmanathanICR Primary Author: Birte Holm Sorensen

2. Principal Performance Ratings

(HS=Highly Satisfactory, S=Satisfactory, U=Unsatisfactory, HL=Highly Likely, L=Likely, UN=Unlikely, HUN=Highly Unlikely, HU=Highly Unsatisfactory, H=High, SU=Substantial, M=Modest, N=Negligible)

Outcome: S

Sustainability: L

Institutional Development Impact: M

Bank Performance: U

Borrower Performance: S

QAG (if available) ICRQuality at Entry: U

Project at Risk at Any Time: NoIf the current six point rating is used the ratings would be as follows: Outcome: MS; Bank performance: MU; Borrower performance: MS; Quality at Entry: MU

3. Assessment of Development Objective and Design, and of Quality at Entry

3.1 Original Objective:Background.

Although the India Immunization Program had accomplished a great deal since inception (Polio cases declined from 24,000 in 1988 to 3,854 in 1998 and reported measles cases declined from 248,000 in 1987 to 34,000 in 1998), there was a plateau in the nineties with overall coverage of full immunization only around 55-60%. Especially the weaker states had very low coverage. In spite of the inclusion of routine polio immunization, India accounted for more than two thirds of all polio cases worldwide in 1998. Both technical and program management competence had declined, as had the quality of the cold chain equipment and transport systems required to ensure effective vaccine availability.

Achieving the global target of polio eradication by 2000, as was the target at the time, would largely depend on the performance of the India Immunization Program. The reduction in polio cases had received a boost from 1995 when National Immunization Days (NIDs) were introduced, followed by strengthened surveillance capacity including the establishment of nine laboratories with the support from Danish International Development Assistance (DANIDA). In 1998 it became clear that massive intensification of general immunization coverage as well as a specific intervention for polio eradication would be required to achieve the global target. With the cost of polio vaccine at US$14 million for each NID, and the cost of social mobilization amounting to about the same, India decided to seek assistance from the International Development Association (IDA) as well as other donors.

The Immunization Strengthening Project became effective on August 18, 2000 and was to be completed by June 30, 2004. With the supplemental credit approved on November 18, 2003, and additional funds provided through savings from existing projects the project closing date was extended until December 31, 2005.

The project development objective was to (a) eradicate poliomyelitis; and (b) reduce vaccine-preventable diseases by strengthening the routine immunization program. The project would represent the first phase of support for immunization, with a second project anticipated about two years after this project became effective, provided that performance was satisfactory.

The first objective was clear, consistent with international goals for polio eradication, timely and appropriate to the needs of the country. It reflected a global commitment shared by a number of bilateral and multilateral agencies, was well financed through a number of donors and was supported by considerable international experience to guide strategies for its achievement. The second objective was relevant since immunization is a basic public health service which government must provide; it was unlikely that polio immunization could be sustained without a general strengthening of the India Immunization Program and there was a high risk that without a special effort immunization may be reduced with all attention given to polio eradication. It was however overly ambitious to expect this objective to be achieved through the limited financial inputs of this project only. With the large and diverse number of states and institutions involved, achieving substantial improvement in immunization coverage would be a complex task. This project could be expected to contribute towards this objective with supplementary activities to be financed under other projects Reproductive and Child Health I (RCH I) and Uttar Pradesh Health Systems Development Project (UPHSDP) as is specifically mentioned in the Project Appraisal Document (PAD).

Worldwide experience (especially well documented for East Africa) indicate that immunization

- 2 -

coverage, especially in countries with weak public health systems, is likely to decline when major attention and resources are directed towards a time limited goal such as polio eradication. In this context the PAD clearly addressed the choices to be made in states with limited capacity between the campaign approach required for polio eradication and the institutional strengthening necessary for improved immunization coverage.

The project was consistent with the Bank’s Country Assistance Strategy (CAS) and its overriding objective to assist India to reduce poverty. The sector- related CAS goal was: poverty reduction and accelerated human development through improved health of poor children and women of reproductive age. Polio, as well as other vaccine preventable diseases, primarily affects the poor, women and children. They cause premature death and lifelong disability that reduced the individual’s capacity to benefit from education and employment opportunities. The Project was intended to be complementary to the Reproductive and Child Health Program, which was partially financed by IDA.

3.2 Revised Objective:The project objectives were not revised during project implementation and were maintained for the supplemental credit agreed to in 2003.

3.3 Original Components:The project had three components. The project design was clearly linked to the objectives described above, and was decided on as a result of the decline in immunization coverage during the years preceding project preparation and the link between overall immunization coverage and polio eradication. To achieve polio eradication, special efforts to improve overall immunization coverage were necessary; for this, a revised and long term strategic framework was required.

The three components were:

Component I: Polio Eradication (US$110.8 million, 69.8% of total baseline cost; in addition US$83.4million were added for this component as supplemental financing – see 5.4)

This component would finance about 50% of polio vaccine and social mobilization activities for NIDs, including information, education, and communication (IEC), training, transport, and house-to-house mop-up campaigns. The type and extent of activities and quantum of financing required would be determined each year by the summer surveillance data on polio transmission and the support mobilized from other sources.

Component II: Strengthening routine immunization (US$43.20 million, 27.2% of total baseline cost)

This component would address, selectively, critical management constraints and finance replacement of cold chain and injection safety equipment nation-wide. Management capacity strengthening would focus on filling critical vacancies, rebuilding training in immunization, and improving program monitoring and vaccine logistics. Lessons in micro-planning from the highly successful mobilization efforts for polio eradication would be captured at national, state, district, block, and village level to strengthen the routine immunization program. In eight weaker-performing states, activities would be implemented under, and supportive to, the RCH I to improve regularity of, and physical access to, community clinic sessions, that would provide immunization within a package of reproductive and child health services, improve information to local communities and mobilize communities to utilize the available services, and subsidize private and non-governmental (NGO) hospitals or clinics to provide services to urban slum communities.

- 3 -

Component III: Strategic Framework development for Vaccine Preventable Diseases (US$4.80, 3% of the total baseline cost)

This component would articulate medium-term goals and strategies for dealing with vaccine-preventable diseases. Certain guiding principles were agreed upon including that (i) the immediate priority would be polio eradication; (ii) strengthening the routine immunization would be a concurrent priority so as to avoid re-emergence of polio and (iii) whereas the Government of India (GOI) would develop national guidelines regarding approaches as well as establish principles to guide the addition of new vaccines and develop plans for technology transfer, the states would have the flexibility to develop their own immunization policies based on their individual needs. This component would support studies and demonstration projects to inform further development of policies and guiding principles, and workshops and conferences to develop national consensus on those policies and guiding principles. It was agreed that the framework would be further elaborated during the 10th Five Year Plan preparation process.

3.4 Revised Components:The components were not revised during project implementation.

3.5 Quality at Entry:Quality at entry is rated as moderately unsatisfactory based on (i) the overly ambitious objectives of full polio eradication and improved immunization coverage during the three and a half year project life; (ii) not preparing for the possibility of an increase in polio cases in 2002 which should have been expected based on the known nature of the epidemic; and (iii) lack of detailed attention to and realistic planning for implementation in the larger weaker states with known limited capacity and well known poor performance in health service delivery. It is however appreciated that there was an urgent need for additional financing for polio, where IDA would be a relatively small but critical player and financier of the urgently needed polio vaccine.

The review process included a formal review of the draft Project Concept Document in July 1999, chaired by the India Country Director, with inputs from WHO representatives, peer reviewers and Bank colleagues; an informal panel of Bank sector staff conducted the final pre-appraisal review of the draft documents on December 10, 1999. A number of important technical papers were also drafted as part of project preparation but the Bank team – understandably – did not undertake the in-depth analysis of the measures required to substantially increase the institutional capacity of the larger, weaker states for program implementation.

Both from peer reviewers and during the informal review meeting held in December 1999, it was stressed that the major goal of the first three years of the project should be polio eradication supplemented with targeted inputs to immunization strengthening while a strategic framework for immunization was prepared and a number of studies (vaccine safety and injection safety) carried out. It was also suggested that the project be supplemented with a larger, long term project to strengthen immunization once polio eradication was achieved. The reviewers finally stressed the need for special efforts to ensure performance of both components in the larger weaker states (specifically Uttar Pradesh and Bihar).

With the limited time horizon of three years (original closing date June 30, 2004) and a limited input of around US$ 40 million towards immunization strengthening, it could have been considered to limit the objectives of the project to Polio Eradication and to include the activities for strengthening immunization as one input towards achieving this objective. The potential trade-offs between the project objectives

- 4 -

could have been more clearly addressed and options prepared as to how to decide between competing demands for limited staff capacity – especially in weaker states.

Appropriate measures to address a number of assumptions and risks factors identified should have been better formulated. These factors later proved to become serious constraints to timely and effective program implementation: (i) Strong commitment to the project at GOI and state level. Experience from project implementation indicates that although there may have been a strong commitment expressed from GOI, this commitment was not there in all states and was especially week in focus states. At the time of project closing, serious efforts to improve immunization coverage are however seen; (ii) Fiscal implications of the project are relatively small. With the protracted polio epidemic, where, after the time of project closing, polio cases still remain in several states, the total expenditure for NIDs as well as Sub-National Immunization Days (S-NIDs) has escalated and necessitated substantial additional financing for this component; (iii) Risk of not eradicating polio in Uttar Pradesh and Bihar. With polio yet to be contained in these two states, the project would have benefited from more thorough work in preparing specific strategies for the weaker states; (iv) Community clinics and associated community mobilization activities may not take place under the RCH project as planned in low immunization coverage states. A specific covenant required that in focus states, at least 60% of planned (immunization) sessions were held. Compliance with this covenant was not regularly monitored but will be in the future as part of the Multi-Year Strategic Plan (2005-2011).

4. Achievement of Objective and Outputs

4.1 Outcome/achievement of objective:The consolidated rating for achievement of the Project Objective is moderately satisfactory. This rating is based on the following:

Development Objective 1: To Eradicate Polio.

The project aimed at achieving zero polio status by 2001. In the late nineties, India appeared close to this goal with continued intensive Supplemental Immunization Activities (SIA). The constantly declining polio cases during this period led to relative complacency among program managers. This, together with the poor quality and early geographic limitation of the SIAs, resulted in a large pool of unprotected children leading to a major polio outbreak in 2002 with over 1600 cases. A quick response from GOI and highly focused all around efforts in partnership with the high risk states and development partners progressively brought back the program on track with substantial reduction in the number of wild polio cases in each succeeding year. 136 polio cases were reported in 2004 and 59 cases were reported for 2005. The India Expert Advisory Group (IEAG) on Immunization meeting of December 5-6, 2005 concluded that “the introduction of mono-valent Oral Polio Vaccine 1 (OPV1) and innovative approaches to reach underserved populations have reduced polio to the lowest incidence and geographic extent ever” and that “the high quality SIAs during the current low transmission season could stop occurrence of new wild polio cases by early 2006”. For that to happen, the MOHFW and the state governments – especially Uttar Pradesh and Bihar - will need to remain focused and sustain pace and quality of efforts. While it is true that the project objective of eradicating polio is yet to be achieved 1 1/2 years after the original closing date, the excellent achievement of the GOI in sustaining a well-executed intensive eradication program guided by high quality surveillance data and the project’s contribution towards achieving the polio related development objective should be acknowledged. This justifies the rating of moderately satisfactory even if the disease eradication objective was not achieved. It is also worth noting that most of the states in India are now polio-free and that the persistent cases have been concentrated in the states of U.P. and Bihar, which are often lagging in implementation of

- 5 -

social programs. Capacities in these states to attain difficult to reach and universal targets are very limited and require long-term investments to strengthen. It is now evident that successfully going this last distance to eradication has and will require an immense effort. The GoI launched this effort in the final years of the project and is continuing it vigorously today.

Development Objective 2: To reduce vaccine-preventable diseases by strengthening the routine immunization program.

The project complemented the ongoing routine immunization strengthening initiatives under RCH I with specific inputs. Though the project largely succeeded in providing the planned inputs, the available data indicates that although a large number of states sustained high immunization coverage, immunization coverage remained either stagnant or even declined in some states. Of the target states one had an increase in coverage, one had unchanged coverage and in six states coverage decreased. Nationally there was a 2.7 % fall in full immunization coverage of children age 12 – 23 months between 1998-99 and 2002-03. The target states were the main contributors to this decrease caused by the necessary but exclusive focus on polio eradication, weak program management at all levels, deficient vaccine procurement and distribution, and varying degrees of state level commitment to routine immunization. The MOHFW has recently renewed efforts to improve immunization coverage and quality under the National Rural Health Mission (NRHM). An in-depth program review was carried out in the six weak performing states of Bihar, Jharkhand, Madhya Pradesh, Orissa, Rajasthan and Uttar Pradesh under the project during the extended period in 2005. In consultation with the states, specific actions to address the deficiencies identified in the review have begun. Recent initiatives include strengthening the Immunization Cell at the MOHFW, improving program monitoring at the state level with focus on immunization sessions, and introducing Auto-Disable syringes to improve injection safety. To provide early feedback on program performance, annual coverage evaluation surveys have been restarted with support from UNICEF. Given these and several other program improvements, routine immunization is expected to improve during the next few years. The project did succeed in providing for a comprehensive overhaul of the immunization program in India however in view of the formally stated objective and the overall decline in immunization coverage rates during the project period, the achievement of this objective is rated as moderately unsatisfactory.

4.2 Outputs by components:Component I: Polio Eradication. The implementation of this component is rated as satisfactory.

Under this component the project was to support about 50% of the costs of polio vaccine and social mobilization required to carry out the NIDs and Sub-NIDs. The number of NIDS and Sub-NIDs were to be determined annually based on surveillance data during the high transmission summer season. This data would be assessed by the Indian Expert Advisory Group, a team of international and national technical experts coordinated by the WHO, and advice provided to GOI and IDA regarding activities required for the next 12 months.

Despite a significant decline in the number of wild polio cases reported, the envisaged goal of reaching polio free status by 2001 is yet to be achieved. Some of the contributory factors for this include: (a) epidemic of Type I polio in 2002 in UP; (b) insufficient quality of supplemental immunization activities leading to accumulation of susceptible children; (c) too early introduction of geographically limited supplemental immunization activities; and (d) weak partner coordination. Many of these issues have been remedied with intensified monitoring of SIAs in states of Uttar Pradesh and Bihar, introduction of new strategies to reach underserved children such as establishment of social mobilization networks in high risk districts, insisting on presence of at least one women volunteer in vaccination teams and a

- 6 -

transit strategy to reach children who are traveling. A mechanism of periodic informal technical briefing of partners was also initiated to supplement the periodic Inter-agency Coordination Committee meetings.

As of December 24, 2005, 59 wild polio cases had been reported out of which 27 were from Uttar Pradesh and 26 from Bihar. One case each was reported from the states of Delhi, Haryana, Punjab and Uttaranchal while Jharkhand reported 2 cases. The total wild polio cases reported during the peak transmission period of August to October was only 25 compared to 68 reported in 2004 and 79 in 2003. High quality surveillance and further improvements in the sensitivity of the reporting system during the past one year, especially in Uttar Pradesh and Bihar, helped to detect transmission in several districts. With 22 cases already reported during the first quarter of 2006, there is again a small increase; India is unlikely to become polio-free as hoped for this year.

Reported Cases of Wild Polio 1998-2005

Component II: Strengthening Routine Immunization. This component is rated as moderately unsatisfactory.

This component was to support a number of activities to improve the quality of routine immunization addressing critical weakness identified in program management and selectively upgrade equipment for the cold chain and injection safety and upgrade disease surveillance capacity in selected laboratories. For service delivery, this component was to closely align with the ongoing IDA supported Reproductive and Child Health Project I (Cr. N-018) under which a specific scheme (immunization outreach) was evolved to improve routine immunization coverage focusing on eight weak performing states namely Assam, Bihar, Gujarat, Orissa, Madhya Pradesh, Uttar Pradesh and West Bengal.

While the hardware and training inputs envisaged under the project were provided to a large extent as planned, the program management improvement initiatives such as institutionalizing computerized monitoring systems, strengthening management structures and updating of program management guidelines faced delays.

The decline in routine immunization coverage reported from RCH household surveys (2003) did in 2004

- 7 -

result in a strong national response leading to 6 state immunization reviews followed by priority attention to improving routine immunization services. This was further intensified with the inclusion of immunization under the NRHM and the creation of a dedicated national immunization cell. However, the response from the states in improving routine immunization services remains mixed. Results from the ongoing coverage evaluation surveys, expected late 2006, will provide data on the impact of the new initiatives.

Percentage of children age 12-23 months who received all type of recommended vaccinations by selected states and India, round1, 1998-99 and round2, 2002-04 of RCH survey

Full immunization#Round-1 (1998-99) Round-2(2002-04)

95% Conf. Interval

95% Conf. Interval

State Percent -2SD +2SD Percent -2SD +2SD Absolute change Assam 47.4 44.7 50.1 26.8 24 29.6 -20.6*Bihar 32.3 30.2 34.3 35.9 33.3 38.4 3.6Gujarat 55.6 52.7 58.4 55.7 53 58.4 0.1Madhya Pradesh

45.7 43.6 47.7 39.5 37.5 41.5 -6.2*

Orissa 54 51.7 56.2 52.9 50.5 55.4 -1.1Rajasthan 41.1 38.9 43.2 36.8 34.5 39.1 -4.3Uttar Pradesh 45 43.5 46.5 38.4 36.9 39.9 -6.6*West Bengal 53.2 50.3 56.1 52.8 49.4 56.1 -0.4India 52.5 52 53 49.8 49.2 50.4 -2.7*

Note: Table includes only surviving children born in three years preceding the survey and estimates are based on un-weighted cases.#BCG, measles, and three doses of DPT and polio vaccines (excluding polio 0) * Statistically significant

Progress made in each of the four sub components is as follows.

Strengthening program management: Under this sub component, the MOHFW was to appoint a cold chain officer and facilitate regular program reviews at national and state levels. Other activities include development of revised immunization program management guidelines for all levels and improved program monitoring at state and district levels.

The MOHFW faced problems with positioning a regular cold chain officer during the project period. This was partly due to the operational constraints in getting an engineer deputed from other ministries and partly due to the subsequent thinking to outsource this role to a consultant agency. However, a cold chain consultant was provided under the project during the initial three years during which most cold chain equipment was procured and training of mechanics was undertaken. UNICEF has recently provided short term services of an international consultant to continue to advise GOI on issues related to cold chain management.

During project implementation, national program reviews were held fairly regularly once every year.

- 8 -

Further, a comprehensive immunization program review was carried out in six states (Bihar, Jharkhand, Orissa, Madhya Pradesh, Rajasthan and Uttar Pradesh) during 2004 in which all development partners supporting the immunization program participated. The Multi Year Strategic Plan 2005-2010 has program management guidelines clearly articulating roles and responsibilities at national, state, district and local levels. Local Bank staff as well as the regular Bank supervision missions may have helped to keep some attention focused on routine immunization although this important issue had lost some salience on the agenda of other development partners and the GoI. For much of the life of this project, these efforts were not very successful, although the situation has turned around at the end of the project period.

The GOI has given a high priority to immunization in the recently launched NRHM. To strengthen the oversight for the immunization program at national level, the MOHFW has created a National Immunization Cell (NIC) under the NRHM headed by a Joint Secretary. The Deputy Commissioner immunization has been designated as the technical advisor for this cell which has four functional units: a Vaccine and Cold chain Logistics unit headed by a full time Director; a Program Implementation unit headed by Assistant Commissioner; a Program Monitoring and Surveillance unit headed by another Assistant Commissioner; and an Administrative and Finance unit headed by a Deputy Secretary. Four support staff have been provided by development partners to the cell. A review of vaccine procurement under the Universal Immunization Program has been undertaken with support from the WHO and, based on the recommendations of this assessment, all public sector vaccine manufacturing units were brought under the control of the Immunization cell and three year vaccine requirement projections made. This will enable the vaccine manufacturers to better plan their production schedules. For the first time, orders for all vaccines have been placed in 2004-05. A core group consisting of experts from the National Polio Surveillance Project, Program for Appropriate Technologies in Health, UNICEF, WHO and Immunization Basics has been formed at MOHFW; the group meets once every week under the chairmanship of the Joint Secretary to review and guide the program.

Despite these positive developments, this review suggests that the program reviews at state level envisaged under the project were not sufficiently rigorous in details, analysis, and actionable results. Also, utilization of funds released for mobility support for supervision and organizing review meetings was slow. Delayed positioning of a regular cold chain officer has impacted the cold chain assessments for projecting future need for replacement and spares as well as exploring the introduction of new technologies using non conventional sources of energy. Finally, evidence that inputs provided under the project have translated into results in the form of improved coverage and quality of routine immunization is yet to be released.

Human Resource Development: To improve the quality and delivery of immunization services, the project envisaged three training programs: (a) mid-level managers training for state and district level officers; (b) training for cold chain mechanics and (c) training for cold chain handlers. This sub-component filled an important gap since UNICEF had recently withdrawn their long standing support to such training. The National Institute for Health & Family Welfare was given the overall responsibility to adapt the WHO mid level managers training modules for India and offer this training with support from five more institutions: Post Graduate Institute of Medical Sciences, Chandigarh, Family Welfare Training & Research Center, Mumbai, National Institute for Cholera and Enteric Diseases, Kolkata, Indian Institute of Health & Family Welfare, Hyderabad and National Institute for Communicable Diseases, New Delhi. Technical inputs are provided by a panel of immunization experts identified by the MOHFW. To ensure continuity, it was envisaged to cover 3 mid level managers from each district in the country which totals 1,856. By the end of the project, 1,786 mid level managers (96% of the target) received hands on modular training covering aspects such as micro-planning,

- 9 -

vaccine forecasting and logistics, program monitoring and evaluation, safe injection techniques, adverse event monitoring, response to vaccine preventable disease outbreaks, social mobilization and community involvement. The cold chain mechanics training was carried out in Regional Training Centre, Pune and all 321 mechanics available at the time received training in operation and maintenance of CFC free cold chain equipment. With the help of UNICEF, the cold chain handler’s training was updated and state level trainers were trained. This training was carried out at the district level by the state level trainers, and as per the available data, 7,862 cold chain handlers were trained. With the high rate of staff transfers in the states, especially of mid-level managers, such training would have to be a continuous process to ensure that all cold chain handlers are well trained at all times.

Strengthening Program monitoring. This component includes consultant support to develop software for computerized monitoring systems for vaccine supply and management information. This component was also to support additional local surveys and studies. Though consultancy contracts planned under the project could not be awarded, ultimately the software was developed with support from the WHO and field tested in Jharkhand, Rajasthan and UP. Training for implementation was near completion by the time of project closing. In addition, a routine immunization outreach session monitoring system is being field tested in Bihar and Uttar Pradesh with the state governments, National Polio Surveillance Project and UNICEF staff are actively involved in this process. The preliminary results suggest improved regularity of sessions, better availability of vaccines and increased number of beneficiaries per session. In UP about 7,000 immunization session sites were visited in October 2005 and findings show that ANMs were present and vaccines were available in over 80% of these sites. The rapid household surveys supported under RCH project provided independent assessment of immunization coverage. However, considerable delay in reporting the results from these surveys limited the utility of these surveys for programmatic changes during the project period. Consequently, the MOHFW has taken a decision to revive annual coverage evaluation surveys with support from UNICEF incorporating some qualitative components such as focus group discussions, in-depth interviews and non participant observation of immunization sessions. The first round of these surveys is in progress in 25 sites covering all major states. Strengthening of cold chain and injection safety equipment and its maintenance. Despite delays in procurement, the project was able to provide most of the critical equipment planned. During the project period over a third of the cold-chain equipment in India was replaced. This includes 13,585 Ice Lined Refrigerators, 12,330 Deep Freezers, 20,234 Voltage Stabilizers, 27 Walk in coolers and 14 Walk in Freezers and 238 Vaccine Vans.

Component III: Strategic Framework Development for Vaccine Preventable Diseases. The implementation of this component is rated as satisfactory since especially the study on injection safety was long overdue and its findings led to a policy decision of introducing Auto Disabled Syringes throughout the country.

This component was to support activities required for developing a strategic framework for India to deal with Vaccine Preventable Diseases during the next 7-10 years. In August 2001, the MOHFW constituted a National Technical Advisory Group on Immunization (NTAGI) which included experts having a wide range of technical and management skills as well as representatives from professional bodies and industry. Five sub committees were formed and the Indian Council for Medical Research provided the secretariat for this group. The five sub committees are: (i) introduction of new vaccines; (ii) operational issues including injection safety; (iii) Monitoring and Surveillance; (iv) Monitoring quality and national regulatory authority and (v) Research needs and further studies. The final recommendations of NTAGI were presented to the MOHFW on April 10, 2003. Based on this report,

- 10 -

India has evolved a Multi Year Strategic Plan 2005-2010, which was finalized in January 2005 though an extensive consultative process involving the states and other stakeholders including the development partners.

Another major activity undertaken under this component is the National Injection Safety Assessment. The study findings suggest that 63% of injections are unsafe; that the sterilization practices are questionable in 20% of injections; and that the practice of needle reuse is noted in 22% of injections. Wrong injection habits were observed in more than half the cases. The study found that the probability of unsafe injections is 1.5 times higher when glass syringes are used. Thus the risk of unsafe injections would be much higher in the Immunization Program which uses the glass syringes. The GOI has organized a series of national consultations and has taken a policy decision to use Auto Disabled Syringes in the Immunization Program from 2005. This study has thus made a significant contribution to improving injection safety in India. The recent initiative to establish model injection corners at some bigger hospitals is expected to have strong demonstration effect on injection safety. This component has disbursed about US$0.7 million equivalent against US$1 million allocated to this component at the mid-term review with the major expenditure incurred on the national injection safety assessment (US$0.66 million).

4.3 Net Present Value/Economic rate of return:No Net Present Value/economic rate of return was estimated for this ICR since the benefit is largely a global public good.

4.4 Financial rate of return:Not applicable

4.5 Institutional development impact:The institutional development impact is rated as modest. A large number of staff have gained experience with participating in a nationwide disease control program and in the better-performing states immunization rates are high and sustained. However, in the weaker states, which account for much of the shortfall in national immunization rates, ownership of the routine immunization program was weak throughout most of the period of this project. The last months of the project saw a significant effort by GoI to reverse this trend, which has resulted in visibly increased focus on the program in the target states although it is too early to predict the results of this effort.

The nationwide polio eradication efforts are of necessity centrally driven and executed in a campaign mode. Although a large share of the technical and managerial inputs are provided by WHO and UNICEF at central and state level, and implementation in the states and districts is supported by contracted staff, a large number of health professionals have gained experience from the implementation of a nationwide disease eradication program.

As mentioned above in section 4.2, utilization of funds released for mobility support for supervision and organization of review meetings for routine immunization was slow, resulting in less than optimal use of the financial and human resources and the delayed positioning of a cold chain officer further hampered the program. During the last year of project implementation, also this aspect of the program has gained renewed importance. This project made valuable contributions to preparing the groundwork for this policy shift. Immunization is seen as an important element under the National Rural Health Mission and a key contributor to achieving the Child Mortality Goal under the Millennium Development Goals. More intensive efforts at monitoring immunization coverage have been launched at the end of the project at central and state level. Local micro-planning initiatives are in a pilot phase in a number of

- 11 -

low performance states, mainly undertaken with support from WHO or UNICEF. The recent focus on central supervision and monitoring of below average performing states is a positive development. More work is needed under the NRHM if effective decentralized planning and focused attention to under-performing locations is to deliver improved outcomes; however a beginning has been made and the project has contributed to institutionalization of some sound management practices for central strategic planning, for undertaking research to inform policy decisions and for monitoring to achieve the desired outcomes.

5. Major Factors Affecting Implementation and Outcome

5.1 Factors outside the control of government or implementing agency:No adverse factors outside the control of the government or implementing agency affected the project’s implementation or outcome. The extensive support, both financial and technical, provided through the project from external partners such as, UNICEF, WHO, EC, DFID and USAID was a very favorable factor. WHO and UNICEF played an important role in project formulation and continued to provide substantial technical inputs in cold chain management, vaccine procurement and general epidemiological guidance and technical support. The main factor ensuring availability of polio vaccine was that this was procured by UNICEF.

5.2 Factors generally subject to government control:The re-occurrence of more than 1000 polio cases in 2002, after 265 in 2000 and 268 in 2001 was profoundly disappointing for the health authorities in India as well as for the global polio eradication efforts. The epidemiological history of polio in India has shown major epidemics roughly every five years; the last epidemic was in 1998-99 so the increase in polio cases in 2002 could have been anticipated. A number of factors further contributed to this situation. With continued poor coverage of routine immunization, down to below 20% of eligible children in some locations, and insufficient quality of the supplementary immunization in the week states (mainly Bihar and Uttar Pradesh), pockets of uncovered populations, large enough to sustain transmission were left unprotected. Limiting the scope of SIAs was an unfortunate decision from the center along with the limited focus on support to the weaker states. The managerial response from the states, in the form of preparing plans of action for polio as well as routine immunization improvement, social mobilization and advocacy was also inadequate for the task at hand. Given India’s overall capabilities and resources, this situation may have been avoided had it been adequately planned for during project preparation and had there been clear political and administrative commitment to do so.

Recognizing the poor performance of routine immunization in a number of large states, improved routine immunization forms an important element in the overall National Rural Health Mission. The result of this renewed focus is especially evident from the increased importance given to immunization during the last year of the project and in the states, where new ‘Work Plans for Strengthening Immunization’ and ‘District level Micro-Planning Initiatives’ are now initiated and regularly monitored at the highest level in a number of low coverage states.

5.3 Factors generally subject to implementing agency control:Central level. 75% of the credit, the major contribution of the project, was allocated to the procurement of polio vaccine. This was done through UNICEF and the procurement and supply performance was excellent. The commissioning of the Medium Term Strategic Framework through the National Technical Advisory Group as well as commissioning the India Injection Safety Assessment that lead to introduction of the Auto Disable were also excellent initiatives. These activities have clearly provided a strategic direction for improvement of routine immunization as well as criteria for inclusion of new vaccines in the general program but only now, when the project has closed. The implementing agency was less efficient in

- 12 -

executing its role of providing routine vaccines, cold chain equipment and funds on time and of ensuring state level overall performance monitoring. Unexplained delays in award of contracts resulted in delayed supply, one case of mis-procurement and a number of cases of cancellation of procurement of cold chain equipment. Since the cold chain provides the backbone of immunization programs, the position of cold chain consultant was crucial as was the computerized monitoring system which is now at the pilot stage.

State level. Field visits indicated that whereas many states have fully internalized program responsibility, in a number of states the managerial response of the state government to assuring improved immunization coverage still needs improvement. Although effective polio eradication/surveillance efforts are now evident and there is a renewed focus on improving immunization, the local UNICEF and/or WHO office and staff still play a leading role. State and district level immunization officers (as well as officers of other health programs) often have multiple tasks, are frequently transferred and are not selected based on the technical and managerial skills required to undertake this important task.

Covenants. Whereas all other important covenants were adhered to, two covenants were not fully complied with: (i) By December 31, 2000 ensure that at least 80% of sanctioned posts for the family welfare program at the state and district levels would be filled and an officer from each district designated to be responsible for immunization; and (ii) fill at least 80% of sanctioned Auxiliary Nurse Midwife (ANM) positions by December 31, 2002 and, in the event that sufficient candidates are not available to fill such posts, by March 31, 2002 to recruit adequate number of candidates in government training institutionsto ensure filling up of the desired posts. In Uttar Pradesh, where recruitment and training of ANMs had been stopped around 1990, this has now been resumed in recognition of the unfortunate impact on core programs such as the routine immunization. It is questionable if a covenant, with such wide ranging consequences for state public health budgets could be expected to be fulfilled under this project.

Financial management issues. A significant part of the funds utilized (over 75%) in the project were advanced to UNICEF for procurement of Polio Vaccines and quarterly financial reports on fund utilization were received on a regular and timely basis. While the project design had provided for the implementation of a computerized financial management system (CFMS) across the states, the experience (and lack of success) with similar efforts in other centrally sponsored projects, led to a decision not to implement the CFMS. The project faced delays in submission of audit reports from the states for the polio operational expenses on a regular basis leading to suspension of disbursements in one instance. Audit reports from six states are pending for the year 2004-05 as on date. Because there were multiple donors financing the same set of activities, the project had the flexibility to charge expenses to various donors (based on annual grant availability) and ensure continuous funds flow to the states.

5.4 Costs and financing:At appraisal the total cost of the project was estimated to be US$158.8 million with about US$146.2 million (SDR106.5 million equivalent) financed from the IDA credit. Due to the unexpected increase in polio cases in 2002, the GOI requested IDA to provide a supplemental credit of US$83.4 million (SDR59.5 million equivalent). In addition, GOI also requested IDA to reallocate savings amounting to US$93.7 million (SDR65.45 million equivalent) from other on-going health projects to fill critical polio eradication financing gaps. The supplemental credit was subsequently approved on December 16, 2003 to finance the additional cost of polio vaccine. The credit amount of US$333.1 million (SDR231.5 million equivalent) including the supplemental credit and reallocations from various IDA projects is fully utilized at the end of project.

6. Sustainability

- 13 -

6.1 Rationale for sustainability rating:The sustainability of the project is rated as likely.

With the international and national focus on polio eradication, and with the Governments strong commitment to eradicate polio in India, it is highly likely that India will be free of polio cases within one or two years. The experience with re-emergence of a large number of cases in 2002 has resulted in an ‘all hands on deck’ mentality at national and state level, especially in Uttar Pradesh and Bihar. With strong political commitment the GOI will be in a position to finance the remainder of the program costs.

The project has provided the foundation for strengthening of routine immunization through the strategic framework and multi-year plan, the injection safety study and the focus on close review of performance in weaker states as well as weaker districts within those states. With immunization strengthening being an integral part of the new NRHM, it should now receive political attention as well as resources. The inclusion of the field level volunteer, the ASHA, to be introduced under this scheme is likely to have a positive impact on immunization coverage (see experience from Pakistan 7.7). The Immunization Cell at the MOHFW has been strengthened in the past year and experience from field visits to a number of states indicate that with strong political backing, public health, including immunization coverage is, perhaps for the first time in India, present on the political agenda.

6.2 Transition arrangement to regular operations:Immunization is proposed to receive multi-donor including IDA financing through the already negotiated Reproductive and Child Health Project II, which is also a part of the National Rural Health Mission. Hence transition to regular operations should be assured.

7. Bank and Borrower Performance

Bank7.1 Lending:

The Bank’s performance in the identification, preparation and appraisal of the project is rated as moderately unsatisfactory.

Following the preparation and start-up of the RCH I project, the Bank had a good working relationship with the Borrower which enabled both parties to prepare the project quickly (identification mission August 1999 and Board presentation April 2000) to cater to the urgent need for additional financing of polio vaccine. In addition, as the project was being prepared it was realized that the recently prepared RCH I project already had a component for Nationwide Improved Quality, Coverage and Effectiveness, and Expanded Content of the Essential Package of RCH Services. An outreach sub-component was to be added and the Immunization Strengthening Project was to be an addition to this component.

The preparation of the project was done in close collaboration with WHO and UNICEF, benefiting from their extensive experience with similar activities both globally and in India. The preparation did include a number of field visits and meetings with officials from the high priority states as well as review of state implementation plans for those states.

Due to the urgent need for financing of polio vaccine, the preparation did not specifically address the institutional issues and social or environmental assessments were not separately undertaken. Preparation of specific District Actions Plans for weaker states including Tribal Plans to strengthen immunization, were to be prepared under the RCH I Project.

The project would have benefitted from more attention to institutional as well as management issues

- 14 -

such as specifying the specific responsibilities at center, state and district level as well as the issue of ownership of project objectives at state and district level.

More attention given to the epidemiology of polio in India as well as the negative effect of low immunization coverage in large states such as Bihar and Uttar Pradesh on polio eradication may have prevented or reduced the increase in polio cases seen in 2002.

Finally, the preparation process would have benefited from further discussion of the project objectives. With global experience of polio eradication efforts at the time and with an investment of only slightly over $40 million it could have been predicted that the project’s ambitious objectives of both polio eradication and strengthening of routine immunization would be at risk. Stating the objectives in more realistic terms, especially while most efforts, resources and attention were on polio eradication, would have lead to more realistic expectations. This mismatch between unrealistic objectives and a project that actually achieved some difficult results is seen mainly as an issue of weak or hasty preparation on the Bank side to accommodate the urgent need for financing of polio vaccine.

7.2 Supervision:Although there was a good relationship between the Bank and the borrower and international expertise was brought in to assist the Government, on balance the Bank’s performance during supervision is rated as moderately unsatisfactory for the reasons given below.

Regular supervision missions were held and the missions had a good mix of expertise; most missions included technical experts from WHO and/or UNICEF which ensured that sound technical advice was provided. Benchmarks were agreed to during each mission with subsequent follow-up of achievement. Once it was clear that routine immunization was lacking, the task team brought in technical experts in routine immunization to lead the dialogue with the government. The mid-term review (MTR) held immediately after the outbreak of an increasing number of polio cases adequately provided an opportunity to focus on the plan of action prepared based on the advice of the India Expert Advisory Group. On hindsight, it was however unfortunate that the earlier supervision teams had not identified the technical and managerial deficiencies which contributed to this increase in polio cases, before they occurred. It is also unfortunate that the supervision teams did not clearly identify the mismatch between the stated objective of improving immunization coverage and the limited interventions and efforts to this effect under the project.

The preliminary results from the RCH survey in mid-2003 indicated a general decline in immunization coverage, especially in the eight target states. In spite of this information, there was little change in the focus on this component of the project. Three factors are likely to account for this: (i) The considerable national and international attention to reaching polio free status for India left little space to generate the strong political commitment, the state and district ownership and the detailed planning required to improved immunization coverage across these states; (ii) the inputs from this project were seen as additional to the RCH I, and subsequently planned RCH II project and therefore not seen as central to this project; (iii) it appears as if no one (Bank or GOI team) expected this project to contribute in a major way to improved immunization - in spite of the stated objective. The Bank task team did not sufficiently focus on addressing this component as an equal priority.

In spite of the perceived poor immunization coverage in target states from project inception and the documented reduction in immunization coverage in 2003, the achievement of the Development Objective was rated as satisfactory until May 2005 when it was rated as moderately satisfactory. The text ofearlier Aide Memoire’s repeatedly stated that the second objective was unlikely to be achieved but this

- 15 -

did not affect the overall rating and did not lead to either a restructuring with a more realistic objective or the addition of process indicators towards the second objective so that at least progress could be monitored even if it was clear that the objective could not be achieved.

The indicator by which to measure the achievement of the second objective was to be collected through the RCH surveys; one was conducted before the project start and used as a baseline and the second was to be conducted in 2002. These surveys could have been supported through actions to strengthen the quality of the regular performance monitoring.

During several supervision missions, the Bank team rated the Borrower's procurement performance as unsatisfactory due to the lengthy process of award of contract. The Bank team however upgraded the rating of the Borrowers procurement performance to satisfactory. When mis-procurement of one contract for Deep Freezers was declared on May 24, 2005, the rating was not changed. One contract for supply of Voltage Stabilizers (small) was ex-post reviewed by the procurement team in March 2003; value of contract was Rupees 6,923,000. A general post-review of all IDA financed projects is now ongoing in the states; contracts financed under this project will form part of this overall post-review.

7.3 Overall Bank performance:Overall Bank performance is rated as moderately unsatisfactory. The preparation was overly optimistic while stating the objectives which could be achieved through this operation; in addition, it did not address a number of institutional and management issues which later proved to present important obstacles to project performance. Supervision missions provided excellent technical support to polio eradication and consistently pursued with the government the importance of more focused attention to addressing the declining immunization coverage. The task team did however not pro-actively address the fact that the objectives were unlikely to be achieved – even though this could have been foreseen during preparation and was clear from an early stage of the project. On hind-sight it is clear that the project would have benefitted from restructuring.

Borrower7.4 Preparation:

The Borrowers overall performance on preparation was moderately satisfactory. The Borrower urgently needed financing for polio vaccines and also acknowledged the need to strengthen routine immunization – both as an important public service and as a means to eradicate polio. This however was not well reflected in the project proposal where the inputs to this second objective were limited both in terms of financing and scope. The necessary documents including state level PIPs for the target states were prepared and discussed with the Bank team during preparation. A community component to ensure sustainability of efforts was also originally proposed but not agreed to by the borrower during negotiations; this proved to be an unfortunate choice. The overall impression based on existing documentation is that the Borrower urgently needed financing for polio vaccine and that other elements of the project were less carefully conceived.

7.5 Government implementation performance:There was - and is - strong political support for the polio eradication agenda. The GoI has mobilized a truly massive effort to try to achieve its contribution to global polio eradication. In large parts of the India, this effort has been successful. Compared to the situation at the start of the project, only a handful of states have continued to report a greatly reduced number of cases and most Indian states have been polio free for a number of years. The GoI's performance was not without faults. More attention to the risks of not achieving polio free status by 2003 would have been desirable and could have better foreseen the temporary resurgence of infections at that time. However, the government's

- 16 -

response to the increase in polio cases in 2002 with strong political backing was a significant renewed effort to address the situation. The central government mobilized substantial financial and human resources from the donor community (including the supplemental and re-allocated credit from IDA) and through this support provided technical assistance to all large states to ensure that the required polio eradication interventions were adequately implemented and monitored. In addition, the central government generated the required political support and managerial response from the important states. Nationwide the information about polio eradication became household knowledge.

While all attention was on addressing polio eradication, a few strategic actions towards the second objective at the end of the extended project period allowed the government to make timely policy decisions based on the findings of the multi year strategic plan for routine immunization as well as the injection safety study. This brought new focus towards immunization strengthening through the politically high profile NRHM. With the strategic ground work done at the center, it was possible to focus on encouraging the states to respond to this need for re-vamping the immunization program.

The Government was less successful in translating its strong political support into ensuring adequate human resources. Till date, there is little attention given in most states to ensuring availability of adequate human resources at all levels for routine immunization. The necessary strengthening of program management at center and state, of program monitoring and of ensuring continues vaccine availability is only now being initiated. The case of Uttar Pradesh, where training of the most important staff for immunization, the ANM, has only recently been re-started after it was discontinued in 1992 is a case in point. Important posts have remained vacant both at the center and in the states, where bureaucratic procedure still takes precedence over ensuring that adequate technical and leadership skills are available for important programs. Although this is a general problem in the sector and not specific to the Immunization Program, lack of leadership as well as important field staff in the states does seriously hamper sustainability of program interventions. The overall performance of the Government in implementation is therefore rated as moderately satisfactory.

7.6 Implementing Agency:It is evident that the major emphasis during the project period from the central and state implementing agents was on polio eradication. This dominated the efforts of the implementing agencies throughout the project period, despite the attention also needed on routine immunization. The number of NIDs and Sub-NIDs was increased to improve effectiveness, thereby also increasing the task of ensuring availability of polio vaccine throughout the country; mono-valent oral polio vaccine was introduced to increase the effectiveness of the NIDs; mid-level managers were trained and state level performance reviews introduced. There was some delay in introducing new strategies such as reaching underserved children; sociologically appropriate communication to reach all communities; establishment of social mobilization networks in high risk districts; insisting on presence of at least one woman volunteer in vaccination teams, ensuring easy access to vaccine for ANMs and a transit strategy to reach children who are traveling. All these actions were however introduced from 2003, when it became evident that much more was required to achieve the desired goal. This was a major task which required substantial leadership and logistical skills from the implementing agents at central and state levels. It is this additional and nationwide attention to detail and to reaching all children which will eventually lead to reaching total polio eradication.

The proposed activities to strengthen routine immunization were carried out, but with the huge task of managing polio eradication, there was limited capacity to ensure or measure if these activities were productive. The major activities to improve immunization strengthening, in which the implementing agency invested substantial efforts, were the injection safety study and the strategic planning. These

- 17 -

efforts are likely to have a long term positive impact on the program.

For the more than 75% of the credit spent on polio vaccine purchased by UNICEF quarterly financial reports on fund utilization were received on a regular basis; however, as mentioned earlier (5.3) the project faced a number of delays in submission of audit reports from the state portion of the expenditures leading to suspension of disbursements in one instance.

A case of mis-procurement was declared in May 2005; the Bank did not agree with the Borrowers proposal to reject all bids for deep freezers due to suspected non-availability of adequate maintenance services in the country. Fortunately polio vaccine, which was the major item purchased, was procured through UNICEF and this procurement was at all times satisfactory.

Two important covenants were not fulfilled: (i) By December 31, 2000 ensure that at least 80% of sanctioned posts for the family welfare program at the state and district levels would be filled and an officer from each district designated to be responsible for immunization; and (ii) fill at least 80% of sanctioned Auxiliary Nurse Midwife (ANM) positions by December 31, 2002. It is questionable if a covenant, with such wide ranging consequences for state public health budgets could be expected to be fulfilled under this project.

The overall rating of the central and state implementing agencies is moderately satisfactory.

7.7 Overall Borrower performance:Whereas the central government is responsible for strategic planning as well as ensuring availability of supplies, the states are responsible for implementation. With all attention on managing the NIDs, especially the weaker states generally did not have adequate capacity to adequately their implementation responsibility regarding routine immunization. Fortunately this has changed in the last year of the project, where renewed attention is given to routine immunization. This effort is however to a large extent dependent on WHO and UNICEF support with Technical Assistance provided at the state level. Although a few large states have not performed well, the rating of the government and the implementing agencies should also reflect the larger progress in India as a whole. The rating therefore is moderately satisfactory.

8. Lessons Learned

Technical.

· Technically sound research in important program areas can lead to better policies. Injection safety research under this project has led to a decision to introduce auto-disabled syringes for the immunization program nationwide. This step could have major positive implications in immunization quality and effectiveness in India.

· Successful monitoring of progress with a large-scale priority program of this kind needs much greater attention to data sources, timeliness, and quality. Survey data collected once in three to four years and unreliable service statistics were not adequate for program monitoring and mid-course corrections. It has been generally proven that service statistics on immunization coverage are not entirely unreliable and not substantiated by survey results, particularly in the worse performing areas. In such cases there is a need to improve on the service statistics including internal surveillance and controls as well as having more frequent sample surveys to ensure that program managers at all times

- 18 -

have current information about program performance.

· Constant and independent technical guidance is critical for disease control programs. For a disease eradication program, it is critical to pool the best available technical talent to guide program planning and implementation based on high quality surveillance data. The polio program has been guided by the IEAG, a body of eminent national and international experts, using surveillance data for decision making which is resulting in rapidly reduced polio cases. Strong technical assistance placed in target states early in the project may have positively affected the project outcome.

· A high level of political awareness and attention is a major asset in reinvigorating a large, and lagging, existing program as has been seen with the focus on immunization through the NRHM. This speaks to the inclusion of much larger advocacy components in such programs.

Institutional and management

· Clearly delineated roles and responsibilities of center and states and follow up to ensure that these roles and functions are being adequately executed will ensure better results. For a nation-wide project covering the size and diversity that is India, there should be a clear demarcation of management functions between the centre and the states. Planned outcomes under this project depend on effective service delivery to be managed by state level with strong support in vaccine and cold chain procurement and monitoring and overall strategic planning from the center. Targeted management support at an early stage to less well performing states could have made a timely contribution to better outcomes

· Shared vision will strengthen partnership between the government and the development partners. The polio eradication program in India is an outstanding example of a strong partnership between the government and the development partners resulting in sound planning and management of the program. The massive financial resources for supplemental immunization activities as well as the recent technical steering of the program has been greatly facilitated by the strong partnership and frequent interaction between the government, donors and technical agencies.

Operational.

· Development objectives must be carefully calibrated to reflect the scope and inputs supported under the project. During project preparation the project objectives should be rooted in what can be achieved from the project inputs. Given the size of India’s routine immunization program, a proposed support of US$40 million, later reduced to less than half that amount, can at best be one input into the overall program effort. Given inputs provided under the project, the objective of “reducing vaccine preventable diseases country-wide” was rather ambitious. If the ‘real’ intention of the project was to support polio eradication onlythen the objectives should have stated that clearly; objectives stated in project documents must be seriously pursued.

· Adequate attention must be paid up-front to operational issues, such as procurement. The project suffered from continuous procurement delays; an early assessment of capacity, or of any other constraints, followed by an action plan, might have mitigated the underlying procurement problems at an early implementation stage.

9. Partner Comments

- 19 -

(a) Borrower/implementing agency:While the draft ICR has been shared with the MOHFW, no comments have been received from them. When the Borrowers comments are received they will be filed in the project files.

(b) Cofinanciers:While there are other financiers of Immunization under the RCH and of specific Polio Eradication efforts, no other partners financed this project; the ICR team met with DFID, EC, USAID, WHO and UNICEF. They did not have an opinion on the Bank's or the GOI's performance in this particular project.

(c) Other partners (NGOs/private sector):

10. Additional Information

In spite of repeated requests the MOHFW has not provided its ICR

- 20 -

Annex 1. Key Performance Indicators/Log Frame Matrix

Outcome / Impact Indicators:

Indicator/Matrix

Projected in last PSR1

Actual/Latest Estimate

No new polio case reported after 2001. No new Polio case caused by wild virus after 2001. (Baseline -- 1126 Wild Polio cases in 1999)

136 wild Polio virus cases reported in 2004; and 59 cases in 2005 by December 24, 2005

Percent of infants fully immunized children by age one with six basic vaccines would increase from 55.3% in 1999 to 60% in 2003.

% of infants fully immunized by age one with 6 basic vaccines to be increased to 60% (Baseline - 55.3% in 1999)

% of infants fully immunized by age one with 6 basic vaccines in 2003: 47.6%

Output Indicators:

Indicator/Matrix

Projected in last PSR1

Actual/Latest Estimate

Reporting of non-polio acute flaccid paralysis (AFP) at the rate of 1 per 100,000 children below 15 years.

Non polio AFP rate > 1/100,000 children below 15 years. (Baseline was 1.83/100,000)

Non polio AFP rate: 5.5/100,000 children below 15 years in 2005

Stool collection from at least 60% of acute flaccid paralysis cases within 14 days.

Stool samples collected from >80% of AFP cases within 14 days (Baseline was 71% )

Stool samples collected from 82% of AFP cases within 14 days in 2005

Percent of districts nationwide with 80% coverage of fully immunized children under 1 year increased from 20% in 2000 to 25% in 2003.

Percent of districts nationwide with 80% coverage of fully immunized children under 1 year increased to 25% in 2003.

The 2003-04 RCH household surveys covering 570 districts estimate that only 16% districts have > 80% coverage

Increase by 5% of fully immunized children under age one by 2003 in each of the Project focus states.

Increase by 5% of fully immunized children under age one by 2003 in each of the Project focus states. (Baseline in 1998-99) Assam: 45 Bihar: 18 Gujarat: 55 Madhya Pradesh: 42 Orissa: 60 Rajasthan: 35 Uttar Pradesh: 39 West Bengal: 52

Fully immunized children under age one in 2002 in each of the Project focus states are the following: Assam: 20 Bihar: 23 Gujarat: 54 Madhya Pradesh: 31 Orissa: 54 Rajasthan: 25 Uttar Pradesh: 27 West Bengal: 52Increased coverage in 3 states and decreased coverage in 5 states.

At least 25% of districts in the eight project focus states achieve 10 % decline in dropout rates by 2003

At least 25% of districts in the eight project focus states achieve 10 % decline in dropout rates by 2003

The % of districts with a 10% decline in dropout rate between BCG to Measles of various states based on the district level survey report of 2002-03 are the following:Assam: 86Bihar: 17Gujarat: 32Madhya Pradesh: 58Orissa: 28Rajasthan: 43Uttar Pradesh: 26West Bengal: 22

Improved cold chain and injection safety equipment

Improved percent PHCs having sterilizers. Acceptable percentage planned cold chain equipment installed. (This information is not available)

Under the project, the following items were provided: (i) 12,190 of Ice Lined Refrigerators (Small), represent 38% of small Ice Lined Refrigerators in place;(ii) 1,395 Ice Lined Refrigerators (Large)

- 21 -

represent 67% large Ice Lined Refrigerators in place;(iii) 10,192 Deep Freezers (Small) represent 34% of small Deep Freezers in place;(iv) 2,138 Deep Freezers (Large) represent 55% of large Deep Freezers in place;(v) 20,334 Voltage Stabilizers, represent 35% of Voltage Stabilizers in place; (vi) 27 Walk in Coolers, represent 19% of Walk Coolers in place; (vii)14 walk in Freezers, represent 50% of Walk in Freesers in place; and (viii) 238 Vaccine Vans, represent 25% of Vaccine Vans in place.

Strategic Framework in place by 2003. Working group established by 2001.Annual progress reports on development and implementation of policy, guidelines, strategies and studies.

National Technical Advisory Group (NTAG) formed in August 2001Multi Year Strategic plan for Universal Immunization (2005-2010) finalized in consultation with the states in 2005. Based on the findings of Injection Safety Assessment, policy decision was taken to introduce AD syringes in immunization from FY 2005. This would be initially funded by GAVI and subsequently by the RCH program.

1 End of project

- 22 -

Annex 2. Project Costs and Financing

Project Cost by Component (in US$ million equivalent)AppraisalEstimate

Actual/Latest Estimate

Percentage of Appraisal

Component US$ million US$ millionPolio Eradication Vaccine 98.90 273.66 277 Social Mobilization 9.60 20.80 217Strengthening Routine Immunization 39.60 51.00 129Strategic Framework Development for VPDs 4.50 1.73 38

Total Baseline Cost 152.60 347.19 Physical Contingencies 2.90 Price Contingencies 3.30

Total Project Costs 158.80 347.19Total Financing Required 158.80 347.19

Note: Although additional financing had been added to the IDA Credit, the total GOI actual expenditure under the project was not available. Therefore, total cost shown above only includes GOI's original financing.

Project Costs by Procurement Arrangements (Appraisal Estimate) (US$ million equivalent)

Expenditure Category ICBProcurement

NCB Method

1

Other2 N.B.F. Total Cost

1. Works 0.00 0.00 0.00 0.00 0.00(0.00) (0.00) (0.00) (0.00) (0.00)

2. Goods 38.65 4.45 108.90 0.00 152.00(32.89) (3.46) (99.91) (0.00) (136.26)

3. Services 0.00 0.00 5.80 0.00 5.80(0.00) (0.00) (5.64) (0.00) (5.64)

4. Incremental Operating Cost

0.00 0.00 1.00 0.00 1.00

(0.00) (0.00) (0.70) (0.00) (0.70) Total 38.65 4.45 115.70 0.00 158.80

(32.89) (3.46) (106.25) (0.00) (142.60)

- 23 -

Project Costs by Procurement Arrangements (Actual/Latest Estimate) (US$ million equivalent)

Expenditure Category ICBProcurement

NCB Method

1

Other2 N.B.F. Total Cost

1. Works 0.00 0.00 0.00 0.00 0.00(0.00) (0.00) (0.00) (0.00) (0.00)

2. Goods 317.15 4.30 15.54 0.00 336.99(313.16) (0.00) (15.20) (0.00) (328.36)

3. Services 0.00 0.00 5.80 0.00 5.80(0.00) (0.00) (1.00) (0.00) (1.00)

4. Incremental Operating Cost

0.00 0.00 4.40 0.00 4.40

(0.00) (0.00) (1.63) (0.00) (1.63) Total 317.15 4.30 25.74 0.00 347.19

(313.16) (0.00) (17.83) (0.00) (330.99)

1/ Figures in parenthesis are the amounts to be financed by the Bank Loan. All costs include contingencies.2/ Includes civil works and goods to be procured through national shopping, consulting services, services of contracted staff

of the project management office, training, technical assistance services, and incremental operating costs related to (i) managing the project, and (ii) re-lending project funds to local government units.

Project Financing by Component (in US$ million equivalent)

Component Appraisal Estimate Actual/Latest EstimatePercentage of Appraisal

IDA Govt. CoF. IDA Govt. CoF. IDA Govt. CoF.Polio Eradication Activities 98.50 9.00 313.16 10.50 317.9 116.7Goods (including pharmaceuticals, vehicles, Equipment, MIS/IEC materials and supplies

37.75 6.75 15.20 5.60 40.3 83.0

Training and Consultants and NGO Services

5.65 0.15 1.00 0.00 17.7 0.0

Incremental Operating Costs

0.70 0.30 1.63 0.10 232.9 33.3

Total 142.60 16.20 330.99 16.20 232.1 100.0

1/ As of February 20, 2006, a total amount of US$330.99 was disbursed from IDA for the Project. The original Credit amount was SDR 106.5 million (US$142.6 million equivalent) and a supplemental financing of SDR 59.6 million (US$87.4 million equivalent) was approved in November 2003. In addition, there were other allocations from five on-going projects. These include US$9.75 from Malaria Control Project (Cr.2964-IN), US$17.7 million from Women & Child Development Project (Cr. N-042-IN); US$10.3 million from Tuberculosis Project (Cr. 2936-IN); US$24.4 million from Orissa Health System Project (Cr. N-041-IN), and US$19.5 million from Gujarat Emergency Earthquake Project.

2/ Although additional financing had been added to the IDA Credit, the total GOI actual expenditure under the project was not available. Therefore, the actual/latest estimates shown above only include GOI's original financing.

- 24 -

Annex 3. Economic Costs and Benefits

NA

- 25 -

Annex 4. Bank Inputs

(a) Missions:Stage of Project Cycle Performance Rating No. of Persons and Specialty

(e.g. 2 Economists, 1 FMS, etc.)Month/Year Count Specialty

ImplementationProgress

DevelopmentObjective

Identification/PreparationAugust 1999 6 Task Leader (1); Public Health

Spec. (2); Management Spec. (2); Financial Man. Spec. (1)

October 1999 11 Task Leader (1); Public Health Spec. (1); Management Spec. (2); Sr. Economist (1); Social dev. Spec (1); Training Spec. (1) Procurement Spec. (1) Financial Man. Spec. (1); WHO specialist (2)

Appraisal/Negotiation01/07/2000 12 Task Leader (1); Team

Leader/Lead Economist (1); Public Health Spec. (1); Management Spec. (2); Sr. Economist (1);Soc. Dev. Spec (1); Training Spec. (1); Procurement Spec. (1); Financial Man. Spec. (1); WHO specialist (1)

Supervision06/15/2000 10 Task Leader (1); Team

Leader/Lead Economist (1); Sr. Public Health Spec (2);Training Specialist (1); Planning and Monitoring Specialist (1); IEC specialist (1); Management Specialist (1); Sr. Procurement Specialist (1); Sr. Financial Management Specialist (1)

11/17/2000 5 Task Leader (1); Team Leader (1); Sr. Public Health Spec (1); Sr. Finance Management (1); Sr. Procurement Specialist (1)

S S

05/30/2001 6 Task Leader (1); Team Leader (1); Sr. Public Health Specialist (2); Sr. Procurement Specialist (1); Sr. Financial Mngement. Specialist (1)

S S

11/19/2001 6 Task Leader (1); Sr. Public Health (2); Procurement (1); Financial Management (1);

S S

- 26 -

Mission Member, WHO (1)05/20/2002 8 Task Leader (1); Team

Leader/Lead Economist (1); Sr. Public Health Specialist (1); Sr. Financial Management Specialist (1); Sr. Procurement Specialist (1); Consultants (3)

S S

11/08/2002 4 Task Leader/Sr. Public. Health Specialist (1); Sr. Procurement Specialist. (1); Financial Management Specialist. (1); Immunization Adviser (1)

S S

06/19/2003 5 Task Leader (1); Sr. Public Health Specialist (1); Sr. Procurement Specialist. (1); Financial Management Specialist. (1); Management Specialist, WHO (1)

S S

12/01/2003 5 Task Leader (1); Sr. Public. Health Specialist (1); Sr. Procurement Specialist. (1); Financial Management Specialist. (1); Management Specialist, WHO (1)

S S

05/03/2004 6 Sr. Public. Health. Specialist (2); Sr. Health. Specialist (1); Sr. Procurement Specialist (1); Financial Management Specialist (1); Team Assistant (1)

S S

09/03/2004 2 Sr. Public Health Spec (1); Sr. Public Health Specialist (1)

S S

04/20/2005 5 Task Leader (1); Sr. Public Health Specialist (1); Sr. Procurement Specialist. (1); Sr. Financial Management Specialist. (1); Team Assistant (1)

S S

12/07/2005 6 Task Leader (1); Sr. Public Health Specialist (1); Sr. Nutrition Specialist (1); Sr. Procurement Specialist. (1); Sr. Financial Management Specialist. (1); Team Assistant (1)

S S

ICR01/30/2006 2 Sr. Public Health Specialist

(1); Operations Officer (1) S S

If the current six point rating was used the ratings would be as follows: Outcome: MS; Bank performance: MU; Borrower performance: MS. The latest ISR ratings were: Objective: MS; Component Ratings: S; MU; S

- 27 -

(b) Staff:

Stage of Project Cycle Actual/Latest EstimateNo. Staff weeks US$ ('000)

Identification/PreparationAppraisal/Negotiation 105.0 419.6Supervision 89.0 356.5ICR 24.0 97.1Total 218.0 873.2

Note: Staff week data prior to FY2001 is not available. The preparation/appraisal staff week number above is an estimate at an average of US$4,000 per staff week.

- 28 -

Annex 5. Ratings for Achievement of Objectives/Outputs of Components(H=High, SU=Substantial, M=Modest, N=Negligible, NA=Not Applicable)

RatingMacro policies H SU M N NASector Policies H SU M N NAPhysical H SU M N NAFinancial H SU M N NAInstitutional Development H SU M N NAEnvironmental H SU M N NA

SocialPoverty Reduction H SU M N NAGender H SU M N NAOther (Please specify) H SU M N NA

Private sector development H SU M N NAPublic sector management H SU M N NAOther (Please specify) H SU M N NA

- 29 -

Annex 6. Ratings of Bank and Borrower Performance

(HS=Highly Satisfactory, S=Satisfactory, U=Unsatisfactory, HU=Highly Unsatisfactory)

6.1 Bank performance Rating

Lending HS S U HUSupervision HS S U HUOverall HS S U HU

6.2 Borrower performance Rating

Preparation HS S U HUGovernment implementation performance HS S U HUImplementation agency performance HS S U HUOverall HS S U HU

If the current six point rating was used the ratings would be as follows: Bank performance in Lending: MU; Supervision: MU; Overall: MU; Borrower performance in Lending: MS; Supervision: S ; for Implementation Agency: MS; Overall : MS.

- 30 -

Annex 7. List of Supporting Documents

1. Report No. 19894-IN; Project Appraisal Document. Immunization Strengthening Project.Health, Nutrition and Population Sector Unit, South Asia Region, World Bank; March 30, 2000.

2. India: Immunization Strengthening Project: Summary of Negotiations, April 9, 2000.3. India: Immunization Strengthening Project: Development Credit Agreement, May 19, 1999.4. Report No. 27330-IN; Supplement Credit Document. Immunization Strengthening Project. Human

Development Unit, South Asia Region, World Bank; November 18, 2004.5. India: Immunization Strengthening Project: Agreement Amending Development Credit Agreement,

February 26, 2004.6. Aide Memoire Supervision Mission, December 2005.7. Aide Memoire Supervision Mission, May 20058. Aide Memoire Supervision Mission, October 2004.9. Aide Memoire Supervision Mission, May 2004.10. Aide Memoire Supervision Mission, December 2003.11. Aide Memoire Supervision Mission, September 2003.12. Aide Memoire Supervision Mission, June 2003. 13. Aide Memoire Supervision Mission, December 2002 (Mid-Term Review)..14. Aide Memoire Supervision Mission, June 2002.15. Aide Memoire Supervision Mission, December 2001.16. Aide Memoire Supervision Mission, June 2001.17. Aide Memoire Preparation Mission, December 2000.18. Aide Memoire Preparation Mission, June 2000.19. Assessment of Injection Practices in India (2002-2003); supported by Ministry of Health and

Family Welfare and the World Bank20. Multi Year Strategic Plan 2005 – 2010; Universal Immunization Program. Ministry of Health and

Family Welfare, January 200521. Minutes of the Meeting of the National Technical Advisory Group on Immunization, December 8,

200422. 14th Meeting of the India Advisory Group for Polio Eradication, December 2005.

Other studies:

Additional References:

- 31 -

- 32 -