who ivb 04.05 (booklet1)
TRANSCRIPT
AssessmentMethodology
WHO
World Health Organization
WHO/IVB/04.05
The Common Assessment Tool for Immunization Services
Immunization,Immunization,Vaccines and BiologicalsVaccines and Biologicals
Booklet 1
The Department of Immunization, Vaccines and Biologicals thanks the donors whose unspecified financial support has made the production of this document possible.
This document was jointly produced by the
Expanded Programme on Immunization
of the Department of Immunization, Vaccines and Biologicals
This document is available on the Internet at:www.who.int/vaccines-documents/
Copies may be requested from:World Health Organization
Department of Immunization, Vaccines and Biologicals
CH-1211 Geneva 27, Switzerland
Fax: +41 22 791 4227 Email: [email protected]
© World Health Organization 2002
All rights reserved. Publications of the World Health Organization can be obtained
from Marketing and Dissemination, World Health Organization, 20 Avenue Appia,
1211 Geneva 27, Switzerland (tel: +41 22 791 2476; fax: +41 22 791 4857;
email: [email protected]). Requests for permission to reproduce or translate
WHO publications – whether for sale or for noncommercial distribution –
should be addressed to Publications, at the above address (fax: +41 22 791 4806;
email: [email protected]).
The designations employed and the presentation of the material in this publication
do not imply the expression of any opinion whatsoever on the part of the World
Health Organization concerning the legal status of any country, territory, city or area
or of its authorities, or concerning the delimitation of its frontiers or boundaries.
Dotted lines on maps represent approximate border lines for which there may not
yet be full agreement.
The mention of specific companies or of certain manufacturers’ products does not
imply that they are endorsed or recommended by the World Health Organization in
preference to others of a similar nature that are not mentioned. Errors and omissions
excepted, the names of proprietary products are distinguished by initial capital
letters.
The World Health Organization does not warrant that the information contained in
this publication is complete and correct and shall not be liable for any damages
incurred as a result of its use.
Contents
About this booklet 5
Forms in this booklet 6
1 The Common Assessment Tool (CAT) for Immunization Services 7
What is the CAT? 7
What is the CAT for? 7
Using the methodology for planning 7
Who is the CAT for? 7
Why do we need the new CAT? 8
2 The assessment approach 11
Principles of the methodology 11
Comparing the new methodology with earlier approaches 12
Why take a health system approach? 14
The three elements of the assessment 17
3 The assessment process 19
Overview of the process 19
Step 1 Initiating the assessment 22
Step 2 Preparing for the assessment 30
Step 3 Planning data collection 33
Step 4 Collecting data 38
Step 5 Analysing findings and developing recommendations 43
Step 6 Debriefing 48
Step 7 Planning for implementation and monitoring 49
Annex 1 Key indicators 51
Annex 2 Conducting a SWOT analysis 59
Carrying out the analysis 59
What to look for under each component of the SWOT analysis 60
Annex 3 Data Collection Guides for analysing data and developing recommendations 63
Annex 4 Incorporating the assessment recommendations into the national plan 67
Annex 4a Identifying activities needed to achieve targets and objectives 71
Annex 4b Estimating costs and funding sources 75
Annex 4c Summarizing projected costs of the immunization programme over the five year period 79
Common Assessment Tool for Immunization ServicesBooklet 1 – Assessment Methodology 3
Annex 5 Assessing capacity building 83
Definition of capacity building (CB) 83
Capacity building must maintain and develop existing abilities 84
Capacity building indicators 84
Annex 6 Financial Sustainability Diagnostic Tool (FSDT) 87
Introduction to the FSDT 87
Objectives of the FSDT 87
Methodology of the FSDT 88
Acronyms 89
Annex 7 New Vaccine Discussion Guide 103
Annex 8 Glossary and abbreviations 105
Common Assessment Tool for Immunization Services4 Booklet 1 – Assessment Methodology
About this booklet
This booklet gives you the information you need to carry out an assessment of immunization services.
It includes:
• details of why it’s important to take account of health system functions when carrying out an assessment
• detailed instructions for carrying out an assessment using the CommonAssessment Tool (CAT)
• detailed instructions for analysing the data you collect
• detailed instructions for making recommendations on how immunizationservices can be improved given the opportunities and constraints in thehealth system as a whole.
Common Assessment Tool for Immunization ServicesBooklet 1 – Assessment Methodology 5
Forms in this booklet
The following photocopiable forms can be found in this booklet on the pages shown.
MS Word versions of the forms are also on the CD enclosed in the folder. The MS Word versions can be modified and adapted for the country being visited.
Form title Form reference Page
Summary of SWOT analysis Form 1.1 61
Data Collection GuideImmunization services Form 1.2 64
Data Collection GuideHealth system Form 1.3 65
Summary tableImmunization services activities Form 1.4 72
Summary tableHealth system activities Form 1.5 73
Estimated costs and funding sourcesImmunization services Form 1.6 76
Estimated costs and funding sourcesHealth system Form 1.7 77
Summary of projected costs (five year period)Immunization services Form 1.8 80
Summary of projected costs (five year period)Health system Form 1.9 81
Financial Sustainability Diagnostic Tool (FSDT) Form FSDT 90 – 102
Common Assessment Tool for Immunization Services6 Booklet 1 – Assessment Methodology
The Common Assessment Tool (CAT) for Immunization Services
What is the CAT?
The CAT describes the methodology for assessing immunization services in the wider context of thehealth system.
What is the CAT for?
Using the CAT should increase your knowledge of the current and potential capacity of the health systemand the individual, programme, and system competencies that are necessary to improve performance.Assessments carried out using the CAT can also provide the information needed to develop or updatenational plans and prepare proposals for securing the support of development partners.
Using the methodology for planning
You can use the methodology in this booklet when you are planning an immunization programme to:
• increase the accessibility and use of routine immunization services, especially for people who are not currently reached
• improve the quality of immunization services
• introduce new vaccines and new technologies
• support increased financing for immunization services
• support health system development.
Who is the CAT for?
The guidelines are primarily for national level staff members who are responsible for planning,implementing, and evaluating immunization services. However, development partners collaborating withnational governments can also use them.
You can also adapt them for local health workers and sub-national staff to use to assess day-to-dayoperations and resource management in their areas of responsibility.
Common Assessment Tool for Immunization ServicesBooklet 1 – Assessment Methodology 7
Development partners are public and private, national andmultinational organizations that offer technical, management, andfinancial support to developing countries.
1Chapter
... take full account of health systems
... acknowledge changes in the external environment and their effects on the health system
... assess whether existing services are ready for the introduction of new vaccines and technologies
... meet more challenging disease control goals
Health sector structure, the location of authority and responsibility, and the mix of public and private participation are changing significantly in many countries. The methodology proposed in these guidelines examines the health system and the external environment in which immunization services operate, as well as the services themselves.
Health systems exist in a wider external environment which is subject to many different types of change including epidemiological change, change in the macro-economic situation and policies, and change in the role of the public sector.
These changes often cause changes in many aspects of health systems, including the structure of the health system, the location of authority and responsibility, and the mix of public and private participation in the health system.
a Changes to healthcare mean changes to the way we approach assessments.
It is important to assess existing services to see if they are ready to introduce and sustain new vaccines and technologies.
If we are to eradicate polio and other vaccine-preventable diseases, new strategies, or improvements to current strategies, will be needed. So we need to assess the capacity of the health system to help determine if new disease control strategies can be implemented.
Why do we need the new CAT?
National and international experts have been carrying out assessments of immunization services for manyyears and these assessments have usually provided valuable information that useful recommendationscould be derived from. However, changes in the healthcare environment mean that we now need a newapproach to assessments.
The table below shows the main reasons why these new guidelines are needed.
The new guidelines ...
Common Assessment Tool for Immunization Services8 Booklet 1 – Assessment Methodology
... provide a common assessment tool
... make sure new investments are sustainable and benefit the whole system
... introduce a new approach to fieldwork
Health sector officials, managers of services, development partners, and non-governmental organizations (NGOs) have asked for a common assessment tool to identify where improvements are needed. They want agreement on the improvements needed so that they can plan individual and joint inputs to immunization services.
The assessment needs to consider activities which benefit the whole health system because investments in outcome-orientated programmes usually have consequences for the whole system.
Your solutions and recommendations should be sustainable, and not those which benefit immunization services in the short term while having a negative impact on other services.
a Solutions and recommendations should benefit the whole health system, not just immunizations services
In the past, many assessments of immunization services focused on gathering quantitative data, with people going into the field with set questions to answer, tables to fill in and boxes to tick. When they met workers at the sub-national and service delivery levels, they questioned them to collect the data they needed to fill in the gaps.
However, it is more productive to look for the ‘issues behind the facts’ – to identify causes and trace them back to their source. This means you may have to talk with people and probe for answers and information.
Instead of questionnaires to fill in, we have provided Data Collection Guides to remind you of the key information you need to collect. You can adapt these and add any other information you need to get to the bottom of a particular issue.
Common Assessment Tool for Immunization ServicesBooklet 1 – Assessment Methodology 9
Common Assessment Tool for Immunization Services10 Booklet 1 – Assessment Methodology
The assessment approach
Principles of the methodology
The assessment methodology described in this booklet is based on the five principles below.
1 Focus on performance
Performance is the measure of quality, efficiency, and the impact of immunization services.
2 Focus on the service delivery level and trace problems you find there back to their origin
Initially, you will investigate performance problems at the service delivery level and track those that donot have local causes through the system until you find their origin.
3 Involve people who know what the problems are, and who will take part in implementing solutions
Change can only take place if the people who know what the problems are, and who will make therecommendations, are engaged in identifying and solving the problems.
4 Apply a health system perspective to immunization services
The health system and the external environment strongly influence who is immunized and howimmunizations are provided, so the assessment methodology examines both the health system and itsenvironment. Immunization is just one of the services the health system provides, so you should notconsider it as operating on its own. The strengths and weaknesses of immunization services will haveimplications for the health system, and changes to one will affect the other.
Common Assessment Tool for Immunization ServicesBooklet 1 – Assessment Methodology 11
Immunization services don’t operate on their own – changes to the health system will affect immunization services.
Questions to ask when you are assessing performance
• Is the target population being immunized?
• Are children returning for second and third doses?
• Are health workers using safe practices?
If the answer to any of these questions is ‘no’, what is impeding performance?
2Chapter
5 Use in-depth inquiries and analysis to identify strengths, weaknesses, opportunities, and threats
Use the Discussion Guides in the other booklets as a guide to making inquiries and probing forinformation about internal strengths and weaknesses, and external opportunities and threats. The information people give you should help you find real problems, real causes of the problems, and practical solutions.
Comparing the new methodology with earlier approaches
From a quantitative approach to a qualitative approach
In the past, the focus of assessments of immunization services was on gathering quantitative data withteams using set questions and lists to gather information. However, we now recognise that we need amore qualitative approach which aims to find more than just facts and figures.
New tools for gathering information
Instead of questionnaires, we have prepared Data Collection Guides for gathering information. These givepeople at the sub-national and service delivery levels more opportunity to talk about what they considerthe key issues to be, and what their possible solutions are.
Common Assessment Tool for Immunization Services12 Booklet 1 – Assessment Methodology
Find real problems, real causes of the problems, and real solutions to the problems.
Adapt the Data Collection Guides for each site you assess
• You may have to adapt the Data Collection Guides for each site so thatyou can talk to different people at each level about the same problem.
ImmunizationServices
Thehealthsystem
The externalenvironment
Stewardship
Human resource development
Finance
Service delivery
Immunization service delivery
Disease surveillance
Logistics
Vaccine supply and training
Advocacy and communications
Multi-level problem analysis
The flowchart below gives an example of how a problem identified at service delivery level can be tracedback to its causes at other levels in the health system.
Common Assessment Tool for Immunization ServicesBooklet 1 – Assessment Methodology 13
• Facility does not meet national coverage targets
• Immunizations cancelled due to staff shortages
• Not enough trained staff in place
• Staff leaving because they are not promoted within the system
• Teams are not promoted according to scale
• No training programmes in place
• No effort to attain national standards for staffing levels
• No master training plan in place
A circular process
• The new process is circular and so needs to be repeated every three to five years so that the implementation of recommendations and changes can be monitored and assessed.
A health system approach
The World Health Organization defines a health system as the activities and organizations whose primarypurpose is to promote, restore, and maintain health.
Health systems have four main functions:
• Stewardship
• Human resource development
• Finance
• Service provision.
Functions of a health system in relation to immunization services
Why take a health system approach?
There are three main reasons why you need to take a health system approach when you are assessingimmunization services.
1 Immunization services are part of the health system
Immunization services are becoming increasingly integrated with other health services operating in thehealth system, so they cannot be viewed as operating on their own.
This means that the strengths and weaknesses of different parts of the health system affect theperformance of the immunization programme. Changes to immunization services can affect other services and other aspects of the health system both negatively and positively.
Improvements to immunization services will also be affected by health system functions, so ifimprovements are to be successful, you need to take these into account.
2 Funding can be focused on health system development
As well as accessing new funds for improving immunization services, there will be opportunities toimprove the health system as a whole, and to create synergies between its parts.
Common Assessment Tool for Immunization Services14 Booklet 1 – Assessment Methodology
Immunization services are part of the health system – they don’t operate on their own.
Stewardship
Finance
Human resourcedevelopment
Service delivery
3 Sustainability depends on the health system
One of the main outcomes of an assessment is a set of recommendations for change.
Any change aimed at strengthening immunization services must be sustainable, whether it is based onroutine activities or on the introduction of new vaccines. A health system approach makes it easier toassess how sustainable a recommendation for change is.
Defining the levels in the health system
The health system in each country operates on three main levels:
National level Responsible for policy-making and standard-setting. This is usually thetop level of government in a country. However, in some large countriespolicy-making and other ‘central’ functions can be at state level.
Sub-national level Responsible for providing management support to the service deliverylevel in accordance with policies and guidelines issued by the nationallevel.
Many countries have two sub-national levels:
• regions, states, or provinces
• districts.
Service delivery level Responsible for providing health services including:
• health facilities
• health centres
• health posts, dispensaries
• hospitals
• private practitioners.
Common Assessment Tool for Immunization ServicesBooklet 1 – Assessment Methodology 15
What makes a change sustainable?
For a long-term change to be sustainable, it must be able to:
• secure necessary inputs effectively
• secure necessary support effectively
• provide a continuing stream of activities and outputs that are valued by its stakeholders as long as they are needed.
What makes a health system sustainable?
A health service is sustainable when it is operated by a system with the long-term ability tomobilize and allocate sufficient resources (including manpower, technology, information,and finance) for activities that meet individual or public health needs and demands.
How the three levels in the health system operate
Common Assessment Tool for Immunization Services16 Booklet 1 – Assessment Methodology
National level
Service delivery level
• Sets policies and standards for health systems
Sub-national level
• Manages the implementation of policies and standards for health services
• Provides health services
The three elements of the assessment
This assessment methodology is based on collecting and analysing data about immunization services, thehealth system, and the external environment.
You will need to collect data for each of these at the national, sub-national, and service delivery levels.Before you can make any recommendations, you will need to analyse the data separately by level andthen link the data for each level in a final analysis of service components and system functions.
1 Immunization services
Immunization service delivery The strategies and activities involved in immunization delivery.
Disease surveillance Monitoring disease incidence, laboratory testing, record keeping, andreporting.
Logistics Transport, cold chain, waste disposal, and delivery of vaccines and other equipment to the place where they will be used.
Vaccine supply and quality Forecasting vaccine needs, procuring vaccines, vaccine utilizationmonitoring and safety.
Advocacy and communications Social mobilization, advocacy, immunization education and promotion.
2 Health system
Stewardship Management of the public’s health, and of the individuals andorganizations that provide health services. Stewardship includes:
• policy-making and standard-setting
• planning
• information management
• coordination among public and private health providers
• cooperation with development partners and sectors and withstakeholders outside the health system
• monitoring
• advocacy
• evaluation
• regulation and legislation
• quality assurance
• infrastructure development and logistics
• programme design and evaluation.
Human resource development Human resource development includes:
• employing and assigning staff
• training
• supervision of staff.
Finance Finance includes:
• budgeting
• identifying funding sources
• collecting and allocating revenues
• tracking expenditures.
Service provision Service provision includes:
• delivery of preventative and curative health services.
Common Assessment Tool for Immunization ServicesBooklet 1 – Assessment Methodology 17
• Immunization services • Health system • External environment
3 External environment
Trends Trends include:
• geographic
• political
• economic
• social and technological, for example, macro-economic reforms
• increased pressure for economic self-sufficiency
• decentralization.
Expectations and needs Stakeholders include:of stakeholders • the public
• politicians
• the education sector and other sectors
• development partners
• competitors and collaborators, for example, private practitioners orthe education sector.
Common Assessment Tool for Immunization Services18 Booklet 1 – Assessment Methodology
The assessment process
Overview of the process
There are seven steps in the assessment process. These are illustrated on the chart on the next page andare described in detail in the rest of this chapter.
Common Assessment Tool for Immunization ServicesBooklet 1 – Assessment Methodology 19
3Chapter
1 Initiating the assessmentStep
2 Preparing for the assessmentStep
3 Planning data collectionStep
4 Collecting dataStep
5 Analysing findingsand developing recommendations
Step
6 DebriefingStep
7 Planning for implementationand monitoring
Step
Step
Initiating the assessment
4 to 6 calendar monthsbefore the assessment begins
Task 1.1Decide that an assessment is needed
Task 1.2Appoint an Assessment Secretariat Task 1.3Prepare Terms of Reference
Task 1.4Identify members of the Assessment Team
Task 1.5Identify members of the Planning Team
Task 1.6Prepare an assessment schedule
Task 1.7Plan support for the assessment
Task 1.8Identify funding sources and mechanisms for all assessment activities
Ministry of Health Planning Team All Assessment Team members
Comprehensive Terms of Reference outlining the reasons for the assessment
Team members identified
Planning datacollection
2 to 3 calendar days
Task 3.1Explain assessment objectives
Task 3.2Brief the team on critical issues for country immunization operations and health system
Task 3.3Provide an overview of the assessment methodology
Task 3.4Reach a consensus on whichcritical issues to pursue
Task 3.5Adapt data collection materials
Task 3.6Reach a consensus on sub-national areas for assessment
Task 3.7Review the data collection process
Task 3.8Prepare sub-teams for data collection
Task 3.9Review special issues
Objectives and methodology understood Information from Planning Team reviewed
Preparing forthe assessment
1 to 2 person months
Task 2.1Collect, compile, and review available information on immunization operations, the health system, and theexternal environment
Task 2.2Identify critical issues for immunization and the health system by conducting a SWOT analysis
Task 2.3Compile key materials for Assessment Team members
Task 2.4Prepare to brief the entire Assessment Team
Task 2.5If appropriate, select sub-national areas for visits
Critical issues forimmunization and the health system identified
Gaps in information uncovered
Time
Tasks
Who
Outcomes
1 Step2 Step3Overview of theassessmentprocess
Common Assessment Tool for Immunization Services20 Booklet 1 – Assessment Methodology
National, Sub-Nationaland Service DeliverySub-Teams
All Assessment Team members with technical staff from key stakeholders
All Assessment Team members
Collecting data
5 calendar days
Task 4.1Visit the national, sub-national and service delivery level facilities
Task 4.2Collect data from the national, service delivery and sub-national level facilities
Task 4.3Debrief staff from the national, service delivery and sub-national level facilities
Data collected from all levels
Staff at each level debriefed on findings and preliminary conclusions
Analysing findingsand developingrecommendations
3 calendar days
Task 5.1Analyse information bysub-national area
Task 5.2Analyse information by technical area
Task 5.3Prepare the main conclusionsof the assessment Task 5.4Develop main recommendations of the assessment
Task 5.5Compile information andprepare the first draft of the Assessment Report
Task 5.6Prepare for debriefing
Findings reviewed
Recommendations agreed
Priorities for actions set
Debriefing report prepared
Debriefing
1 to 2 calendar days
All Assessment Team members with technicalstaff from key stakeholders
1 to 2 calendar days todevelop first draft
Task 6.1Debriefing
Task 6.2Complete the final draft of theAssessment Report
Debriefing report delivered
Planning forimplementationand monitoring
Task 7.1Incorporate recommendations into a draft multi-year national plan
Task 7.2Prepare a draft budget and financing plan
Task 7.3Plan howimplementationwill be monitored
Implementation of therecommendations planned
Step4 Step5 Step6 Step7
Common Assessment Tool for Immunization ServicesBooklet 1 – Assessment Methodology 21
What happens during Step 1?
The Ministry of Health (MOH) decides that an assessment is needed and liaises with the InteragencyCoordinating Committee (ICC) to decide the dates for the assessment and who should take part in it.They outline their expectations in a Terms of Reference document.
Tasks for Step 1
Task 1.1 Decide that an assessment is needed
Task 1.2 Appoint an Assessment Secretariat
Task 1.3 Prepare Terms of Reference
Task 1.4 Identify members of the Assessment Team
Task 1.5 Identify members of the Planning Team
Task 1.6 Prepare an assessment schedule
Task 1.7 Plan support for the assessment
Task 1.8 Identify funding sources and mechanisms for all assessment activities
Task 1.1 Decide that an assessment is needed
The Ministry of Health and the national ICC decide if an assessment is needed and when it is needed.
This decision should be made four to six months before the assessment to make sure there is enough timeto prepare for the assessment and to translate the assessment booklets if necessary. The MOH may ask forthe help of a technical partner such as WHO or UNICEF in the preparation phase and in other phases.
Task 1.2 Appoint an Assessment Secretariat
The MOH may decide to appoint an Assessment Secretariat to arrange the logistics for the assessment.
A secretariat is a team of two or three people, often including staff from the immunization programme,WHO or UNICEF, and other key partners. The secretariat supports the Planning Team’s activities before theAssessment Team arrives, and provides support during the preparation, data collection, analysis, andrecommendation steps.
Task 1.3 Prepare Terms of Reference
The reasons for conducting an assessment will be different for each country but they should be explicitlystated in a Terms of Reference document.
This document should tell stakeholders, such as MOH officials and development partners, what thepurpose of the assessment is and how it will be conducted. The Terms of Reference should not be morethan three pages long.
Common Assessment Tool for Immunization Services22 Booklet 1 – Assessment Methodology
Initiating the assessment1Step
Task 1.4 Identify members of the Assessment Team
Who should be on an Assessment Team?
Most Assessment Team members will need knowledge and experience of immunization programmes. Team members can include:
• planning or health system specialists who have extensive knowledge of how health systems work indifferent contexts
• finance staff
although they can contribute to a broader view of the health system, these staff need to be assigned to a teamwith people who have experience of immunization programmes
• maternal and child health staff
although they can contribute to a broader view of the health system, these staff need to be assigned to a teamwith people who have experience of immunization programmes
• national, sub-national and service delivery health managers from within the country and neighbouringcountries
their participation can be very helpful, and can have a long-lasting impact on the improvement of services intheir country
• international experts
they should be assigned to sub-teams with national team members who know the country and its health system,and, if necessary, its language.
What skills do Assessment Team members need?
All members of the Assessment Team need to have critical thinking and communication skills that willhelp them to discuss the situation with people at each level and find the ‘issues behind the facts’.
The leader of the Assessment Team should have experience in:
• immunization programmes
• team leadership
• using this methodology.
How many team members should there be?
The number of team members will vary depending on the purpose of the assessment, the size of thecountry, and the number and location of data collection sites.
Common Assessment Tool for Immunization ServicesBooklet 1 – Assessment Methodology 23
A successful team has a balance of national and international representation so that different knowledge, experience, and perspectives can be considered.
No matter how many people there are on the Assessment Team, you will need to assign some members to the national team and some to the sub-national teams.
Suggested team sizes
National Sub-Team
Number of members At least five.
Comments Individual team members may go to:
• different offices in the Ministry of Health
• partner agencies
• NGOs
• private associations
• other ministries, for example, planning and finance
• health facilities in the capital area.
Booklet 3 includes a Guide for Data Collection which national team membersshould use for this.
Sub-National and Service Delivery Sub-Teams
Number of members At least two for each district or sub-national level.
Comments After initially visiting the sub-national office, the team should divide again forvisits to health facilities. For example, in a country with regions and districts, thewhole sub-team of four would divide into teams of two to visit the selecteddistricts and health facilities within the districts.
Teams should visit public and private sector health facilities and meet withrelevant sub-national level officers, politicians and members of the communityand their leaders.
Is a full-time commitment needed?
Participating in the assessment means you will have to work full-time in preparation, data collection,analysis, making recommendations, and reporting activities. Some Assessment Team members may also beneeded at the end of the assessment to:
• finalise cost estimates for alternative strategies
• incorporate assessment recommendations into the multi-year plan of action for immunization
• complete writing the Assessment Reportsee Step 6.
Task 1.5 Identify members of the Planning Team
The Planning Team prepare for the assessment before the sub-teams go out into the field. Planning Team members need:
• data collection skills
• analysis skills
• knowledge of the immunization programme and health system in the country
• access to key data and other key documents.
As well as this, at least one member of the Planning Team will need to be familiar with health systemassessment procedures and tools.
More information
There is a summary of Planning Team tasks on pages 30-32 of this booklet, and there are Data CollectionGuides and Discussion Guides for the Planning Team in Booklet 2.
Common Assessment Tool for Immunization Services24 Booklet 1 – Assessment Methodology
Task 1.6 Prepare an assessment schedule
An ideal schedule for a two-week assessment in a country that begins its working week on Monday would be like this:
Common Assessment Tool for Immunization ServicesBooklet 1 – Assessment Methodology 25
How long will the Planning Team’s work take?
The Planning Team will take between four and six weeks to prepare for the assessment. They may also continue to take part in the assessment as members of the sub-teams.
Suggested assessment schedule
Day Step Activity Notes
Week 1Wednesday Step 3 planning
Thursday planning
Friday planning
Saturday planning, free day or travel
Sunday travel
Week 2Monday Step 4 data collection
Tuesday data collection
Wednesday data collection
Thursday data collection
Friday data collection
Saturday travel
Sunday Step 5 sub-teams analyse
Week 3Monday analysis and recommendations
Tuesday analysis and recommendations
Wednesday Step 6 debrief
Thursday Step 7 planning for implementation
Friday planning for implementation
Common Assessment Tool for Immunization Services26 Booklet 1 – Assessment Methodology
The length of an assessment depends on its purpose and the size and nature of the geographical areabeing assessed.
If the work of the Planning Team is not included, an assessment using this methodology can becompleted in between two and three weeks, including travel. You will probably spend between 60% and70% of that time on fieldwork which can be split up so that:
• 25% of the time is for the team to plan for data collection – see Step 3, pages 33-37
• 50% of the time is for data collection, including interim analyses and meetings with stakeholders – see Step 4, pages 38-42
• 25% of the time is for analysis, developing recommendations, and briefing government authorities and other major stakeholders – see Steps 5 and 6, pages 43-48.
Schedule the assessment so that data collection starts on the first day of the working week, if possible.
Task 1.7 Plan support for the assessment
It is the job of the Assessment Secretariat to plan support for the assessment. This may include:
• helping to obtain government agreements for the assessment
• making sure information about the assessment is given to development partners
• coordinating invitations for Planning and Assessment Team members
• arranging for salaries, consultant fees, and daily wages for team members
• making hotel reservations
• coordinating transport
• reserving meeting space for Steps 3, 5, 6 and 7
• providing secretarial support and business equipment
• making sure there are adequate supplies of items like stationery
• making sure sub-national and service delivery sites are told about visits.
Common Assessment Tool for Immunization ServicesBooklet 1 – Assessment Methodology 27
How many booklets are needed for the assessment?
Use the table to work out how many copies of each booklet you will need.
Booklet 1 Assessment Methodology
One copy for each member of the Assessment Team =
One copy for each member of the Planning Team =
15 extra copies = 15
Total number of Booklet 1 needed =
Booklet 2 Planning Team Assessment Instructions
One copy for each member of the Assessment Team =
One copy for each member of the Planning Team =
15 extra copies = 15
Total number of Booklet 2 needed =
Booklet 3 National Sub-Team Assessment Instructions
One copy for each member of the Assessment Team =
One copy for each member of the National Sub-Team =
15 extra copies = 15
Total number of Booklet 3 needed =
Booklet 4 Sub-National Sub-Team Assessment Instructions
One copy for each member of the Assessment Team =
One copy for each of the sub-national levels to be visited =
15 extra copies = 15
Total number of Booklet 4 needed =
Booklet 5 Service Delivery Sub-Team Assessment Instructions
One copy for each member of the Assessment Team =
One copy for each service delivery level to be visited =
15 extra copies = 15
Total number of Booklet 5 needed =
As well as this, every sub-national team should have:
• multiple copies of the Data Collection Guides
• one set of Data Collection Guides for each sub-national office and each health facility they visit
• enough copies of the Assessment Summary sheets to allow them to give a written summary to every level they visit.
What other documents will be needed?
Each sub-national team will also need a set of relevant documents. These may include:
• the national plan for the health system
• the national plan for immunization operations
• the health system budget
• the immunization budget
• national immunization policies
• operational guidelines
• reports of reviews and evaluations assessments of immunization operations conducted in the last three years.
What supplies will be needed?
The list below gives you details of the supplies the team will need.
• Notepads
each Assessment Team member will need a notepad to record the information they receive during field visits
• Flip chart paper
if this is expensive or difficult to find in the country, you can use newsprint instead. Newsprint is the paper newspapers are printed on. You can buy it from newspaper publishers for a small fee
• Index cards
approximately 4" x 6", but not smaller. If index cards are not available, you can use A4 notebook paper cut in half instead
• Masking tape or adhesive clay
for example, Blu-Tac
• Paper, pens, and pencils
• Thick felt-tip marker pens
Common Assessment Tool for Immunization Services28 Booklet 1 – Assessment Methodology
Task 1.8 Identify funding sources and mechanisms for all assessment activities
Development partners and internal sources may provide sources of funds.
What should be included in the budget for the assessment?
The budget may include:
• salaries and daily wages for Planning and Assessment Team members
• travel and hotels
• secretarial support
• communication costs
• hire of meeting rooms
• photocopying and printing costs
• supplies and equipment
• translation costs.
Common Assessment Tool for Immunization ServicesBooklet 1 – Assessment Methodology 29
What happens during Step 2?
The Planning Team is responsible for:
• collecting, compiling, reviewing, and collating material on immunization operations, the healthsystem, and the external environment
• carrying out a SWOT analysis to identify critical issues for immunization services and the health systemand their internal strengths and weaknesses and any external opportunities and threats they may face
• uncovering gaps in information
• identifying issues related to data quality.
Tasks for Step 2
Task 2.1 Collect, compile, and review available information on immunization operations, the health system, and the external environment
Task 2.2 Identify critical issues for immunization and the health system by conducting a SWOT analysis
Task 2.3 Compile key materials for the Assessment Team members
Task 2.4 Prepare to brief the entire Assessment Team
Task 2.5 If appropriate, select sub-national areas for visits
Task 2.1 Collect, compile, and review available information on immunization operations,
the health system, and the external environment
The Planning Team do not need to carry out any primary data collection, but do identify possible criticalissues using the data that is available.
You will need to work with the national immunization manager and members of the assessment to:
• review the Planning Team guidelines in Booklet 2
• consult with MOH staff and partners to find the most reliable sources of information
these could be people or documents with relevant information
• prepare a time schedule for Planning Team activities
• review information available at the national level.
More information
There are more details about the information the Planning Team needs to gather in Booklet 2 along withtools to help summarise the information.
Common Assessment Tool for Immunization Services30 Booklet 1 – Assessment Methodology
Preparing for the assessment2Step
Task 2.2 Identify critical issues for immunization and the health system by conducting
a SWOT analysis
You will need to carry out a SWOT analysis to identify the strengths and weaknesses of immunizationservices and the health system, and the external opportunities and threats they face.
More information
There is more information about carrying out a SWOT analysis in Annex 2, pages 59-61.
After you have carried out the SWOT analysis, you will need to prepare a summary of critical issues whichshould include:
• any conclusions based on the SWOT analysis
• the possible implications of these conclusions on immunization services and the health system
• a brief description of any further assessment you think may be needed
• suggestions for where the Assessment Team could gather more information on the administrative level or levels.
Task 2.3 Compile key materials for the Assessment Team members
You should now prepare an assessment library of any documents the Assessment Team will need toconsult during the assessment process.
Common Assessment Tool for Immunization ServicesBooklet 1 – Assessment Methodology 31
Special studies
The government may ask for an in-depth study of a particular aspect of immunization services such as financing, the cold chain, vaccine procurement, laboratory capacity, or health worker communication skills. In these cases, you will need to gather specific data which isbeyond the scope of these guidelines. However, national or international experts should be able to give you advice.
Introducing new vaccines
If a new vaccine or other innovation is being assessed before being introduced, the Assessment Team will need specific information on the requirements and guidance for its introduction. This information is available from WHO or UNICEF.
What types of documents should be included in the assessment library?
Consider including:
• national plans
• budgets
• policy documents
• standards
• guidelines for the health system and immunization services.
There is a list of suggested documents in Booklet 2.
Task 2.4 Prepare to brief the entire Assessment Team
You now need to brief the full Assessment Team on your findings and give them copies of the completedforms and Data Collection Guides.
This briefing should highlight:
• the basic characteristics of the national immunization programme and the health system
• your preliminary conclusions about critical issues for immunization services and the health system, andthe external opportunities and threats that seem to be present
• any areas where there are gaps in the information or where the quality of the data may be in question
• your recommendations about issues that need further analysis.
Task 2.5 If appropriate, select sub-national areas for visits
The MOH and ICC may ask you for advice on which sub-national areas to select for data collection. There is more information about this in Task 3.6, page 36.
Common Assessment Tool for Immunization Services32 Booklet 1 – Assessment Methodology
Keep the presentation short – no more than 30 minutes – so that there’s timefor discussions and questions from the Assessment Team.
What happens during Step 3?
During Step 3 all members of the Assessment Team get together for the first time to:
• learn what the objectives of the assessment are
• review the information on immunization operations and the health system that the Planning Teamgathered during Step 2
• familiarise themselves with the methodology.
Tasks for Step 3
Task 3.1 Explain the assessment objectives
Task 3.2 Brief the Assessment Team on critical issues for country immunization operations and the health system
Task 3.3 Provide an overview of the assessment methodology
Task 3.4 Reach a consensus on which critical issues to pursue
Task 3.5 Adapt data collection materials
Task 3.6 Reach a consensus on sub-national areas for assessment
Task 3.7 Review the data collection process
Task 3.8 Prepare sub-teams for data collection
Task 3.9 Review special issues
How long will Step 3 take?
You will need between two and three days to:
• learn what the objectives of the assessment are
• review the information
• familiarise yourselves with the assessment methodology
• make adaptations to the Data Collection Guides and other tools, if appropriate.
Common Assessment Tool for Immunization ServicesBooklet 1 – Assessment Methodology 33
Planning data collection3Step
Task 3.1 Explain the assessment objectives
The National Immunization Manager and the Team Leader will present the Terms of Reference and clearly explain:
• why the assessment is needed
• what questions it will try to answer
• how they will use the findings and conclusions from the assessment.
Common Assessment Tool for Immunization Services34 Booklet 1 – Assessment Methodology
Suggested planning schedule
Day Activity
Day 1 Welcome and introductions
Explanation of the Terms of Reference and assessment objectivescarried out by Immunization Manager and Team Leader
Briefing on critical issues for country immunization operations and the health systemcarried out by Planning Team
Overview of the assessment methodology, including a briefing on the use of the SWOT techniquecarried out by Planning Team and Team Leader
Day 2 Identification of critical issues for sub-teams to follow upcarried out by technical work group
Adaptation of data collection materialscarried out by technical work group
Selection of first level of sub-national areas, if the Planning Team have not already done thiscarried out by technical work group
Day 3 Study of data collection processcarried out by group work and exercises
Preparation of sub-teams for data collectioncarried out by sub-team
Travel preparationscarried out by sub-teams with support from the Secretariat
Task 3.2 Brief the Assessment Team on critical issues for country immunization
operations and the health system
Members of the Planning Team will brief you on their findings on critical issues for immunizationoperations, the health system, and the external environment.
Task 3.3 Provide an overview of the assessment methodology
You now need to review the assessment methodology with the Planning Team to make sure everyoneunderstands the principles of the assessment, the health system approach, and how to carry out a SWOTanalysis.
Task 3.4 Reach a consensus on which critical issues to pursue
You now need to divide the whole Assessment Team into six technical work groups according to eachmember’s expertise and interests. Each work group should represent the health system and each of the fiveimmunization services components.
Once you have formed work groups, each group should:
• review any information provided by the Planning Team that applies to their technical area
• review the critical issues presented
• assess how adequate and reliable the data is
• identify gaps in the information and possible problem areas.
You should then present your findings to the whole Assessment Team which will decide:
• the strengths the data suggests
• the weaknesses the data suggests
• the issues that seem to be most important
• how these issues can be investigated further.
Common Assessment Tool for Immunization ServicesBooklet 1 – Assessment Methodology 35
The five immunization services components
• Immunization service delivery • Vaccine supply and quality
• Disease surveillance • Advocacy and communications
• Logistics
You may need to form some additional groups depending on the objectivesof the assessment, for example, if you are introducing a new vaccine orintervention, you may want to form an additional group for this.
Task 3.5 Adapt data collection materials
Each technical group now needs to review the Data Collection Guides and Discussion Guides in Booklets 3, 4 and 5 and focus on the sections that relate to their technical area. Each technical groupshould consider the adaptations they need to make to the Data Collection Guides, taking into account the critical issues they agreed to investigate and the characteristics of the country.
After each group has done this, they should present the adaptations they are suggesting to the fullAssessment Team and reach consensus.
Task 3.6 Reach a consensus on sub-national areas for assessment
You will need to select sub-national areas for assessment, if the Planning Team has not already done this.
Consider the objectives of the Terms of Reference and identify immunization services and health systemcriteria to help you select sub-national areas for assessment. Also, think about things like the size of theAssessment Team, the location of sub-national areas and the transport available.
Examples of criteria to use when you are selecting sub-national areas
Area for assessment Criteria
Immunization operations Access to immunization services indicated by BCG or DPT1 coverage
Drop-out rateindicated by BCG-measles and/or DPT1-measles and/or DPT1-DPT3
Quality of disease surveillanceindicated by AFP rate or completeness and timeliness of routine reporting
Progress towards disease controlindicated by incidence of a disease in the area compared to the national average
Health system Extent to which health sector reforms have been introduced
Number of hospitals, health centres, and health posts
Adequacy of staffing – the number and capability of staff
Funding level
Public/private mix
External support
External environment Percentage of urban vs. rural population
Special populations in area
Social-economic factors, for example, literacy, poverty level, and theinfrastructure
Extent of decentralization
Common Assessment Tool for Immunization Services36 Booklet 1 – Assessment Methodology
Adapt the Data Collection Guides taking account of any critical issues.
Task 3.7 Review the data collection process
The Planning Team, Assessment Secretariat and the Team Leader now use group work exercises, role play,and other suitable techniques to make sure all members of the Assessment Team are familiar with datacollection methods.
Task 3.8 Prepare sub-teams for data collection
You will now be allocated to a sub-team depending on your expertise. Sub-teams should meet separatelyto decide how to conduct the assessment. This should include studying the Data Collection Guides andDiscussion Guides, so that you do not have to refer to them much when you are in the field.
You should also review the Assessment Summary Sheets in Booklets 3, 4 and 5 which you will fill in andleave with unit staff after each visit.
Task 3.9 Review special issues
If the Planning Team has identified special issues, the full Assessment Team must decide how they willinvestigate them and make any adaptations to the Data Collection Guides. There is more informationabout this in Annexes 5, 6 and 7, pages 83-103.
Common Assessment Tool for Immunization ServicesBooklet 1 – Assessment Methodology 37
An alternative method
An alternative method for making sure Assessment Team members are familiar with data collection methods is to organize a hands-on practice session in a health facilitythat will not be used as part of the assessment.
This will give you the opportunity to use the guidelines and debrief one another, and to compare techniques and calibrate interpretations of what you have seen and heard.
If you decide to do this, you may need to set aside an extra day for the practice session.
Special issues that may have been identified for the assessment include things like:
• assessing the capacity of the health system to add a new vaccine to the routine schedule
• identifying challenges in the introduction of auto-disable syringes
• capacity building
• financing sustainability.
National level
Sub-national level
Service delivery level
1
1 Visit national level offices
2 Visit sub-national level offices to identify health and service delivery facilities
3 Visit health and service delivery facilities and collect data
4 Debrief health and service delivery staff
5 Visit sub-national level offices to follow up questions from health and service delivery facilities
6 Debrief sub-national level staff
7 Debrief national level staff
2
3
5
4
7
6
What happens during Step 4?
During Step 4 sub-teams visit the national level and the sub-national areas selected during Step 3 tocollect the data and select second level sub-national areas and service delivery level facilities for datacollection.
Timeline showing the sequence of what happens during Step 4
Common Assessment Tool for Immunization Services38 Booklet 1 – Assessment Methodology
Collecting data4Step
More information
• Booklets 3, 4 and 5 include guidance about the data the sub-teams need to collect.
Tasks for Step 4
Task 4.1 Visit the national, sub-national, and service delivery level facilities
Task 4.2 Collect data from the national, service delivery, and sub-national level facilities
Task 4.3 Debrief staff from the national, service delivery, and sub-national level facilities
Before you start Step 4, make sure all members of the team are familiar with SWOT methodology and theDiscussion Guides for the sub-national and service delivery level.
Task 4.1 Visit the national, sub-national, and service delivery level facilities
Visit the sub-national offices to identify health and service delivery facilities
When you arrive at a sub-national office, you will need to meet with relevant staff members to:
• brief them on the assessment objectives and methods
• obtain basic information
• find out if you need to pay a courtesy call to local leaders
• ask for knowledgeable members of staff to join the sub-team.
Staff members you may need to meet with include the Medical Officer and the Health Management Team.
After you have briefed staff about the purpose of the assessment, you will need to ask them to help youidentify second level sub-national areas such as districts or facilities for the assessment. You can do this by applying the criteria in Step 3, pages 33-37, or by asking a few basic questions.
Examples of questions that may help you to select second level sub-national areas
• What was the area’s immunization coverage last year for DPT3, TT2+ and measles?
• Has immunization coverage in the area risen, fallen, or remained stable over the last three years? Are there any places with more noticeable changes?
• Which areas have the highest performance for immunization services and health services? Which are the lowest performing areas?
• Which areas have the most health resources? Which have the least?
• Is private provision of healthcare an important factor in this area? Where is it more prevalent? Where is it less prevalent?
The number of sub-national levels you will need to visit will depend on the structure of the government.If the country has more than one sub-national level – both regions and districts, for example – you willneed to agree which areas to include with staff at the sub-national level.
When you are choosing second level sub-national areas think about the distance you will need to travel tothem and the available transport. Include both high and low performing areas.
Considerations for selecting health facilities
When you are selecting health facilities try to:
• include facilities in both urban and rural locations
• only visit facilities with comparatively high populations in their catchment areas
• include facilities with comparatively high numbers of people who are not currently reached
• select peripheral facilities such as health centres for most of the visits
• include private facilities (both profit and non-profit) if a large percentage of them offer immunization in the area
Common Assessment Tool for Immunization ServicesBooklet 1 – Assessment Methodology 39
• include facilities offering both fixed and outreach sessions
• include underserved areas, for example, those where services are less frequent, where there are nopermanent health workers, or those that are not covered by outreach sessions.
Visit health and service delivery facilities
Before you begin collecting data:
• meet with the director of the facility and any other facility staff who can contribute to the assessment
• explain the objectives of the assessment to staff members.
Task 4.2 Collect data from the national, service delivery, and sub-national level facilities
Collect data from health and service delivery facilities
The ten steps below give you information about the best way to do this:
1 Talk to staff members about the facility’s immunization services and other services.
2 Observe health workers when they are preparing and giving immunizations. If possible, go to outreach sessions with health workers.
3 Talk to clients to find out about their opinions and concerns about the services offered by thefacilities.
4 Talk to people in communities who do not use, or who do not have access to, the services to find outwhat they think the barriers preventing them from using the services are.
5 Meet with relevant community leaders to find out their opinion on immunization services in the area.
6 Use the Discussion Guide in Booklet 3 for ideas on how to initiate discussions, and how to find the‘issues behind the facts’.
7 Use the Data Collection Guides in Booklet 5 to record your findings. You don’t need to fill out everyline in the Data Collection Guides – only the information that will help you to clarify critical issues.
8 Use the SWOT technique described in Annex 2, pages 59-61, to identify the strengths and weaknessesof immunization services and the health system, as well as any external opportunities and threats thatmay have an impact on services. Summarize your conclusions on the forms provided.
9 Summarize what you think the critical issues for immunization services and the health system mightbe. This summary should include:
• any conclusions you have made based on the SWOT analysis
• possible implications these conclusions have for immunization services and the health system
• a brief summary of draft recommendations including suggestions on the administrative level or levels.
10 Meet with other sub-team members to:
• compare interpretations of what you have found and observed
• discuss problems implementing the assessment methodology
• modify planned visits, if appropriate.
Common Assessment Tool for Immunization Services40 Booklet 1 – Assessment Methodology
Before you leave the health facility, give a completed Assessment Summary sheet to the staff. This willserve as a reminder of the discussions and will also be a benchmark for any future assessments.
Repeat tasks 4.2 and 4.3 until you have visited all the health facilities or areas selected for the assessment.Use a separate Data Collection Guide and Assessment Summary for each one.
Collect data from the sub-national office
You now need to return to the sub-national office and collect data for that level. You should also followup on any questions or problems identified at the service delivery level.
Use the Discussion Guides and Data Collection Guides in Booklet 4 to help you with this.
Task 4.3 Debrief staff from the national, service delivery, and sub-national level facilities
Debrief health and service delivery staff
When you have finished collecting data, meet with health facility staff to discuss your findings andpossible solutions for the problems identified. Consider including community leaders or key communityrepresentatives in the debriefing.
Before the meeting, write your findings on the strengths and weaknesses on flip chart paper or newsprintand stick it on the wall. Make sure that you allow local staff members and partners to take part fully inthe meeting. It should take between 45 minutes and an hour.
Common Assessment Tool for Immunization ServicesBooklet 1 – Assessment Methodology 41
Finding the ‘issues behind the facts’
The Discussion Guide in Booklet 3 gives you ideas to help you find ‘the issues behind the facts’.
However, you should not use the Discussion Guide as a questionnaire, and you do not need to ask allthe questions in the discussion points column.
Instead, focus on the critical issues you agreed with the rest of the Assessment Team during Step 1.
You will have more success gathering the information if you try to establish an open dialogue withhealth staff and their stakeholders.
Local problems – local solutions
Local problems, such as immunization session scheduling, can often be solved at the local level.
You should discuss both the problem and the possible solution with health facility staff before leavingthe health facility.
You can then report the outcome of the discussion to people at the next level.
Share findings with those who are most concerned with them, and with those who can help to solve them.
Suggested debriefing meeting agenda
Review and discussion of findings Describe the strengths and weaknesses you have observed inimmunization services and health system functions. Ask staff ifthey agree with the findings, and invite them to make suggestionsor add information that will help you understand the situation.
Consensus on recommendations Ask staff what recommendations they suggest for solving problemsidentified.
Discuss suggested responsibilities for implementingrecommendations. For example, health facility staff can agree tocheck all their vaccines and remove any whose expiry date haspassed, but they probably cannot solve a vaccine supply problemwithout involving other levels.
Conclusions Review what health facility staff have suggested they will dothemselves, and what you have agreed to bring to the attention ofstaff at the next levels.
Debriefing sub-national level staff
Now meet with members of the Health Management Team and other staff members and stakeholders. Use this meeting to:
• report on findings at second level sub-national areas and the service delivery level
• reach agreement on the possible causes of the critical issues in these areas
• identify health system and external opportunities and threats that may affect immunization services
• agree on what second level sub-national and health facility staff can do
• agree on what the Assessment Team needs to discuss at the next level.
Before leaving the sub-national office, give staff a copy of the Assessment Summary for each health facilityvisited, and the summary for their sub-national level.
Why is the debriefing meeting important?
Debriefing sessions at the sub-national level are of great practical benefit as they allow sub-national staffto advise you on your findings, and give you feedback that can be included in the final conclusions andrecommendations.
Common Assessment Tool for Immunization Services42 Booklet 1 – Assessment Methodology
Remember to congratulate health facility staff on any positive findings, and to explore options for solving problems which respect existing lines of authority and responsibility.
What happens during Step 5?
When the data has been collected, all members of the Assessment Team meet again at the national levelto review regional findings, agree on conclusions and recommendations, and to set priorities for actions.
At this stage, technical staff from key stakeholders or partner agencies may join the Assessment Team asthey can contribute to accomplishing the proposed tasks.
Tasks for Step 5
Task 5.1 Analyse information by sub-national area
Task 5.2 Analyse information by technical area
Task 5.3 Prepare the main conclusions of the assessment
Task 5.4 Develop main recommendations of the assessment
Task 5.5 Compile information and prepare the first draft of the Assessment Report
Task 5.6 Prepare for debriefing
Task 5.1 Analyse information by sub-national area
Your sub-team should spend between half a day and a day organizing notes and completing the report onthe sub-national level they visited. You will also need to analyse your findings, prepare conclusions basedon the SWOT analysis, and write preliminary recommendations.
Each sub-team should prepare a 10 to 15 minute presentation to the whole Assessment Team describing:
• their main conclusions about critical issues for immunization services and the health system in their area
• their main conclusions about immunization services and the health system in their area
• their recommendations.
Common Assessment Tool for Immunization ServicesBooklet 1 – Assessment Methodology 43
Analysing findings and developing recommendations
There ...
Strategies and activities
Here ...
... is where we want to be
... is where we are now
5Step
In health planning there are three important questions to ask:
1 Where are we now? ... ‘here’the current status of the health system
2 How do we get from ‘here’ to ‘there’?strategies and activities that have to be organized, and the management support that is needed to organize them
3 Where do we want to be in the future? ... ‘there’the country’s goals and objectives
Task 5.2 Analyse information by technical area
At this stage the assessment changes from analysis by sub-national area to analysis by immunizationservices components and health system functions.
Copies of each sub-team’s reports showing findings, conclusions, and recommendations should bedistributed to the technical work groups formed in Task 3.4.
Each technical group should analyse the data and then summarise their findings on the Data CollectionGuides in Annex 3, pages 63-65. Each group should then make a ten minute presentation to the fullAssessment Team explaining their analysis. You can summarize any conclusions or recommendations on a large index card and stick it on the wall.
Immunization services work groups
Each of the five immunization services work groups should analyse its component and reach agreement on:
• progress and achievements made
• the most important conclusions for each component
• recommendations related to each conclusion
• any possible implications these recommendations may have for the health system.
Health system work group
If the health system technical group has extensive data to analyse, it may be helpful to allocate morepeople to this group and divide them into four smaller groups to analyse each function separately.
The health system work group uses the health system information gathered by each sub-team to analyseinformation on health system functions for each of the three levels. It should reach agreement on:
• progress and achievements made
• the most important conclusions for each function of the health system
• recommendations related to each conclusion
• any possible implications these recommendations may have for immunization services.
Common Assessment Tool for Immunization Services44 Booklet 1 – Assessment Methodology
Service delivery problems may need health system solutions
Some problems may involve other parts of the health system, so it is important to consider that solutions to these problems may be system wide.
Preparing conclusions on special issues
If a special issue was included in the assessment, one of the technical work groups will need to reviewthe proposed indicators to assess this special issue and draw conclusions. It is also important to makesure that the recommendations and activities you propose in Steps 6 and 7, pages 48-50, also addressthese issues.
Examples of special issues include things like the introduction of new vaccines or AD syringes, capacitybuilding, and financing sustainability.
Task 5.3 Prepare the main conclusions of the assessment
Before beginning this task, you should remind yourself of the country’s current immunization targets,goals, and objectives.
You should then identify:
• the progress and achievements made
• your main conclusions about critical issues for immunization services and the health system in the country
• your main conclusions about immunization services and the health system in the country.
There are several methods you could use to achieve this task. But whatever method you choose, it isimportant to make sure that it:
• promotes ownership by all key stakeholders in the assessment
• allows the different sub-teams and technical groups to give their input
• helps identify priority conclusions and recommendations.
Common Assessment Tool for Immunization ServicesBooklet 1 – Assessment Methodology 45
Aims for Task 5.3
The aims of this task are to report on:
• the immunization programme’s strengths and weaknesses
• its efficiency and effectiveness
• its impact on disease burden
• its capacity to adapt to new demands – both those generated by health sector reform and thosethat arise in response to the population’s need for access to vaccines.
Identifying priority conclusions and recommendations using index cards
When all the technical groups have made their presentations, you will have a large number of conclusions and recommendations.
1 Ask each technical group to summarize their report including the main achievements, conclusions,and suggested recommendations on a large index card.
2 Stick all the cards on the wall – there will probably be between 20 and 30.
3 Discuss each conclusion on the cards and consolidate them where possible by:
• moving the cards around so that you group all the cards relating to the same problem together
• comparing the health system and immunization services cards and putting together those that have a direct impact on each other
• deciding which conclusions are most important and then putting those cards together and removing the others.
Task 5.4 Develop main recommendations of the assessment
You should now come to an agreement on the main assessment recommendations. You can achieve thisby following an approach similar to the one you used to select priority conclusions andrecommendations.
You do not need to make recommendations for each of the immunization services components andhealth system functions. For example, it is possible that most of the recommendations for solvingimmunization coverage will involve the health system in the areas of planning, training, and supervisiononly.
How can you tell if the recommendations are relevant?
To check if your recommendations are relevant, answer the questions below:
• Are the strategies consistent with the country’s objectives, policies, strategies, and plans forimmunization services?
• Are they consistent with the country’s health sector development goals and the national health plan?
• Are they consistent with the universal goals of equity, public participation, and accountability?
• Are they feasible?
• Are they affordable?
• Will these recommendations be viewed as collaborative or competitive by non-governmentalorganizations, UN organizations, and other stakeholders in the country?
• Are these recommendations consistent with national and partner’s criteria for obtaining, giving, using,and accounting for financial support?
How can you identify the ways in which immunization services may provide an opportunityfor strengthening health systems?
You may find it helpful to ask:
• What are the critical health systems functions that will allow immunization services to perform better?
• How can these be improved in a sustainable way which will benefit other services and not damagethem?
• What are the critical health systems issues if new technologies or vaccines are introduced?
• How can these be improved in a sustainable way which will benefit other services and not damagethem?
• Are they coherent with health sector development priorities?
Common Assessment Tool for Immunization Services46 Booklet 1 – Assessment Methodology
Some problems may need more study
You may not have been able to find enough information about some issues. These may need more investigation, so you should recommend this in your final report.
Preparing recommendations on special issues
If a special issue was included in the assessment, it is important to make sure that the recommendations and activities you propose address this issue.
Examples of special issues include things like the introduction of new vaccines or AD syringes, capacity building, and financing sustainability.
Task 5.5 Compile information and prepare the first draft of the Assessment Report
Once you have developed your recommendations, you should start to consolidate the information thesub-teams and Planning Team gathered into the first draft of the final Assessment Report. You will use thisfirst draft to debrief high level officials and partners during Step 6.
Suggested table of contents for the final Assessment Report
1 Objectives of the assessment
2 Assessment methodology – a brief description of the process, including the number of sitesvisited and people interviewed
3 The context – a brief discussion of the political, economic and other critical issues in theexternal environment that have an impact on health status, immunization services, and thehealth system
4 Conclusions and recommendations relating to immunization services and the health system
5 Conclusions and recommendations on the feasibility of introducing a new vaccine or anotherinnovation in the country, if applicable
6 Next steps – for example, incorporating the recommendations into the current national plan,preparing cost estimates and a financing plan, further in-depth study, or monitoring aparticular area
Annexes Country fact sheetData Collection Guides including conclusions and recommendationsCopies of each sub-national team’s reports
Task 5.6 Prepare for debriefing
You should now prepare a summary description of the assessment process, and of your conclusions andrecommendations which you will present to the government and stakeholders.
Common Assessment Tool for Immunization ServicesBooklet 1 – Assessment Methodology 47
What happens during Step 6?
The Assessment Team, or a sub-group of its members, gives a debriefing about the assessment and theirmajor recommendations.
Tasks for Step 6
Task 6.1 Debriefing
Task 6.2 Complete the final draft of the Assessment Report
Task 6.1 Debriefing
You now need to give a debriefing based on the conclusions and recommendations compiled in Step 5 to:
• high-level national officials and policy-makersMinister of Health, local government, and finance
• ICC members
• NGOs
• other partners.
You may also need to provide an executive summary of the preliminary report. You can prepare the fullfinal report later.
Task 6.2 Complete the final draft of the Assessment Report
During Task 6.1 you will have presented your draft conclusions and recommendations but it is unlikelythat you will have finished the final draft of the report.
You should now complete this report so that it is available before the Assessment Team separates. If possible, you should submit it to the MOH no more than one month after the assessment.
Common Assessment Tool for Immunization Services48 Booklet 1 – Assessment Methodology
Debriefing6Step
Why is the debriefing meeting important?
The debriefing gives you the opportunity to:
• make sure that proposed changes are consistent with government policies and goals
• obtain high-level political commitment for implementing the recommendations
• promote intra-sector and inter-sector support for proposed changes
• discuss resource and funding needs.
Remember to include in the report any input that senior MOH staff and key stakeholders give you during the debriefing.
What happens during Step 7?
The Assessment Team proposes the implementation phases and outlines the key steps for monitoring.
Tasks for Step 7
Task 7.1 Incorporate recommendations into a draft multi-year national plan
Task 7.2 Prepare a draft budget and financing plan
Task 7.3 Plan how implementation will be monitored
Task 7.1 Incorporate recommendations into a draft multi-year national plan
Selected Assessment Team members, MOH planners, immunization managers, finance staff, and technicalstaff from key partners will need to work together on this task.
Assessment recommendations need to be integrated into the national plan. This will help to make surethat they are supported by decision-makers and implemented by national and sub-national staff. The process described in Annex 4, pages 67-69, should help you with this task.
You may not be able to complete this task in the time available, but it is important to initiate it and takepart in preparing an initial draft. The immunization manager and partners should prepare a clear timelinefor completing the draft.
If you cannot complete it in time, some members of the Assessment Team could stay behind to helpfinalize it.
Common Assessment Tool for Immunization ServicesBooklet 1 – Assessment Methodology 49
Completing the planning process
When you are planning for implementation, you will need to:
• contrast your findings with current goals and targets
• establish specific objectives
• select indicators to evaluate progress
• make plans to determine what priorities need to be tackled, what strategies and activities haveto be implemented, and what management support is needed.
Planning for implementationand monitoring7Step
Task 7.2 Prepare a draft budget and financing plan
Estimates of the costs of current and future activities are important for financing immunization services.Cost analysis may include the total estimated costs, programme specific costs, and recurrent, variable, non-personnel costs. See Annex 4, pages 67-69, for more information.
You will need to clearly identify the source of financing for each component of the immunizationprogramme. You can use information about estimated costs and financing information to identifyfinancing gaps. You should also assess any financial gaps both with, and without, the planned changesand new activities.
You will need to estimate the cost of delivering new antigens, interventions and technologies before youcan budget or find support for their introduction.
Task 7.3 Plan how implementation will be monitored
It is essential to develop a mechanism that will help you to monitor progress and evaluate it in the threeto five years after the assessment.
Everyone who was involved in the assessment and subsequent revisions of the national plan will want to know what the impact of new strategies is on performance. If there is not currently a system formonitoring plan activities, you should encourage the MOH to develop a system that includes:
• appointing someone in the MOH to monitor the implementation of the plan
• preparing progress reports for the ICC, development partners, and other key stakeholders
• following up progress in sub-national officespeople from sub-national offices who were also on the Assessment Team can do this.
Common Assessment Tool for Immunization Services50 Booklet 1 – Assessment Methodology
Evaluating implementation
You can evaluate implementation of the multi-year plan in two ways:
1 By assessing which activities have been implemented compared to what was proposed in the multi-year plan of action or annual work plan.
2 By assessing if the indicators show that the vaccine-preventable disease burden has beenreduced.
Key indicatorsIndicators are essential for analysing the current situation, for expressing specific targets, and for assessingif these targets are being met.
This annex shows the key indicators you will need to consider for the five immunization servicescomponents and the health system.
The tables show indicators for the three levels – national, sub-national, and service delivery.
The Discussion Guides in Booklets 3, 4 and 5 are based on these indicators.
Service delivery: Immunization services key indicator – 1
Common Assessment Tool for Immunization ServicesBooklet 1 – Assessment Methodology 51
National level
Coverage level for each vaccineduring the last three years
National drop-out rate
Completeness and timeliness ofroutine coverage reporting fromthe sub-national levels
Proportion of sub-national unitsby coverage level for each vaccine(e.g. <50%, 50-79%, >80%)
Existence of a national plan forimmunization
Completion of a standardizedimmunization injection safetyassessment
Existence of a policy, plan, andbudget for injection safety
System for detecting,investigating, and reportingadverse events followingimmunization (AEFIs)
Sub-national level
Coverage level for each vaccineduring the last three years
Sub-national area drop-out rate
Completeness and timeliness ofroutine coverage reporting fromthe catchment area
Proportion of catchment areas bycoverage level for each vaccine
Proportion of facilities offeringvaccinations
Supervision system for injectionsafety
Distribution and maintenancesystem for supplies of safeinjections
System for detecting,investigating, and reporting AEFIs
Service delivery level
Coverage level for each vaccineduring the last three years
Health facility drop-out rate
Completeness and timeliness ofroutine coverage reporting to thesub-national level
Effective outreach schedule
Use of one sterile needle and onesterile syringe for each injection
Collection of sharps in puncture-proof containers
Appropriate disposal of injectionequipment
Knowledge of what should bereported as an AEFI
1Annex
Disease surveillance: Immunization services key indicator – 2
Common Assessment Tool for Immunization Services52 Booklet 1 – Assessment Methodology
National level
Vaccine-preventable diseaseincidence
Non-polio AFP rate
% of measles outbreaksinvestigated
% of measles cases withinformation on age andvaccination status
Completeness and timeliness ofroutine reporting
Sub-national level
Vaccine-preventable diseaseincidence
Non-polio AFP rate
% of measles outbreaksinvestigated
% of measles cases withinformation on age andvaccination status
Completeness and timeliness ofroutine reporting
Service delivery level
Vaccine-preventable diseaseincidence
Non-polio AFP rate
% of measles outbreaksinvestigated
% of measles cases withinformation on age andvaccination status
Completeness and timeliness ofroutine reporting
Logistics: Immunization services key indicator – 3
National level
Existence of guidelines on:
• vaccine management
• transport management
• cold chain
• disposal and destruction
Supplies, equipment andconsumables are available wherethey are needed and in theamount needed
Cold-chain equipment operatingand in good repair
Staff monitor status and stock of supplies, equipment, andconsumables when visiting sub-national and service deliveryareas
Sub-national level
Supplies, equipment andconsumables are available wherethey are needed and in theamount needed
Cold-chain equipment operatingand in good repair
Staff monitor status and stock of supplies, equipment, andconsumables when visiting service delivery areas
Staff have an emergency plan forsub-national area
Service delivery level
Good quality supplies,equipment, and consumables areavailable in the amount needed
Cold-chain equipment operatingand in good repair
Staff have an emergency plan forhealth facility
Common Assessment Tool for Immunization ServicesBooklet 1 – Assessment Methodology 53
Vaccine supply and quality: Immunization services key indicator – 4
National level
Supply
• vaccine forecasting
• vaccine utilization and wastagemonitoring
Quality
• an assessment of qualityperformed by a fully functionalregulatory authority (NRA), orother independent assessmentof quality performed
• manufacturer viable or vaccinesprocured from prequalifiedsources
Source and finance
• system for selection of sources
• sustainable financingmechanism
Sub-national level
Supply
• vaccine forecasting
• vaccine utilization and wastagemonitoring
Service delivery level
Supply
• vaccine forecasting
• vaccine utilization and wastagemonitoring
Quality
• vaccine stored and handledproperly
• quality and expiry date ofvaccine checked before use
Advocacy and communications: Immunization services key indicator – 5
National level
Active support of routineimmunizations:
• by political leaders
• by development partners
Active public promotion ofimmunizations
Sub-national level
Active support of routineimmunizations by political leadersand other influential people andgroups in the area
Active public promotion ofimmunizations by units
Service delivery level
Knowledge of public, includingparents, about immunizations
Active attempts to reach theunreached, defaulters, and non-users
Health staff communicateeffectively with clients
Community involvement inplanning and monitoring ofhealth services
Common Assessment Tool for Immunization Services54 Booklet 1 – Assessment Methodology
Stewardship: Health system key indicator – 1
National level
Policy-making and standard-setting
National health policies:
• address the needs of theunserved and under-served
• assure equity of access tohealth services
• address health interventionsthat are priorities for thecountry
As well as governing the publicsector, policies and regulationsgovern the activities of multi- and bilateral agencies, non-governmental organizations, andthe private health sector
Sub-national level
Policy-making and standard-setting
Sub-national policies and plans:
• address the needs of theunserved and under-served
• assure equity of access to health services
• address health interventionsthat are priorities for thecountry and the area
Service delivery level
Policy-making and standard-setting
The health facility’s schedule:
• addresses the needs of theunserved and under-served
• assures equity of access
• includes health interventionsthat are priorities for thecountry and the area
Planning
Existence of a multi-year nationalhealth plan and budget as well asa plan and budget for the currentyear
Adaptability of staff to changeplans and adjust budgets basedon current events such as sectorreforms
Planning
Existence of a plan for each unitin the sub-national level and abudget for the current year
Planning
Existence of a health facility planand a budget for the current year
Information management
Staff at all levels receive timelyinformation on new policies andguidelines
Staff at all levels receive reportson national progress towardsmeeting disease reduction andother health goals
Staff use information to plan andto make adjustments in strategies
National statistics are submittedto WHO regional and globaloffices on time
Information management
Staff receive timely informationon new policies and guidelines
Staff receive reports on nationalprogress towards meeting diseasereduction and other health goals
Staff get feedback on reportssubmitted
Staff use information to plan andto make adjustments in strategies
Information management
Staff receive timely informationon new policies and guidelines
Staff receive reports on nationalprogress towards meeting diseasereduction and other health goals
Staff get feedback on reportssubmitted
Staff use information to plan andto make adjustments in strategies
Common Assessment Tool for Immunization ServicesBooklet 1 – Assessment Methodology 55
Stewardship: Health system key indicator – 1 continued
National level
Coordination amonghealthcare providers
Staff coordinate planning,implementation, and monitoringamong representatives of privateand public healthcare providers
Staff keep all representatives ofhealthcare providers informed ofnew policies, guidelines, andchanges in public healthadministration
Sub-national level
Coordination amonghealthcare providers
Staff coordinate planning,implementation, and monitoringamong representatives of privateand public healthcare providers
Staff keep all representatives ofhealthcare providers informed ofnew policies, guidelines, andchanges in public healthadministration
Service delivery level
Coordination amonghealthcare providers
Staff coordinate planning,implementation, and monitoringamong representatives of privateand public healthcare providers
Staff keep all representatives ofhealthcare providers informed ofnew policies, guidelines, andchanges in public healthadministration
Cooperation
Staff coordinate support providedby partners
Staff keep partners and othersinformed of activities andchanges
Cooperation
Staff coordinate support providedby partners
Staff keep partners and othersinformed of activities andchanges
Cooperation
Staff coordinate support providedby partners (not usually applicableat the service delivery level)
Evaluation
Staff periodically evaluateprogress towards theachievement of national goalsand objectives, and the impact of strategies on the health of the nation
Evaluation
Staff periodically evaluateprogress towards theachievement of the sub-nationallevel goals and objectives
Evaluation
Staff periodically evaluateprogress towards theachievement of the unit’s goalsand objectives
Common Assessment Tool for Immunization Services56 Booklet 1 – Assessment Methodology
Human resource development: Health system key indicator – 2
National level
Staffing
Personnel in all locations meetnational staffing levels andstandards
There are enough staff with theappropriate skills to meet theneeds in all locations
Staff receive adequate salaries ona regular basis
Working conditions are adequate
Staff job performance is regularlyevaluated and feedback isprovided
Sub-national level
Staffing
Personnel in all locations meetnational staffing levels andstandards
There are enough staff with theappropriate skills to meet theneeds in all locations
Staff receive adequate salaries ona regular basis
Working conditions are adequate
Staff job performance is regularlyevaluated and feedback isprovided
Service delivery level
Staffing
Personnel in all locations meetnational staffing levels andstandards
There are enough staff with theappropriate skills to meet theneeds in all locations
Staff receive adequate salaries ona regular basis
Working conditions are adequate
Staff job performance is regularlyevaluated and feedback isprovided
Training
All staff have the knowledge andskills they need to do their jobs
Training
All staff have the knowledge andskills they need to do their jobs
Training
All staff have the knowledge andskills they need to do their jobs
Supervision
Guidelines for supervisors ofhealth workers and otherpersonnel in the system havebeen issued
Supervisors are able to providethe administrative and technicalsupport needed
Supervision
Staff provide supervision and theadministrative and technicalsupport needed at the servicedelivery level
Supervision
Staff receive the supervision andtechnical and administrativesupport they need
Common Assessment Tool for Immunization ServicesBooklet 1 – Assessment Methodology 57
Finance: Health system key indicator – 3
National level
Budgets are consistent with plans
Available funding meets theneeds of the healthcare systemand immunization servicesdescribed in the national planand budget
Funds approved are allocated
Funds allocated are spentaccording to plan
Generally accepted accountingpractices are followed
Sub-national level
Budgets are consistent with plans
Available funding meets theneeds of the healthcare systemand immunization servicesdescribed in the national planand budget
Funds approved are allocated
Funds allocated are spentaccording to plan
Generally accepted accountingpractices are followed
Service delivery level
Staff members are aware of thecost of services, sources offinancing, and the need forefficiency in the use of resources
Common Assessment Tool for Immunization Services58 Booklet 1 – Assessment Methodology
Conducting a SWOT analysisSWOT stands for strengths, weaknesses, opportunities, and threats*. When you are assessingimmunization services, you will need to use a SWOT analysis to study the strengths and weaknesses ofimmunization services and the health system at each level. You will also need to identify opportunitiesand threats from outside immunization services and the health system that may have an effect on them.
Carrying out the analysis
1 Make lists of the internal strengths and weaknesses of immunization services and the health system,and the external opportunities and threats they face. Enter them in the appropriate quadrants of theSummary SWOT analysis.
2 Compare, discuss, and analyse the possible implications of the items in the list.
3 Formulate strategies and recommendations based on the analysis.
Common Assessment Tool for Immunization ServicesBooklet 1 – Assessment Methodology 59
SWeaknessesStrengthsW
OpportunitiesO
ThreatsT
Study the internal strengths and weaknesses of immunization services and the health system at each level.
Identify opportunities and threats from outside immunization services and thehealth system that may havean effect on them.
Discussion and analysis are just as important as listing the strengths,weaknesses, opportunities, and threats.
* John M. Bryson, Strategic Planning for Public and Nonprofit Organizations: A guide to strengthening and sustainingorganizational achievement. Revised edition. 1995. San Francisco, Jossey-Bass Publishers.
2Annex
What to look for under each component of the SWOT analysis
Strengths
Factors that contribute to good system performance and that can be built on to achieve new objectives.
Examples of internal strengths might include an effective cold chain, or public acceptance of the need for immunization.
Weaknesses
Factors in the system that hinder the achievement of objectives.
Examples of weaknesses include frequent occurrence of adverse events following immunization, or staffdissatisfaction with salaries.
Opportunities
Factors outside the health system that you can take advantage of when planning change. Usually fit into one of the following categories:
• political, economic, social, and technological trends
• stakeholders who control resources
• actual or potential collaborators or competitors.
Examples of opportunities include the appointment of a new key decision-maker who supportsimmunizations, or public demand for injection safety.
Threats
Factors outside the health system that have a negative effect on it. Examples include a general decrease ingovernment revenue, or an extensive turnover or transfer of staff.
Common Assessment Tool for Immunization Services60 Booklet 1 – Assessment Methodology
61
Wea
knes
ses
Ther
e is
an
MS
Wo
rd v
ersi
on
of
this
fo
rm o
n t
he
CD
. If
you
nee
d t
o, y
ou
can
ed
it o
r am
end
it f
or
the
cou
ntr
y yo
u a
re a
sses
sin
g.
aLo
ok
for
Form
1.1
Stre
ng
ths
Op
po
rtu
nit
ies
Thre
ats
Cou
ntry
.....
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
.
Dat
e ...
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
.
aPh
otoc
opy
this
for
m f
or e
ach
Sum
mar
y SW
OT
anal
ysis
.
Sum
mar
y o
f SW
OT
anal
ysis
SW
eakn
ess
es
Str
en
gth
sW
Op
po
rtu
nit
ies
OTh
reatsT
Common Assessment Tool for Immunization Services62 Booklet 1 – Assessment Methodology
Data Collection Guides for analysing data and developing recommendations
During Step 5 of the methodology, you will use the forms in this section to summarize your conclusionsand recommendations.
1 Fill in one form for each of the immunization services components.
2 Fill in one form for each of the health system functions.
Common Assessment Tool for Immunization ServicesBooklet 1 – Assessment Methodology 63
3Annex
64
DA
TA C
OLL
ECTI
ON
GU
IDE
Imm
un
izat
ion
ser
vice
s
Ther
e is
an
MS
Wo
rd v
ersi
on
of
this
fo
rm o
n t
he
CD
. If
you
nee
d t
o, y
ou
can
ed
it o
r am
end
it f
or
the
cou
ntr
y yo
u a
re a
sses
sin
g.
aLo
ok
for
Form
1.2
Imm
uniz
atio
n se
rvic
e de
liver
yVa
ccin
e su
pply
and
qua
lity
Cou
ntry
.....
......
......
......
......
......
......
......
......
......
......
......
......
......
.....
Dis
ease
sur
veill
ance
Adv
ocac
y an
d co
mm
unic
atio
nsD
ate
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
..
Logi
stic
sa
Phot
ocop
y th
is f
orm
for
eac
h im
mun
izat
ion
serv
ices
com
pone
nt. T
ick
whi
chev
er a
pplie
s.
Ach
ieve
men
ts
Co
ncl
usi
on
s–
Impl
icat
ions
for
imm
uniz
atio
n se
rvic
esC
on
clu
sio
ns
– Im
plic
atio
ns f
or h
ealth
sys
tem
Rec
om
men
dat
ion
s
65
DA
TA C
OLL
ECTI
ON
GU
IDE
Hea
lth
sys
tem
Ther
e is
an
MS
Wo
rd v
ersi
on
of
this
fo
rm o
n t
he
CD
. If
you
nee
d t
o, y
ou
can
ed
it o
r am
end
it f
or
the
cou
ntr
y yo
u a
re a
sses
sin
g.
aLo
ok
for
Form
1.3
Stew
ards
hip
Serv
ice
prov
isio
nC
ount
ry ..
......
......
......
......
......
......
......
......
......
......
......
......
......
......
..
Hum
an r
esou
rce
deve
lopm
ent
Fina
nce
Dat
e ...
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
.....
aPh
otoc
opy
this
for
m f
or e
ach
heal
th s
yste
m f
unct
ion.
Tic
k w
hich
ever
app
lies.
Ach
ieve
men
ts
Co
ncl
usi
on
s–
Impl
icat
ions
for
imm
uniz
atio
n se
rvic
esPl
ease
incl
ude
info
rmat
ion
for
each
of
the
sub-
func
tions
, e.g
. pol
icy-
mak
ing,
pla
nnin
g et
cC
on
clu
sio
ns
– Im
plic
atio
ns f
or h
ealth
sys
tem
Plea
se in
clud
e in
form
atio
n fo
r ea
ch o
f th
e su
b-fu
nctio
ns,
e.g.
pol
icy-
mak
ing,
pla
nnin
g et
c
Rec
om
men
dat
ion
s
Common Assessment Tool for Immunization Services66 Booklet 1 – Assessment Methodology
Incorporating the assessmentrecommendations into the national planAssessment recommendations need to be integrated into the national plan. This will help to make surethat they are supported by decision-makers and implemented by national and sub-national staff.
Planning is a dynamic process that does not always take place in a purely logical way. However, the stepsbelow give a useful sequence for developing a plan of action.
1 Review the goals for immunization services and the health system in the multi-year plan
• Ask if the plan’s targets and goals are achievable, or if they are too ambitious or too restricted,given the current health system and external environment.
• Ask if the plan’s targets and goals are consistent with the Assessment Team’s recommendations.
• Resolve any difficulties.
• Make the necessary adjustments to the national plan.
2 Write the interim objectives or milestones
• These must be met if goals are to be achieved.
3 Review current and planned strategies
Review the strategies that are currently in the plan and compare them to the Assessment Team’srecommendations. Choose or plan strategies that are:
• relevantthey are most likely to contribute to the achievement of interim objectives or milestones
• sustainablethey are most likely to have a lasting impact on immunization coverage and disease incidence, and contribute to health systems development
• effectivethey are most likely to have a positive impact on the health system
• feasiblegiven existing or projected financial and human resources
• acceptableto decision-makers and the public.
Common Assessment Tool for Immunization ServicesBooklet 1 – Assessment Methodology 67
4Annex
There may be several ways to achieve objectives, but as planning proceeds, any practicalconstraints should become clearer and you may have to revise original strategies.
4 Identify the necessary indicators to monitor progress
Indicators are essential for analysing the current situation, for expressing specific targets, and forassessing if these targets are being met.
Also identify:
• the level each indicator should be monitored at
• the source of the information
• the frequency of collection
• the staff responsible for consolidating and analysing the information.
If you are proposing any new indicators, assess the benefits of collecting additional informationagainst the benefits of improving the quality and completeness of existing indicators.
5 Review the activities described in the national plan
Identify all necessary activities and tasks and determine if they will contribute to achieving the newobjectives and strategies, and how they will combine with them.
You will also need to:
• work out the implications for staffing, facilities, supplies, and transport budgets.
6 Estimate the cost of planned activities
Categorize the activities on the basis of whether they can be implemented:
• at minimal additional cost
• at additional cost where funds are available
• at additional cost where new funds are needed.
Often there will not be enough resources to implement all the recommendations, so decision-makersmay ask planners and finance specialists for a variety of ‘what if’ scenarios so that they can study thecost implications of different strategies and combinations of strategies.
7 Schedule activities and assign responsibility
When you have selected the activities that will reach the objectives most effectively, you will need toschedule them and assign responsibility for their implementation. You will also need to estimate howlong activities will take and then fix realistic starting and completion dates. You may find summarytables like those in Annex 4a helpful.
8 Re-estimate costs and prepare budgets for the next three to five years
Allocate funds that you know are available, or are pledged, and estimate any potential funding gaps.You may find the forms in Annex 4b helpful for this.
Common Assessment Tool for Immunization Services68 Booklet 1 – Assessment Methodology
Common Assessment Tool for Immunization ServicesBooklet 1 – Assessment Methodology 69
Incorporating the assessment recommendations into the national plan
Common Assessment Tool for Immunization Services70 Booklet 1 – Assessment Methodology
Identifying activities needed to achieve targets and objectives
The forms on the following pages will help you to identify the activities that need to be achieved to meet established targets and specific objectives.
1 Fill in one form for each of the immunization services components and one for each of the health system functions.
2 Identify marginal costs of adding new vaccines or interventions.
Common Assessment Tool for Immunization ServicesBooklet 1 – Assessment Methodology 71
4aAnnex
Ob
ject
ive
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
.....
Ind
icat
or
Targ
etA
chie
ved
to
Mai
n
Act
ivit
ies
Tim
elin
eO
ffic
e an
dEs
tim
ated
Sou
rce
of
dat
e/st
atu
sco
nst
rain
tle
vel
cost
fun
ds
resp
on
sib
leC
apita
lU
S$Re
curr
ent
US$
Q1
Q2
Q3
Q4
Year
1Ye
ar 2
Q1
Q2
Q3
Q4
SUM
MA
RY T
AB
LE
Imm
un
izat
ion
ser
vice
sac
tivi
ties
Imm
uniz
atio
n se
rvic
e de
liver
yVa
ccin
e su
pply
and
qua
lity
Cou
ntry
.....
......
......
......
......
......
......
......
......
......
......
......
......
......
.....
Dis
ease
sur
veill
ance
Adv
ocac
y an
d co
mm
unic
atio
nsD
ate
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
..
Logi
stic
sa
Phot
ocop
y th
is f
orm
for
eac
h im
mun
izat
ion
serv
ices
com
pone
nt. T
ick
whi
chev
er a
pplie
s.
Ther
e is
an
MS
Wo
rd v
ersi
on
of
this
fo
rm o
n t
he
CD
. If
you
nee
d t
o, y
ou
can
ed
it o
r am
end
it f
or
the
cou
ntr
y yo
u a
re a
sses
sin
g.
aLo
ok
for
Form
1.4
Ob
ject
ive
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
.....
Ind
icat
or
Targ
etA
chie
ved
to
Mai
n
Act
ivit
ies
Tim
elin
eO
ffic
e an
dEs
tim
ated
Sou
rce
of
dat
e/st
atu
sco
nst
rain
tle
vel
cost
fun
ds
resp
on
sib
leC
apita
lU
S$Re
curr
ent
US$
Q1
Q2
Q3
Q4
Year
1Ye
ar 2
Q1
Q2
Q3
Q4
SUM
MA
RY T
AB
LE
Hea
lth
sys
tem
acti
viti
es
Stew
ards
hip
Serv
ice
prov
isio
nC
ount
ry ..
......
......
......
......
......
......
......
......
......
......
......
......
......
......
..
Hum
an r
esou
rce
deve
lopm
ent
Fina
nce
Dat
e ...
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
.....
aPh
otoc
opy
this
for
m f
or e
ach
heal
th s
yste
m f
unct
ion.
Tic
k w
hich
ever
app
lies.
Ther
e is
an
MS
Wo
rd v
ersi
on
of
this
fo
rm o
n t
he
CD
. If
you
nee
d t
o, y
ou
can
ed
it o
r am
end
it f
or
the
cou
ntr
y yo
u a
re a
sses
sin
g.
aLo
ok
for
Form
1.5
Common Assessment Tool for Immunization Services74 Booklet 1 – Assessment Methodology
Estimating costs and funding sources
The forms on the following pages will help you to identify the estimated costs and funding sources that are needed to achieve established targets and specific objectives.
1 Fill in one form for each of the immunization services components and one for each of the health system functions.
2 Identify marginal costs of adding new vaccines or interventions.
Common Assessment Tool for Immunization ServicesBooklet 1 – Assessment Methodology 75
4bAnnex
Ob
ject
ive
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
.....
Typ
e o
f co
stD
om
esti
c so
urc
eEx
tern
al s
ou
rce
Tota
l fu
nd
ing
Sho
rtfa
llav
aila
ble
Cap
ital
cost
s
• Bu
ildin
g sp
ace
• Ve
hicl
es
• Eq
uipm
ent
• O
ther
Cap
ital
cost
s su
bto
tal
Rec
urr
ent
cost
s
• Pe
rson
nel
• Va
ccin
es
• Su
pplie
s
• Tr
ansp
ort
• Su
rvei
llanc
e an
d m
onito
ring
• Tr
aini
ng
• C
omm
unic
atio
n, in
clud
ing
heal
thpr
omot
ion,
etc
.
• M
aint
enan
ce a
nd o
verh
eads
• O
ther
Rec
urr
ent
cost
s su
bto
tal
Tota
l
Perc
enta
ge
of
tota
l
Cen
tral
go
vern
men
tLo
cal
gove
rnm
ent
Hea
lth
insu
ranc
eO
ther
Gra
nts
Loan
sO
ther
ESTI
MA
TED
CO
STS
AN
DFU
ND
ING
SO
UR
CES
Imm
un
izat
ion
ser
vice
s
Imm
uniz
atio
n se
rvic
e de
liver
yVa
ccin
e su
pply
and
qua
lity
Cou
ntry
.....
......
......
......
......
......
......
......
......
......
......
......
......
......
.....
Dis
ease
sur
veill
ance
Adv
ocac
y an
d co
mm
unic
atio
nsD
ate
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
..
Logi
stic
sa
Phot
ocop
y th
is f
orm
for
eac
h im
mun
izat
ion
serv
ices
com
pone
nt. T
ick
whi
chev
er a
pplie
s.
Ther
e is
an
MS
Wo
rd v
ersi
on
of
this
fo
rm o
n t
he
CD
. If
you
nee
d t
o, y
ou
can
ed
it o
r am
end
it f
or
the
cou
ntr
y yo
u a
re a
sses
sin
g.
aLo
ok
for
Form
1.6
Ob
ject
ive
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
.....
Typ
e o
f co
stD
om
esti
c so
urc
eEx
tern
al s
ou
rce
Tota
l fu
nd
ing
Sho
rtfa
llav
aila
ble
Cap
ital
cost
s
• Bu
ildin
g sp
ace
• Ve
hicl
es
• Eq
uipm
ent
• O
ther
Cap
ital
cost
s su
bto
tal
Rec
urr
ent
cost
s
• Pe
rson
nel
• Va
ccin
es
• Su
pplie
s
• Tr
ansp
ort
• Su
rvei
llanc
e an
d m
onito
ring
• Tr
aini
ng
• C
omm
unic
atio
n, in
clud
ing
heal
thpr
omot
ion,
etc
.
• M
aint
enan
ce a
nd o
verh
eads
• O
ther
Rec
urr
ent
cost
s su
bto
tal
Tota
l
Perc
enta
ge
of
tota
l
Cen
tral
go
vern
men
tLo
cal
gove
rnm
ent
Hea
lth
insu
ranc
eO
ther
Gra
nts
Loan
sO
ther
ESTI
MA
TED
CO
STS
AN
DFU
ND
ING
SO
UR
CES
Hea
lth
sys
tem
Stew
ards
hip
Serv
ice
prov
isio
nC
ount
ry ..
......
......
......
......
......
......
......
......
......
......
......
......
......
......
..
Hum
an r
esou
rce
deve
lopm
ent
Fina
nce
Dat
e ...
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
.....
aPh
otoc
opy
this
for
m f
or e
ach
heal
th s
yste
m f
unct
ion.
Tic
k w
hich
ever
app
lies.
Ther
e is
an
MS
Wo
rd v
ersi
on
of
this
fo
rm o
n t
he
CD
. If
you
nee
d t
o, y
ou
can
ed
it o
r am
end
it f
or
the
cou
ntr
y yo
u a
re a
sses
sin
g.
aLo
ok
for
Form
1.7
Common Assessment Tool for Immunization Services78 Booklet 1 – Assessment Methodology
Summarizing projected costs of the immunization programme over the five year period
The forms on the following pages will help you to summarise the projected costs of achieving establishedtargets and specific objectives.
1 Fill in one form for each of the immunization services components.
2 Fill in one form for each of the health system functions.
Common Assessment Tool for Immunization ServicesBooklet 1 – Assessment Methodology 79
4cAnnex
SUM
MA
RY O
F PR
OJE
CTE
D
CO
STS
(FIV
E Y
EAR
PER
IOD
)
Imm
un
izat
ion
ser
vice
s
Imm
uniz
atio
n se
rvic
e de
liver
yVa
ccin
e su
pply
and
qua
lity
Cou
ntry
.....
......
......
......
......
......
......
......
......
......
......
......
......
......
.....
Dis
ease
sur
veill
ance
Adv
ocac
y an
d co
mm
unic
atio
nsD
ate
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
..
Logi
stic
sa
Phot
ocop
y th
is f
orm
for
eac
h im
mun
izat
ion
serv
ices
com
pone
nt. T
ick
whi
chev
er a
pplie
s.
Ther
e is
an
MS
Wo
rd v
ersi
on
of
this
fo
rm o
n t
he
CD
. If
you
nee
d t
o, y
ou
can
ed
it o
r am
end
it f
or
the
cou
ntr
y yo
u a
re a
sses
sin
g.
aLo
ok
for
Form
1.8
Ob
ject
ive
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
.....
Co
mp
on
ent
and
Esti
mat
edD
om
esti
c so
urc
eEx
tern
al s
ou
rce
Tota
l fu
nd
ing
Sho
rtfa
llty
pe
of
cost
cost
fo
r av
aila
ble
five
yea
r p
erio
d
Cap
ital c
ost
Recu
rren
t co
st
Cap
ital c
ost
Recu
rren
t co
st
Cap
ital c
ost
Recu
rren
t co
st
Cap
ital c
ost
Recu
rren
t co
st
Cap
ital c
ost
Recu
rren
t co
st
Cap
ital c
ost
Recu
rren
t co
st
Cen
tral
go
vern
men
tLo
cal
gove
rnm
ent
Hea
lth
insu
ranc
eO
ther
Gra
nts
Loan
sO
ther
Serv
ice
del
iver
y
Dis
ease
su
rvei
llan
ce
Log
isti
cs
Vac
cin
e su
pp
ly a
nd
qu
alit
y
Ad
voca
cy a
nd
co
mm
un
icat
ion
s
Hea
lth
sys
tem
m
gm
t fu
nct
ion
s
Cap
ital
co
sts
sub
tota
l
Rec
urr
ent
cost
s su
bto
tal
Gra
nd
to
tal
SUM
MA
RY O
F PR
OJE
CTE
D
CO
STS
(FIV
E Y
EAR
PER
IOD
)
Hea
lth
sys
tem
Stew
ards
hip
Serv
ice
prov
isio
nC
ount
ry ..
......
......
......
......
......
......
......
......
......
......
......
......
......
......
..
Hum
an r
esou
rce
deve
lopm
ent
Fina
nce
Dat
e ...
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
.....
aPh
otoc
opy
this
for
m f
or e
ach
heal
th s
yste
m f
unct
ion.
Tic
k w
hich
ever
app
lies.
Ther
e is
an
MS
Wo
rd v
ersi
on
of
this
fo
rm o
n t
he
CD
. If
you
nee
d t
o, y
ou
can
ed
it o
r am
end
it f
or
the
cou
ntr
y yo
u a
re a
sses
sin
g.
aLo
ok
for
Form
1.9
Ob
ject
ive
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
......
.....
Co
mp
on
ent
and
Esti
mat
edD
om
esti
c so
urc
eEx
tern
al s
ou
rce
Tota
l fu
nd
ing
Sho
rtfa
llty
pe
of
cost
cost
fo
r av
aila
ble
five
yea
r p
erio
d
Cap
ital c
ost
Recu
rren
t co
st
Cap
ital c
ost
Recu
rren
t co
st
Cap
ital c
ost
Recu
rren
t co
st
Cap
ital c
ost
Recu
rren
t co
st
Cap
ital c
ost
Recu
rren
t co
st
Cap
ital c
ost
Recu
rren
t co
st
Cen
tral
go
vern
men
tLo
cal
gove
rnm
ent
Hea
lth
insu
ranc
eO
ther
Gra
nts
Loan
sO
ther
Serv
ice
del
iver
y
Dis
ease
su
rvei
llan
ce
Log
isti
cs
Vac
cin
e su
pp
ly a
nd
qu
alit
y
Ad
voca
cy a
nd
co
mm
un
icat
ion
s
Hea
lth
sys
tem
m
gm
t fu
nct
ion
s
Cap
ital
co
sts
sub
tota
l
Rec
urr
ent
cost
s su
bto
tal
Gra
nd
to
tal
Common Assessment Tool for Immunization Services82 Booklet 1 – Assessment Methodology
Assessing capacity building
Definition of capacity building (CB)
In the context of strengthening immunization services, capacity building means significantly enhancingthe ability of the national immunization programme to:
• increase and maintain access to immunization services
• decrease the burden of vaccine-preventable diseases
• expand the use of safe and cost-effective vaccines.
In capacity building, special attention is given to:
• safety and quality
• consistency with national health sector goals
• identifying funding shortfalls
• progressing towards self-reliance in identifying and generating resources
• managing knowledge so that work experience can be retained and expertise disseminated throughoutthe organization.
Common Assessment Tool for Immunization ServicesBooklet 1 – Assessment Methodology 83
5Annex
How does this definition fit in with GAVI milestones?
1 By 2010 or sooner, all countries will have routine immunization coverage at 90% nationally with at least80% coverage in every district.
2 By 2002, 80% of all countries with adequate delivery systems will have introduced hepatitis B vaccine.By 2007, all countries.
3 By 2005, 50% of the poorest countries with high disease burdens and adequate delivery systems will haveintroduced Hib vaccine.
4 By 2005, the world will be certified polio-free.
5 By 2005, the vaccine efficacy and burden of disease will be known for all regions for rotavirus andpneumococcal vaccine, and mechanisms identified to make the vaccines available to the poorest countries.
Capacity building must maintain and develop existing abilities
As well as developing new abilities, the capacity building process must maintain and develop existingabilities. It must be based on a methodology which can:
• assess the current situation
• define future goals
• plot a way of reaching goals.
Any capacity building strategy must also address three levels:
• the individual
• the institutions
• the overall health system.
Capacity building indicators
Countries should identify their own indicators to measure progress in addressing each of the healthsystem functions. The GAVI Inter-Taskforce Sub-group on Capacity Building has also suggested indicatorsfor each function which are consistent with global targets.
Capacity building indicators help countries to:
• carry out needs assessments
• develop plans that address gaps
• monitor implementation.
Many capacity building indicators overlap with the critical indicators included in the CAT. The information below gives details of capacity building indicators and tells you if they are also includedin the CAT.
Finance – proposed CB indicators
CB indicators
The most recent indicators are summarized in Annex 6, section 6
CAT indicator that may overlap with the CB indicators
See Annex 6, section 6
Management – proposed CB indicators
CB indicators
Timeliness and completeness of reports, reflecting institutional capacity to monitor its performance
CAT indicator that may overlap with the CB indicators
Completeness and timeliness of routine coverage reporting from sub-national levelssee Immunization services – immunization service delivery
Completeness and timeliness of routine surveillance reportssee Immunization services – disease surveillance
Staff at all levels receive timely information on new policies and guidelinessee Health system – stewardship
National statistics are submitted to WHO regional and global offices on timesee Health system – stewardship
Common Assessment Tool for Immunization Services84 Booklet 1 – Assessment Methodology
CB indicators
Provision of adequate immunization supplies as reflected by lack of stock-outs
CAT indicator that may overlap with the CB indicators
Supplies, equipment, consumables are available where they are needed and in the amount neededsee Immunization services – logistics
Staff monitor status and stock of supplies, equipment, and consumables when visiting sub-national,service delivery areassee Immunization services – logistics
Vaccine forecastingsee Immunization services – vaccine supply and quality
Vaccine utilization and wastage monitoringsee Immunization services – vaccine supply and quality
CB indicators
Number of days spent by national management team members in districts – proxy for assessingsupervision of sub-national level activities and district micro-planning
CAT indicator that may overlap with the CB indicators
Guidelines for supervisors of health workers and other personnel in the system have been issuedsee Health system – human resource development
Supervisors are able to provide the administrative and technical support neededsee Health system – human resource development
Strengthening human and institutional resources – proposed CB indicators
CB indicators
Percentage of total staff at each level that have received training in past two years, including the specificsof those training activities
CAT indicator that may overlap with the CB indicators
All staff have the knowledge and skills they need to do their jobssee Health system – human resource development
CB indicators
National training plan developed, funded, implemented
CAT indicator that may overlap with the CB indicators
Existence of a multi-year national health plan and budget for the current yearsee Health system – stewardship
CB indicators
Adequacy of staffing (country to define specific levels), as defined by the rate of staff turnover at allrelevant levels
CAT indicator that may overlap with the CB indicators
Personnel in all locations meet national staffing needs and standardssee Health system – human resource development
There are enough staff with the appropriate skills to meet needs in all locationssee Health system – human resource development
Common Assessment Tool for Immunization ServicesBooklet 1 – Assessment Methodology 85
Immunization services operations/service provision – proposed CB indicators
CB indicators
Suggest using indicators included in the CAT for the five components of service delivery
CAT indicator that may overlap with the CB indicators
All indicators overlapsee Immunization services
Common Assessment Tool for Immunization Services86 Booklet 1 – Assessment Methodology
Financial Sustainability Diagnostic Tool (FSDT)
This annex is adapted from the "Guidelines for preparing a National Immunization Program FinancialSustainability Plan" prepared by the GAVI Financing Task Force (FTF), and includes information on theFinancial Sustainability Diagnostic Tool (FSDT), developed by the Access to Technologies Team (ATT).
Introduction to the FSDT
Countries receiving funding from GAVI through the Vaccine Fund are required to prepare financialsustainability plans at the end of the second year. The Financial Sustainability Diagnostic Tool (FSDT) isdesigned to facilitate the process of preparing Financial Sustainability Plans.
Countries could use the FSDT to diagnose their current situation in the areas defined in the FinancialSustainability Plan guidelines, and thus guide the development and implementation of the FinancialSustainability Plan.
The FSDT is a qualitative tool, based on possible key benchmarks for financial sustainability ofimmunization programmes. From the results of the assessment, countries should be able to: identify thestrengths of the current financing of immunization programmes, highlight areas for further improvement,and identify what type of capacity building is required to prepare and implement viable FinancialSustainability Plans.
Common Assessment Tool for Immunization ServicesBooklet 1 – Assessment Methodology 87
6Annex
Objectives of the FSDT
This tool is intended to:
• assess the current level of financial sustainability of countries to monitor progress toward financialsustainability and better management of national immunization programmes within health sectordevelopment, using simple qualitative indicators
• assess the country capacity to formulate and implement a Financial Sustainability Plan using theframework provided by the guidelines for Financial Sustainability Plan preparation developed by the GAVI FTF
• help identify the strengths of countries current immunization programmes and highlight areas for further improvement in achieving financial sustainability, prior to submission of FinancialSustainability Plans
• provide guidance and information on technical assistance and training needed to support anddevelop current immunization financing systems, structures, staffing and strategies.
Methodology of the FSDT
The FSDT is linked to the Financial Sustainability Plan guidelines and the overall objectives of the keycomponents. The FSDT appears as an annex to the Financial Sustainability Plan guidelines.
The tool should be used as best suited to country needs. For many countries, this might mean a jointlyconducted diagnosis at the outset of the financial sustainability plan development process with externalexperts, as well as the internal core expert team. Other countries might opt to use the tool as a self-assessment tool.
The FSDT is formulated as a questionnaire, including guiding criteria and possible sources of information.All questions lead in to straight “Yes” or “No” answers. Following the completion of the FSDT, conclusionsand recommendations should be drawn, summarizing the country’s strengths, the areas susceptible toimprovement and indicating possible needs for capacity building and/or technical assistance whererequired.
The FSDT would be completed through a series of interviews with all relevant stakeholders both atnational, sub-national and operational level. These stakeholders include, amongst others, Ministry ofHealth (national immunization staff, planning department within the Ministry of Health), Ministry ofFinance (departments responsible for planning, budgeting, expense control, HIPC/PRSP), Ministry ofPlanning, ICC members and other partners. The answers should be validated where possible. Sources ofvalidation could be the national budget, minutes of meetings of the ICC, public expenditure reviews,immunization costing and financing studies.
A maximum of one week should be estimated to complete the FSDT, including the drawing up ofconclusions, recommendations and a work plan for developing the Financial Sustainability Plan.
Common Assessment Tool for Immunization Services88 Booklet 1 – Assessment Methodology
Acronyms
EPI Expanded Programme on Immunization
FIC Fully Immunized Child
GDP Gross Domestic Product
Hep B Hepatitis B Vaccine
Hib Haemophiylus Influenzae Type B Vaccine
HIPC II/PRSP Heavily Indebted Poor Countries II / Poverty Reduction Strategy Paper
HIV/AIDS Human Immunodeficiency Virus / Auto-Immune Deficiency Syndrome
ICC Interagency Coordinating Committee
MoF Ministry of Finance
MoH Ministry of Health
NIDs National Immunization Days
NIP National Immunization Programme
SWAP Sector Wide Approach
TB Tuberculosis
VVM Vaccine Vial Monitor
Common Assessment Tool for Immunization ServicesBooklet 1 – Assessment Methodology 89
90
Ther
e is
an
MS
Wo
rd v
ersi
on
of
this
fo
rm o
n t
he
CD
. If
you
nee
d t
o, y
ou
can
ed
it o
r am
end
it f
or
the
cou
ntr
y yo
u a
re a
sses
sin
g.
aLo
ok
for
Form
FSD
T
1.00
Co
un
try
and
hea
lth
sys
tem
co
nte
xt
Wh
at a
re t
he
curr
ent
con
dit
ion
s an
d p
lan
ned
ch
ang
es in
th
e p
ub
lic s
ecto
r o
rgan
izat
ion
an
d f
inan
cin
g t
hat
are
like
ly t
oh
ave
an im
po
rtan
t ef
fect
on
th
e th
e fi
nan
cial
su
stai
nab
ility
of
the
imm
un
izat
ion
pro
gra
mm
e?
Co
ntr
ibu
tors
to
th
is s
ecti
on
sh
ou
ld in
clu
de:
Min
istr
y of
Fin
ance
, M
inis
try
of H
ealth
and
Lea
ding
Don
or A
genc
ies
Qu
esti
on
sA
nsw
ers
Gu
idin
g c
rite
ria
Rem
arks
1.01
Is im
mun
izat
ion
cove
rage
a m
ajor
dev
elop
men
t in
dica
tor
with
in t
he c
urre
nt h
ealth
sys
tem
?■■
Yes
■■N
o
1.02
Has
a m
ulti-
year
pla
n be
en d
evel
oped
for
the
hea
lthse
ctor
?■■
Yes
■■N
o
1.03
Doe
s th
e go
vern
men
t bu
dget
con
tain
vac
cine
and
/or
othe
r im
mun
izat
ion
line
item
s?■■
Yes
■■N
o
1.04
Hav
e po
tent
ial h
ealth
sec
tor
prog
ram
me
prio
ritie
s th
atco
uld
impa
ct f
unds
for
the
NIP
bee
n fo
rese
en in
the
budg
et a
lloca
tion
proc
ess?
■■Ye
s■■
No
– H
IV/A
IDS
– TB
– M
alar
ia–
Cur
ativ
e he
alth
care
– O
ther
s
1.05
Has
the
pot
entia
l im
pact
of
publ
ic/h
ealth
sys
tem
orga
niza
tion
on t
he f
undi
ng o
f th
e N
IP b
een
take
n in
toac
coun
t by
the
nat
iona
l im
mun
izat
ion
man
ager
inbu
dget
ing
and
fund
ing
of t
he p
rogr
amm
e?
■■Ye
s■■
No
– D
ecen
tral
izat
ion
– In
tegr
atio
n–
Con
trac
ting
with
the
priv
ate
sect
or–
Oth
ers
Fin
anci
al S
ust
ain
abili
ty
Dia
gn
ost
ic T
oo
l (FS
DT)
Cou
ntry
. . .
. . .
. . .
. . .
. . .
. . .
. . .
. . .
. . .
Dat
e of
ass
essm
ent
. . .
. . .
. . .
. . .
. . .
. . .
. . .
. . .
. . .
Ass
essm
ent
perf
orm
ed b
y. .
. . .
. . .
. . .
. . .
. . .
. . .
. . .
. . .
.
Poin
ts o
f co
ntac
t. .
. . .
. . .
. . .
. . .
. . .
. . .
. . .
. . .
.
91
Ther
e is
an
MS
Wo
rd v
ersi
on
of
this
fo
rm o
n t
he
CD
. If
you
nee
d t
o, y
ou
can
ed
it o
r am
end
it f
or
the
cou
ntr
y yo
u a
re a
sses
sin
g.
aLo
ok
for
Form
FSD
T
Qu
esti
on
sA
nsw
ers
Gu
idin
g c
rite
ria
Rem
arks
1.06
Are
the
re a
ny p
lann
ed p
ublic
/hea
lth r
efor
ms
unde
r w
ayin
how
prio
ritie
s ar
e se
t w
ithin
nat
iona
l or
finan
cing
part
ner
budg
ets
that
wou
ld h
ave
impo
rtan
t im
plic
atio
nsfo
r th
e im
mun
izat
ion
prog
ram
me?
■■Ye
s■■
No
– SW
Aps
– Pu
blic
/priv
ate
part
ners
hips
– O
ther
s
1.07
Are
the
re a
ny p
lann
ed c
hang
es in
the
fin
anci
ng s
trat
egy
or f
inan
cing
mec
hani
sms
to f
und
the
heal
th s
yste
m t
hat
are
likel
y to
hav
e a
posi
tive
impa
ct o
n th
e fu
ndin
g of
the
NIP
?
■■Ye
s■■
No
– So
cial
insu
ranc
e–
Priv
ate
insu
ranc
e–
Com
mun
ity f
inan
cing
– Pa
yrol
l tax
es–
Oth
ers
1.08
Are
the
re a
ny m
acro
econ
omic
issu
es f
acin
g th
e co
untr
yth
at a
re e
xpec
ted
to p
ositi
vely
aff
ect
the
futu
re f
undi
ngof
the
NIP
?
■■Ye
s■■
No
– Ec
onom
ic g
row
th–
Deb
t re
lief
– In
flatio
n–
Oth
er
1.09
Is t
here
an
adeq
uate
rem
uner
atio
n sy
stem
in p
lace
to
ensu
re m
otiv
atio
n of
hea
lth s
taff
so
that
sal
ary,
per
diem
,an
d tr
avel
allo
wan
ces
are
paid
whe
n du
e*?
■■Ye
s■■
No
* W
hen
due:
dep
endi
ng o
nco
ntra
ctua
l agr
eem
ents
e.
g. w
eekl
y, m
onth
ly
1.10
Hav
e th
e fu
ll-tim
e na
tiona
l im
mun
izat
ion
man
ager
and
full-
time
natio
nal c
old
stor
e m
anag
er b
een
in p
lace
for
at le
ast
a ye
ar?
■■Ye
s■■
No
92
Ther
e is
an
MS
Wo
rd v
ersi
on
of
this
fo
rm o
n t
he
CD
. If
you
nee
d t
o, y
ou
can
ed
it o
r am
end
it f
or
the
cou
ntr
y yo
u a
re a
sses
sin
g.
aLo
ok
for
Form
FSD
T
2.00
Fin
anci
al m
anag
emen
t
Wh
at c
on
stra
ints
do
bu
dg
etin
g, f
inan
cial
man
agem
ent,
dis
bu
rsem
ent
and
pro
cure
men
t sy
stem
s p
lace
on
th
e fi
nan
cial
sust
ain
abili
ty o
f th
e im
mu
niz
atio
n p
rog
ram
me?
Co
ntr
ibu
tors
to
th
is s
ecti
on
sh
ou
ld in
clu
de:
Min
istr
y of
Hea
lth F
inan
ce D
epar
tmen
t an
d/or
Min
istr
y of
Fin
ance
Qu
esti
on
sA
nsw
ers
Gu
idin
g c
rite
ria
Rem
arks
2.01
Are
acc
ount
ing
polic
ies
and
acco
unt
code
cla
ssifi
catio
nspu
blis
hed
and
appl
ied?
■■Ye
s■■
No
2.02
Is t
he n
atio
nal b
udge
t ap
prov
ed w
ithin
3 m
onth
s of
the
star
t of
the
fin
anci
al y
ear?
■■Ye
s■■
No
2.03
Is t
he r
eque
sted
bud
get
cons
iste
nt w
ith t
he a
lloca
ted
budg
et t
o th
e le
vel o
f at
leas
t 75
%?
■■Ye
s■■
No
2.04
Is im
mun
izat
ion
expe
nditu
re t
rack
ing
carr
ied
out
with
in3
mon
ths
afte
r cl
ose
of t
he q
uart
er?
■■Ye
s■■
No
2.05
Are
all
dono
r fu
nds
rele
ased
with
in 3
mon
ths
afte
rre
ques
t?■■
Yes
■■N
o
2.06
Are
gov
ernm
ent
fund
s re
leas
ed w
ithin
3 m
onth
s af
ter
requ
est?
■■Ye
s■■
No
2.07
Are
req
uest
ed f
unds
ava
ilabl
e at
hea
lth d
eliv
ery
leve
lw
ithin
3 m
onth
s of
due
dat
e?■■
Yes
■■N
o
93
3.00
Pro
gra
mm
e ch
arac
teri
stic
s, o
bje
ctiv
es a
nd
str
ateg
ies
Wh
at a
re t
he
pro
gra
mm
e o
bje
ctiv
es t
hat
fo
rm t
he
bas
is f
or
the
fin
anci
ng
req
uir
emen
ts o
f th
e im
mu
niz
atio
n p
rog
ram
me
ove
r th
e co
min
g f
ive
to s
even
yea
rs?
Co
ntr
ibu
tors
to
th
is s
ecti
on
sh
ou
ld in
clu
de:
Min
istr
y of
Hea
lthN
atio
nal I
mm
uniz
atio
n Pr
ogra
mm
e
Qu
esti
on
sA
nsw
ers
Gu
idin
g c
rite
ria
Rem
arks
3.01
Are
the
re s
peci
fic p
lans
to
intr
oduc
e ne
w a
ntig
ens
toth
e N
IP o
ver
the
next
5–7
yea
rs?
■■Ye
s■■
No
– H
ep B
– H
ib–
Yello
w F
ever
– O
ther
3.02
Are
the
re s
peci
fic p
lans
to
intr
oduc
e di
ffer
ent
form
ulat
ions
of
stan
dard
ant
igen
s ov
er t
he n
ext
5–7
year
s?
■■Ye
s■■
No
– Q
uadr
aval
ent
– Pe
ntav
alen
t–
Oth
er
3.03
Are
the
re a
ny p
lann
ed p
rogr
amm
e im
prov
emen
ts t
o th
eN
IP o
ver
the
next
5–7
yea
rs?
■■Ye
s■■
No
– C
old
chai
n im
prov
emen
tsan
d ex
pans
ion
– Re
duci
ng w
asta
ge–
Usi
ng a
uto
dest
ruct
or
Uni
ject
syr
inge
s
3.04
Are
the
re a
ny s
peci
fic p
lans
to
incr
ease
imm
uniz
atio
nco
vera
ge in
har
d to
rea
ch a
reas
and
/or
spec
ific
popu
latio
n su
b-gr
oups
?
■■Ye
s■■
No
3.05
Are
the
re a
ny p
endi
ng c
hang
es in
the
ran
ge o
fde
velo
pmen
t pa
rtne
rs t
hat
are
likel
y to
incr
ease
fin
anci
alpa
rtic
ipat
ion?
■■Ye
s■■
No
– D
onor
s–
Priv
ate
sect
or–
Tech
nica
l coo
pera
tion
agen
cies
– Bi
late
ral a
genc
ies
– M
ultil
ater
al a
genc
ies
– O
ther
3.06
Are
all
plan
ned
chan
ges
to t
he N
IP, t
heir
budg
etim
plic
atio
ns a
nd p
oten
tial f
undi
ng s
ourc
es a
ppro
ved
byna
tiona
l aut
horit
ies
and
part
ners
prio
r to
incl
usio
n in
the
mul
ti-ye
ar p
lan?
■■Ye
s■■
No
94
4.00
Bas
elin
e an
d c
urr
ent
pro
gra
mm
e co
sts
and
fin
anci
ng
Can
th
e co
un
try
pro
vid
e a
det
aile
d p
ictu
re o
f p
rog
ram
me
spec
ific
cu
rren
t fu
nd
ing
req
uir
emen
ts f
or
the
imm
un
izat
ion
pro
gra
mm
e?
Co
ntr
ibu
tors
to
th
is s
ecti
on
sh
ou
ld in
clu
de:
Min
istr
y of
Hea
lth N
atio
nal I
mm
uniz
atio
n Pr
ogra
mm
e, M
inis
try
of H
ealth
Pla
nnin
gD
epar
tmen
t, M
inis
try
of F
inan
ce
Qu
esti
on
sA
nsw
ers
Gu
idin
g c
rite
ria
Rem
arks
4.01
Are
the
cos
ting
and
budg
etin
g re
quire
men
ts o
f al
lim
mun
izat
ion
stra
tegi
es in
clud
ed in
the
cur
rent
NIP
?■■
Yes
■■N
o–
Rout
ine
imm
uniz
atio
n–
NID
s–
Mop
ups
4.02
Doe
s th
e cu
rren
t im
mun
izat
ion
budg
et in
clud
e a
recu
rren
t (o
pera
tiona
l) co
st b
reak
dow
n?■■
Yes
■■N
o–
Vacc
ines
– In
ject
ion
supp
lies
– Pe
rson
nel
– C
old
chai
n m
aint
enan
ce–
Tran
spor
t–
Soci
al m
obili
zatio
n–
Shor
t-te
rm t
rain
ing
– Su
rvei
llanc
e an
d m
onito
ring
4.03
Doe
s th
e cu
rren
t im
mun
izat
ion
budg
et in
clud
e a
capi
tal
cost
bre
akdo
wn?
■■Ye
s■■
No
– Tr
ansp
ort
– C
old
chai
n eq
uipm
ent
– Bu
ildin
g sp
ace
– Lo
ng-t
erm
tra
inin
g–
Oth
er
4.04
Doe
s th
e cu
rren
t im
mun
izat
ion
budg
et in
clud
e a
cost
brea
kdow
n of
imm
uniz
atio
n he
alth
sta
ff?
■■Ye
s■■
No
– Re
crui
tmen
t co
sts
– Sa
larie
s–
Perd
iem
s–
Ince
ntiv
es
4.05
Is t
he t
otal
cur
rent
fun
ding
for
the
NIP
know
n by
fund
ing
sour
ce?
■■Ye
s■■
No
– N
atio
nal g
over
nmen
t–
Sub-
natio
nal g
over
nmen
t–
Lend
ers
– D
onor
s–
Oth
er
95
Ther
e is
an
MS
Wo
rd v
ersi
on
of
this
fo
rm o
n t
he
CD
. If
you
nee
d t
o, y
ou
can
ed
it o
r am
end
it f
or
the
cou
ntr
y yo
u a
re a
sses
sin
g.
aLo
ok
for
Form
FSD
T
Qu
esti
on
sA
nsw
ers
Gu
idin
g c
rite
ria
Rem
arks
4.06
Is t
here
an
annu
al r
evie
w o
f th
e im
mun
izat
ion
prog
ram
me
fund
ing
and
expe
nditu
res
by t
hose
resp
onsi
ble
for
the
plan
ning
, bu
dget
ing
and
reso
urce
mob
iliza
tion
proc
ess
with
in t
he h
ealth
sec
tor?
■■Ye
s■■
No
– M
oH–
MoF
– IC
C–
Oth
er
4.07
Is t
he im
mun
izat
ion
budg
et f
orm
ulat
ed u
sing
rec
ent
cost
ing
data
and
/or
form
ulas
?■■
Yes
■■N
o–
Tran
spor
t op
erat
ing
cost
s–
Vacc
ines
– In
ject
ion
supp
lies
96
Ther
e is
an
MS
Wo
rd v
ersi
on
of
this
fo
rm o
n t
he
CD
. If
you
nee
d t
o, y
ou
can
ed
it o
r am
end
it f
or
the
cou
ntr
y yo
u a
re a
sses
sin
g.
aLo
ok
for
Form
FSD
T
5.00
Futu
re r
eso
urc
e re
qu
irem
ents
an
d p
rog
ram
me
fin
anci
ng
Giv
en r
eso
urc
e re
qu
irem
ents
an
d f
un
din
g p
rob
abili
ties
, wh
at is
th
e lik
elih
oo
d o
f th
e fu
nd
ing
gap
bei
ng
fill
ed?
Co
ntr
ibu
tors
to
th
is s
ecti
on
sh
ou
ld in
clu
de:
Min
istr
y of
Hea
lth N
atio
nal I
mm
uniz
atio
n Pr
ogra
mm
e, M
inis
try
of H
ealth
Pla
nnin
gD
epar
tmen
t, M
inis
try
of F
inan
ce,
ICC
part
ners
Qu
esti
on
sA
nsw
ers
Gu
idin
g c
rite
ria
Rem
arks
5.01
Hav
e co
stin
g an
d bu
dget
ing
calc
ulat
ions
bee
n m
ade
for
impr
ovin
g im
mun
izat
ion
stra
tegi
es o
f th
e N
IP o
ver
the
next
5 y
ears
?
■■Ye
s■■
No
– Ro
utin
e im
mun
izat
ion
– N
IDs
– M
op u
ps
5.02
Has
a 5
yea
r im
mun
izat
ion
budg
et b
een
deve
lope
din
clud
ing
a re
curr
ent
(ope
ratio
nal)
cost
bre
akdo
wn?
■■Ye
s■■
No
– Va
ccin
es–
Inje
ctio
n su
pplie
s–
Pers
onne
l–
Col
d ch
ain
mai
nten
ance
– Tr
ansp
ort
– So
cial
mob
iliza
tion
– Sh
ort-
term
tra
inin
g–
Surv
eilla
nce
and
mon
itorin
g
5.03
Has
a b
reak
dow
n of
the
imm
uniz
atio
n pr
ogra
mm
eca
pita
l cos
t re
quire
men
ts b
een
calc
ulat
ed f
or t
he n
ext
5 ye
ars?
■■Ye
s■■
No
– Tr
ansp
ort
– C
old
chai
n eq
uipm
ent
– Bu
ildin
g sp
ace
– Lo
ng-t
erm
tra
inin
g–
Oth
er
5.04
Hav
e co
stin
g an
d bu
dget
ing
calc
ulat
ions
bee
n m
ade
for
impr
ovin
g th
e ef
ficie
ncy
of t
he N
IP o
ver
the
next
5 y
ears
?■■
Yes
■■N
o–
Col
d ch
ain
impr
ovem
ents
and
expa
nsio
n–
Redu
cing
was
tage
– U
sing
aut
o de
stru
ct o
r U
nije
ctsy
ringe
s–
Oth
er
97
Ther
e is
an
MS
Wo
rd v
ersi
on
of
this
fo
rm o
n t
he
CD
. If
you
nee
d t
o, y
ou
can
ed
it o
r am
end
it f
or
the
cou
ntr
y yo
u a
re a
sses
sin
g.
aLo
ok
for
Form
FSD
T
Qu
esti
on
sA
nsw
ers
Gu
idin
g c
rite
ria
Rem
arks
5.05
Hav
e co
stin
g an
d bu
dget
ing
calc
ulat
ions
bee
n m
ade
for
addi
ng n
ew a
ntig
ens
to t
he N
IP o
ver
the
next
5 y
ears
?■■
Yes
■■N
o–
Hep
B–
Hib
– Ye
llow
Fev
er–
Oth
er
5.06
Hav
e co
stin
g an
d bu
dget
ing
estim
ates
bee
n m
ade
for
incr
easi
ng im
mun
izat
ion
cove
rage
of
popu
latio
n gr
oups
or
geog
raph
ic a
reas
tha
t ar
e be
low
the
nat
iona
l ave
rage
?
■■Ye
s■■
No
– O
utre
ach
cost
s–
Staf
f in
cent
ives
– Pe
rdie
ms
– O
ther
5.07
Hav
e co
stin
g an
d bu
dget
ing
estim
ates
for
hea
lth s
taff
been
mad
e in
line
with
incr
easi
ng c
over
age
targ
ets
over
the
next
5 y
ears
?
■■Ye
s■■
No
– Re
crui
tmen
t co
sts
– Sa
larie
s–
Perd
iem
s–
Ince
ntiv
es
5.08
Is t
he t
otal
fun
ding
req
uire
men
t fo
r th
e im
mun
izat
ion
prog
ram
me
by f
undi
ng s
ourc
e kn
own
for
the
next
5 y
ears
?■■
Yes
■■N
o–
Nat
iona
l gov
ernm
ent
– Su
b-na
tiona
l gov
ernm
ent
– Le
nder
s–
Don
ors
– O
ther
5.09
Are
gov
ernm
ent
fund
s fo
r th
e im
mun
izat
ion
prog
ram
me
pred
icta
ble
for
up t
o 5
year
s?■■
Yes
■■N
o
5.10
Are
don
or f
unds
for
the
imm
uniz
atio
n pr
ogra
mm
epr
edic
tabl
e fo
r up
to
5 ye
ars?
■■Ye
s■■
No
5.11
Is t
here
an
estim
ate
for
the
pote
ntia
l fun
ding
gap
for
imm
uniz
atio
n fo
r th
e ne
xt 5
yea
rs?
■■Ye
s■■
No
98
Ther
e is
an
MS
Wo
rd v
ersi
on
of
this
fo
rm o
n t
he
CD
. If
you
nee
d t
o, y
ou
can
ed
it o
r am
end
it f
or
the
cou
ntr
y yo
u a
re a
sses
sin
g.
aLo
ok
for
Form
FSD
T
Qu
esti
on
sA
nsw
ers
Gu
idin
g c
rite
ria
Rem
arks
5.12
Is t
here
any
indi
catio
n of
cha
nges
in f
undi
ng p
riorit
ies
ofna
tiona
l or
exte
rnal
fun
ding
par
tner
s th
at a
re li
kely
to
have
pos
itive
aff
ect
on t
he f
undi
ng o
f th
e im
mun
izat
ion
prog
ram
me?
■■Ye
s■■
No
5.13
Are
new
sou
rces
and
mec
hani
sms
of in
tern
al a
ndex
tern
al f
undi
ng b
eing
con
side
red
for
futu
re f
undi
ng o
fth
e im
mun
izat
ion
prog
ram
me?
■■Ye
s■■
No
– H
IPC
II/P
RSP
– N
ew d
onor
s–
Vacc
ine
inde
pend
ence
initi
ativ
e–
Revo
lvin
g fu
nds
– H
ealth
insu
ranc
e–
Oth
er
99
Ther
e is
an
MS
Wo
rd v
ersi
on
of
this
fo
rm o
n t
he
CD
. If
you
nee
d t
o, y
ou
can
ed
it o
r am
end
it f
or
the
cou
ntr
y yo
u a
re a
sses
sin
g.
aLo
ok
for
Form
FSD
T
6.00
Sust
ain
able
fin
anci
ng
str
ateg
ic p
lan
an
d in
dic
ato
rs
Wh
at a
re t
he
mai
n is
sues
an
d p
rob
lem
s im
ped
ing
fin
anci
al s
ust
ain
abili
ty, a
ctio
ns
that
will
be
take
n t
o a
dd
ress
th
e is
sues
and
pro
ble
ms,
an
d t
he
ind
icat
ors
an
d t
arg
ets
that
will
be
use
d t
o m
on
ito
r an
d e
valu
ate
the
acti
on
s?
Co
ntr
ibu
tors
to
th
is s
ecti
on
sh
ou
ld in
clu
de:
Min
istr
y of
Hea
lth N
atio
nal I
mm
uniz
atio
n Pr
ogra
mm
e, M
inis
try
of H
ealth
Pla
nnin
gD
epar
tmen
t, M
inis
try
of F
inan
ce,
ICC
part
ners
Qu
esti
on
sA
nsw
ers
Gu
idin
g c
rite
ria
Rem
arks
6.01
Has
a f
orm
al p
lan
been
dev
elop
ed a
nd e
ndor
sed
by t
heM
oF d
etai
ling
an in
crea
se in
the
gov
ernm
ent’s
sha
re o
ffu
ndin
g fo
r th
e N
IP?
■■Ye
s■■
No
6.02
Are
the
re a
ny c
ontin
genc
y pl
ans
or im
mun
izat
ion
rese
rve
mec
hani
sms
in p
lace
to
deal
with
pos
sibl
ere
sour
ce c
onst
rain
ts?
■■Ye
s■■
No
– Pr
iorit
izat
ion
of a
ctiv
ities
– Li
mite
d in
trod
uctio
n of
new
ant
igen
s–
Oth
er
6.03
Has
a lo
ng-t
erm
(5–
10 y
ears
) ca
pita
l fin
anci
ng p
lan
incl
udin
g al
l fun
ding
sou
rces
for
the
NIP
been
dev
elop
edan
d en
dors
ed b
y th
e IC
C?
■■Ye
s■■
No
6.04
Has
a f
orm
al p
lan
for
the
purc
hase
of
vacc
ines
and
AD
syrin
ges
been
dev
elop
ed a
nd e
ndor
sed
by t
he IC
C f
orth
e pe
riod
of p
ost
vacc
ine
fund
sup
port
?
■■Ye
s■■
No
6.05
Has
a f
orm
al p
lan
been
dev
elop
ed a
nd e
ndor
sed
by t
heIC
Cad
dres
sing
pos
sibl
e fu
ndin
g sh
ortf
alls
as
a re
sult
ofco
mpl
etio
n of
the
pol
io e
radi
catio
n pr
ogra
mm
e*?
■■Ye
s■■
No
* Po
lio f
unds
are
oft
en u
sed
tosi
mul
tane
ousl
y st
reng
then
the
rout
ine
prog
ram
me
6.06
Has
a m
ulti-
year
pla
n fo
r re
sour
ce m
obili
zatio
n be
ende
velo
ped
and
endo
rsed
by
the
ICC
?■■
Yes
■■N
o
100
Ther
e is
an
MS
Wo
rd v
ersi
on
of
this
fo
rm o
n t
he
CD
. If
you
nee
d t
o, y
ou
can
ed
it o
r am
end
it f
or
the
cou
ntr
y yo
u a
re a
sses
sin
g.
aLo
ok
for
Form
FSD
T
Qu
esti
on
sA
nsw
ers
Gu
idin
g c
rite
ria
Rem
arks
6.07
Doe
s th
e m
ulti-
year
pla
n in
clud
e co
st s
avin
g m
easu
res
toop
timiz
e th
e us
e of
vac
cine
s an
d im
mun
izat
ion-
rel
ated
supp
lies?
■■Ye
s■■
No
– Re
duci
ng w
asta
ge–
Usi
ng V
VM
– Ra
tiona
lizin
g se
ssio
ns–
Stoc
k co
ntro
l–
Prev
entiv
e m
aint
enan
ce–
Oth
er
6.08
Doe
s th
e m
ulti-
year
pla
n in
clud
e in
dica
tors
to
mea
sure
the
effic
ient
use
of
reso
urce
s?■■
Yes
■■N
o–
Cos
t pe
r FI
C–
Oth
er
6.09
Wer
e at
leas
t 75
% o
f th
e re
com
men
datio
ns o
f th
e la
stna
tiona
l rev
iew
of
the
NIP
on e
ffic
ienc
y an
d fin
anci
ngim
plem
ente
d?
■■Ye
s■■
No
6.10
Are
the
re s
elec
ted
indi
cato
rs t
o m
onito
r pr
ogre
ss m
ade
in t
erm
s of
sus
tain
able
fin
anci
ng?
■■Ye
s■■
No
– %
incr
ease
in n
atio
nal
expe
nditu
re o
n N
IP a
s a
shar
e of
GD
P af
ter
adju
stm
ent
for
debt
ser
vice
over
the
nex
t 5
year
spr
ogra
mm
e sp
ecifi
c co
sts
– D
epre
ciat
ion
sche
dule
for
repl
acin
g ca
pita
l ite
ms
– %
incr
ease
in s
ecur
edfu
ndin
g fo
r th
e N
IP–
%de
crea
se in
the
NIP
fund
ing
gap
– O
ther
101
Ther
e is
an
MS
Wo
rd v
ersi
on
of
this
fo
rm o
n t
he
CD
. If
you
nee
d t
o, y
ou
can
ed
it o
r am
end
it f
or
the
cou
ntr
y yo
u a
re a
sses
sin
g.
aLo
ok
for
Form
FSD
T
7.00
Stak
eho
lder
co
mm
ents
Is t
he
Fin
anci
al S
ust
ain
abili
ty P
lan
co
nsi
sten
t w
ith
th
e IC
C m
emb
ers’
per
spec
tive
s?
Co
ntr
ibu
tors
to
th
is s
ecti
on
sh
ou
ld in
clu
de:
Min
istr
y of
Hea
lth –
Fin
ance
and
/or
Plan
ning
Dep
artm
ent,
Min
istr
y of
Fin
ance
, ot
her
ICC
par
tner
s
Qu
esti
on
sA
nsw
ers
Gu
idin
g c
rite
ria
Rem
arks
7.01
Are
the
maj
or s
take
hold
ers
of t
he N
IP r
epre
sent
ed in
the
ICC
?■■
Yes
■■N
o–
MoF
, M
oP,
MoH
– D
onor
s–
Dev
elop
men
t ba
nks
– In
tern
atio
nal a
genc
ies
– O
ther
7.02
Has
the
cur
rent
imm
uniz
atio
n w
ork
plan
bee
n ap
prov
edby
the
ICC
?■■
Yes
■■N
o
7.03
Is t
he IC
Cop
erat
iona
l, w
ith a
leas
t 2
mee
tings
a y
ear
for
whi
ch m
eetin
g no
tes
are
avai
labl
e an
d Te
rms
ofRe
fere
nce
exis
t?
■■Ye
s■■
No
7.04
Are
the
rol
es a
nd r
espo
nsib
ilitie
s of
the
ICC
and
indi
vidu
al m
embe
rs,
in t
he p
lann
ed im
plem
enta
tion
ofth
e Fi
nanc
ial S
usta
inab
ility
Pla
n, a
ppro
pria
te a
nd w
ell
defin
ed?
■■Ye
s■■
No
– M
onito
r N
IPpr
ogre
ss–
App
rove
ann
ual w
ork
plan
s–
App
rove
mul
ti-ye
ar p
lan
– A
dvoc
acy
for
fund
ing
– A
ddre
ssin
g fin
anci
alsu
stai
nabi
lity
of t
he N
IP–
Oth
er
7.05
Is t
he t
rack
ing
of r
esou
rces
for
the
NIP
org
aniz
ed b
y bo
thIC
Cm
embe
rs,
as w
ell a
s st
aff
from
the
MoH
and
MoF
?■■
Yes
■■N
o
102
Ther
e is
an
MS
Wo
rd v
ersi
on
of
this
fo
rm o
n t
he
CD
. If
you
nee
d t
o, y
ou
can
ed
it o
r am
end
it f
or
the
cou
ntr
y yo
u a
re a
sses
sin
g.
aLo
ok
for
Form
FSD
T
Qu
esti
on
sA
nsw
ers
Gu
idin
g c
rite
ria
Rem
arks
7.06
Are
the
fun
ding
prio
ritie
s of
ICC
mem
ber
inst
itutio
nsco
nsis
tent
with
the
Fin
anci
al S
usta
inab
ility
Pla
n?■■
Yes
■■N
o
7.07
Is t
here
a c
ompr
ehen
sive
list
of
actio
ns t
o be
tak
en b
yth
e go
vern
men
t an
d by
ICC
mem
bers
to
mov
e to
war
dth
e ac
hiev
emen
t of
the
FS?
■■Ye
s■■
No
7.08
Are
dia
gnos
es o
f fin
anci
al s
usta
inab
ility
issu
es a
nd t
hest
rate
gies
to
addr
ess
cons
trai
nts
and
oppo
rtun
ities
inth
e Fi
nanc
ial S
usta
inab
ility
Pla
n co
nsid
ered
by
the
ICC
to
be a
ppro
pria
te a
nd r
ealis
tic?
■■Ye
s■■
No
New Vaccine Discussion Guide
Use this Data Collection Guide if you have been asked to assess the feasibility of introducing a newvaccine into the national immunization programme. Talk with epidemiologists and health managers inthe Ministry of Health to get information from a national perspective.
Checklists for new vaccine introduction, guidelines for new vaccine disease burden assessment, andguidelines for planning the introduction of new vaccines are available from WHO and UNICEF.
To further assess the capacity of the system to introduce and sustain the new vaccine, you will also needto consider data on the immunization services components and the health system.
Common Assessment Tool for Immunization ServicesBooklet 1 – Assessment Methodology 103
7Annex
? Question: What is known about the disease?
Discussion points
Ask clinicians whether they have ever seen the disease and whether they thinkit’s an important public health problem.
? Question: What are the disease burden estimates?
Discussion points
Ask for a copy of any disease burden studies for this disease that have beenmade in the country.
? Question: Do staff members believe it to be important to public health?
Discussion points
Ask managers what they know about the disease – is it a public healthproblem?
? Question: What is known about the new vaccine? Is it seen as beneficial?
Discussion points
Ask epidemiologists what they think the benefits and shortcomings of the newvaccine are.
Common Assessment Tool for Immunization Services104 Booklet 1 – Assessment Methodology
Glossary and abbreviations
Several definitions have been adapted from WHO’s Health Promotion Glossary and Family PlanningManagement Terms, Management Sciences for Health, 1996.
Advocacy
individual and social actions designed to gain political commitment, social acceptance, and organizationaland system support for a particular health goal or programme
Capacity
the ability of individuals, organizations, and systems to perform effectively and efficiently. Also, theirpotential to perform
Communication
informing the public about health concerns and keeping important health issues on the public agenda.Includes health promotion, health education, and social mobilization
Components
the five parts of the immunization services. These are:
• immunization service delivery
• disease surveillance
• logistics
• vaccine supply and quality
• advocacy and communications
Development partners
private and public, national and multinational organizations that offer technical, management, andfinancial support to developing countries
Disease surveillance
monitoring disease incidence, record keeping, reporting, and laboratory testing
Equity
fairness – all people have an equal opportunity to develop and maintain their health
External environment
the political, economic, social, and technological forces, trends, and stakeholders that have an impact on services
Common Assessment Tool for Immunization ServicesBooklet 1 – Assessment Methodology 105
8Annex
Functions
the actions a system is intended to perform. In this assessment methodology, the health system functionsexamined are:
• stewardship
• human resource development
• finance
• service provision
Immunization service delivery
the strategies and activities involved in providing vaccinations to clients
Indicator
a quantitative or qualitative statement that provides a basis for measuring progress towards objectives
Innovation
a new vaccine or technology introduced or considered for introduction into immunization operations, for example, injection equipment
Logistics
• the delivery of vaccines and other equipment in good condition to the place of use
• transport
• maintenance of the cold chain
• communications
• vaccine management
• waste disposal
Stakeholder
any person, group, or organization with an interest in an activity or service. Stakeholders include clients,health workers, the public, politicians, and development partners
Stewardship
the careful and responsible management of the public’s health, and of the individuals and organizationsthat provide health services
SWOT analysis
strengths, weaknesses, opportunities, and threats – an analysis technique that involves specifying thestrengths and weaknesses of an organization’s performance and the external opportunities and threatsthat may have an impact on it
Sustainability
the ability of an organization to:
• provide quality services to its clients
• increase or maintain demand for services
• expand services to reach people who are not currently reached
• generate financial resources
Common Assessment Tool for Immunization Services106 Booklet 1 – Assessment Methodology
Vaccine supply and quality
stock management, wastage monitoring, and the procurement of vaccines from manufacturers
Abbreviations
AEFI Adverse events following immunization
CB Capacity building
EPI Expanded Programme on Immunization
FIC Fully Immunized Child
FSDT Financial Sustainability Diagnostic Tool
GAVI Global Alliance for Vaccines and Immunization
GDP Gross Domestic Product
Hep B Hepatitis B vaccine
Hib Haemophilus Influenzae Type B vaccine
HIPC/PRSP Heavily indebted poor countries II/Poverty reduction strategy paper
HIV/AIDS Human Immunodeficiency Virus / Auto-Immune Deficiency Syndrome
ICC Interagency Coordinating Committee
MoF Ministry of Finance
MOH Ministry of Health
NGO Non-governmental organization
NIDs National immunization days
NIP National immunization programme
NRA National Regulatory Authority
SWAPS Sector Wide Approach Processes
SWOT Strengths, Weaknesses, Opportunities, Threats
TB Tuberculosis
VVM Vaccine vial monitor
Common Assessment Tool for Immunization ServicesBooklet 1 – Assessment Methodology 107
Common Assessment Tool for Immunization Services108 Booklet 1 – Assessment Methodology