guide for country multistakeholder policy reviews (for ... · guide for country multistakeholder...

1 Working draft 07 February 2014

Guide for country multistakeholder policy reviews (for country teams to adapt as required)

The country multi-stakeholder review will take place any time between end of January and March 2014, with the final report to be received by PMNCH and WHO by end-April 2014. Background and objectives

1. The ‘success factors’ study is a collaborative effort that seeks to draw lessons from high-performing countries on accelerating progress towards reducing preventable maternal and child mortality. The study uses different methods: a literature review, statistical analysis and econometric modelling of data across 144 low and middle-income countries; and policy analysis and multi-stakeholder reviews in 10 countries that were ‘on track’ in 2012 to achieving MDGs 4 and 5a.

2. The ten countries that were selected were: Bangladesh, Cambodia, China, Egypt, Ethiopia, Lao PDR, Nepal, Peru, Rwanda and Viet Nam. A draft policy analysis has been developed for each country.

3. Countries use a range of resources and reports to track their progress on women’s and children’s health, such as national MNCH reviews, MDGs reviews, post-2015 consultations etc. This exercise is an opportunity to synthesize lessons learned from the range of resources and to capture different stakeholder’s perspectives, from both within and outside the health sector, on key policies and programmes that made a difference for women’s and children’s health.

4. Countries can also use this opportunity to consider how to incorporate identified best practices into country policies, long term strategic plans or implementation plans for women’s and children’s health.

5. There are also opportunities to promote “south-south learning” among high performing countries, particularly as part of the post-2015 review and planning process, for example:

⋅ Post-2015 consultations, including a side session at the 2014 World Health Assembly in May 2014

⋅ Partners’ Forum, co-hosted by PMNCH, iERG, Countdown to 2015 and A Promise Renewed, in Johannesburg in June, 2014 where the final report will launched

6. The objective is to review the draft country policy analysis for each country with key health and development stakeholders to ensure that it presents: • Policy analyses informed by robust evidence and plausible pathways and which consider the

main elements of policy analysis: content of policy, actors, context and processes;1 • An accurate synthesis of the factors, catalysts and contexts that contributed to reductions in

maternal and child mortality– focusing on how countries achieved the improvements; with an emphasis on policy and program management best practices;

• An analysis of factors both within the health sector and outside of the health sector – with an emphasis on linkages between health and other sectors2;

• A discussion of lessons learned and future priorities.

1 See Walt et al. (2008) for the policy analysis framework. Available at: http://heapol.oxfordjournals.org/content/23/5/308.full 2 Building on the broader development approach promoted by the Commission on Investing in Health and quantified in the Global Investment Framework (Advancing social and economic development by investing in women's and children's health: a new Global Investment Framework. (November 2013) Stenberg K et al on behalf of the Study Group for the Global Investment Framework for Women's Children's Health. The Lancet DOI: 10.1016/S0140-6736(13)62231-X) which indicates that investments in women’s and children’s health lead to 9 times return on investment – in several areas including social cohesion, productivity and other factors.

http://heapol.oxfordjournals.org/content/23/5/308.full


Guide for reviewing the country policy analysis

There are different ways to develop country policy analyses on progress towards women’s and children’s health. Each policy analysis is organised by a semi-structured, deductive framework, based on identified policy factors that characterised high-performance in countries related to reductions in maternal and child mortality. The multi-stakeholder review process will provide a more inductive, grounded approach. It will bring together the stories and experiences of a range of stakeholders to inform the key findings and conclusions of each policy analysis. This approach draws on the premise that some of the most effective leadership lessons are communicated through good stories.3 The guide provides an outline of the stakeholder review process which will be overseen by the MOH in the country and supported by WHO H4+ and other health and development partners. The review will be facilitated by local and international consultants working with PMNCH. Part 1 provides a one-page summary of key review tasks and example questions. Annex 1 provides a more detailed overview of key questions to guide the review and possible sources of data. Part 2 of the guide provides details for organising the multi-stakeholder review. Box 1 provides a summary of methods for conducting the country multistakeholder review. The review will ideally be conducted by convening a 1 day stakeholder meeting. If a stakeholder meeting is not possible, consultants will conduct one-on-one or small group meetings with stakeholders to review the Success Factors document and findings.

3 See Gardner, H. Leading Minds: An Anatomy of Leadership. New York: BasicBooks.

Box 1: Success Factor Country Multistakeholder Policy Review methods 1. Literature review of peer-reviewed and grey literature, policy

documents, programme evaluations and sector strategies and plans. 2. Review of quantitative data and trends from population-based

surveys, routine data systems, international databases and other sources.

3. Multistakeholder policy review meeting, wherever possible, stakeholders participated in a one-day meeting to review and discuss the evidence and reach consensus on the key policy and programme milestones to reducing maternal and child mortality.

4. Additional key informant interviews and group meetings to address information gaps, help validate findings on success factors based on interviewees’ expertise and experience.

5. Review of the final country policy report by national stakeholders and experts, with final sign-off by ministries of health.


Part 1: Summary table of key review tasks (see Annex 1 for detailed overview)

Section of the document

Review Tasks Example questions to guide the policy analysis review

1. Executive Summary

⋅ Highlights of the policy analysis. Review/update as required.

⋅ Does it capture the main highlights?

2. Introduction

⋅ Methods and background. Review/update as required. (see Annex 2 for the general method)

⋅ Should the introduction provide any other background details?

3. Country context for health and development

⋅ Brief narrative on country context. Review/update as required.

⋅ Are the data accurate?

4. Key trends, timelines and challenges

⋅ Graphical overview of Trends on MMR, NMR, U5MR, TFR and nutrition provided. Review and update as required

⋅ Timeline – highlight key reform phases in both sectors ⋅ Review and update key challenges to progress

⋅ Are additional impact data needed to summarize current RMNCH status?

5. Health sector initiatives and investments

⋅ Based on existing reviews/reports and inputs, synthesize the 3-5 most important inputs possibly associated with mortality reduction and improved coverage

⋅ Review related trends and coverage levels using robust

evidence, to substantiate these inputs/ examples ⋅ For each example, construct a policy story based on the

evidence and develop the plausible pathway (see Annex 3). Please cover all elements of the policy analysis framework.

⋅ Are the most important improvements in RMNCH intervention coverage established?

⋅ What systems and program inputs led to improved coverage – what was done and how?

6. Initiatives/ investments in sectors outside of health

(e.g. education, WASH, nutrition, infrastructure, labour sector, science and innovation, etc)

⋅ Based on existing reviews/reports and inputs, highlight the 3-5 most important inputs outside the health sector possibly associated with mortality reduction and improved coverage

⋅ Review related trends and coverage levels using robust evidence, to substantiate these inputs/ examples

⋅ For each example, construct a policy story based on the

evidence and develop the plausible pathway. Please cover all elements of the policy analysis framework.

⋅ Are the most important inputs outside of the health sector identified?

⋅ Are/were there linkages across sectors that made a difference?

7. Key actors and political economy

⋅ Key actors and political factors which influenced and promoted successful adoption and implementation of key policies

⋅ Add a section on actors and political economy factors that have influenced strategic and policy decisions

⋅ What political factors drive adoption of health policy change?

⋅ Was there a champion for a key policy or intervention area?

8. Governance and leadership

⋅ Overall coordination and commitment, as well as specific to the health sector and sectors outside health. Review/update as required.

⋅ What national and sub-national organization or structural factors promote good governance and accountability?

9. Lessons learned and future priorities

⋅ Highlight the lessons learned in terms of key factors, catalysts and contexts.

⋅ What are the implications you would draw for post-2015 eg. in terms of goals, targets, approaches.

⋅ What are the general lessons from the Success Factor analysis?

⋅ How should findings inform the post-2015 agenda?

10. References ⋅ Cite all key references and list contributing partners ⋅ Are all references and contributing stakeholders included?

Working draft 07 February 2014 4

Part 2: Organising the multi-stakeholder review

1. Preparing for the multi-stakeholder review • The process will be led by the Ministry of Health. The WHO Country Office as the focal agency for

this project, will convene, or identify a convening organisation, for a multi-stakeholder review meeting. Preferably, this review process would be part of a regular meeting, or planned event.

• WHO, together with the convening organisation, will contract an expert country consultant to: review the draft policy analysis in advance and collect additional data as needed; assist with the planning and facilitation of the review; and finalise the revisions of the country Success Factor document. For continuity, this consultant could be a lead expert who had advised on the development of the draft country policy summaries (country contact list provided separately) (See Annex 4 for a draft ToR for the national consultant).

• International consultants are available to support the analysis and writing, and to facilitate a

standard approach across the country policy summaries.

• Logistics will be coordinated with WHO, the convening organisation and consultant/s. The review will ideally be conducted by convening a 1 day stakeholder meeting (logistics include: inviting participants, finalising the agenda, arranging the venue and materials. Key preparation tasks and an example timetable for a stakeholders meeting are summarized in Box 2 and Table 1 at the end of this section. If a stakeholder meeting is not possible, consultants should consider one-on-one or small group meetings with stakeholders to review the Success Factors document and findings.

• In advance of the stakeholder consultation, the national consultant supported by the international

consultant, will: 1) contact local staff involved in the development of the first draft of the policy analysis to ensure that they understand the local process used, and stakeholders consulted (contacts provided by PMNCH); 2) obtain and review relevant articles from the literature search (provided by PMNCH); 3) review and revise data tables and figures as needed (most recent global data provided by PMNCH); 3)Review the country policy analysis using the guide provided. The purpose of the guide is to ensure that the necessary data are included and accurate, that all relevant literature and reports have been incorporated, and that identified success factors meet reasonable plausibility criteria. Edits will be made as necessary and the revised summary will be sent to stakeholders prior to the review if possible.

2. Participants for the multi-stakeholder review (consider around 25 people) • Participants will be identified from all stakeholder constituency groups, including those involved in

development of the draft country policy analysis (see contact list). • Participants will also be consulted from other sectors (e.g. finance, planning, water and sanitation,

nutrition, education, gender).


3. Time required for the multi-stakeholder review (to be conducted between February and March

2014) • The time required for the multi-stakeholder review meeting will be one day. • The time required for one-on-one or small group stakeholder meetings will be variable –1-3 weeks. • Additional time will be required by the national and international consultant to review the current

draft of the Success Factors document, edit data and update the document prior to the stakeholders consultation – 1-2 weeks.

4. Conducting the review • The review will be informed by Part 1 of this document,

supported by Annex 1. This guide can be used both for the preliminary review by consultants and for the stakeholder review. Reviewers should provide feedback on the content, organization and format of the country Success Factor policy analysis – and make recommendations for edits and changes. If more data are needed, they should indicate where these data can be obtained, or key informants who can provide further information. Box 3 outlines the plausibility criteria for identifying success factors (see Annex 2 for more details).

5. Finalise the country policy summary and multi-stakeholder review report

• The consultant/s will collect any additional data and interview any additional key informants after

the initial stakeholder review meeting/consultations.

• The consultant/s will revise the country policy summary based on participants’ feedback/additional data in coordination with WHO, PMNCH and other study partners as required.

• The revised country policy summary can be sent by email to participants, including the Ministry of Health for finalisation. Any additional feedback from this step can be incorporated into the country summary by the consultant.

• The consultant also will prepare a short report on the multi-stakeholder review process, what

worked well, what did not and lessons learned and distribute to all key stakeholders.

• Country teams are requested to send the final country policy summary and the multi-stakeholder review report to PMNCH and WHO by end-April 2014.

Box 3: Plausibility criteria to help identify success factors Policy and programme inputs had to meet four plausibility criteria to be considered as success factors: 1. Potential impact (likely to have

contributed to mortality reduction based on an impact framework and available data);

2. Temporal association (had been implemented long enough to have influenced mortality);

3. Scale (had reached a large enough target population to influence mortality); and

4. Consensus (broad agreement between key stakeholders within and outside the health sector).


Box 2: Key Preparations tasks for multi-stakeholder review meeting The consultant, WHO coordinator, or RMNCH technical group should help arrange: • Invitations for all participants in the multi-stakeholder review meeting. Participants for the meeting

should receive an invitation, the most recent version of the document and a draft timetable for the meeting.

• A meeting room large enough for all participants.

• Materials and supplies, including:

- Paper and pencils for all participants; - A laptop computer and an overhead projector to use with the laptop computer; - Flipchart and markers.

• A budget for all proposed activities. This may include; per diems for the local consultant, technical

support staff and participants in the large group meeting; costs of transportation for the consultant (data gathering); venue costs; materials and supplies; document photocopying costs.

• Documents needed for the multi-stakeholder review, including:

- At least one copy of the journal references identified in advance and collected by the local consultant. These will serve as reference documents for the large group meeting.

- Participant materials, including:

• Summary of the review template and questions (Included in this document) • Most recent Success Factors document • Timetable


Table 1: Example timetable: Multi-stakeholder review meeting:

Time Activities 0900 -0930 Introduction

• Welcoming remarks • Participants introduction • Introduction of facilitators – consultant and coordinator • Background to Success Factors Analysis: - MOH RMNCAH • Current status of the document: methods, data used - consultant • Overview: how the Review meeting will be conducted – consultant

0930-1030 Plenary session: Review data in document - RMNCAH health status and coverage indicators (country context; key mortality trends and timelines; coverage trends) Agreement on data sources and main findings; edits as needed

1030-1045 Tea break 1045-1230 Small group work: Health sector investments/initiatives – review systems

and policies Break into 2-4 groups to review systems or intervention areas identified in the document as being success factors. Agreement on data used; factors identified; edits as needed. Groups summarise findings in plenary session – group discussion

1230-1330 Lunch 1330-1430 Small group work: Investments/initiatives outside of the health sector –

review activities and policies Break into 2-4 groups to review non-health areas identified in the document as being success factors. Agreement on data used; factors identified; edits as needed. Groups summarise findings in plenary session – group discussion

1430-1445 Tea break 1445-1545 Small group work: Political economy; leadership and governance

Break into 3 groups to review political, leadership, governance and private sector issues identified in the document as being important. Agreement on data used; factors identified; edits as needed. Groups summarise findings in plenary session – group discussion

1545 - 1645 Plenary session: Review proposed next steps and priorities and post-2015 agenda issues Agreement on main findings; additions, changes edits as needed Summary of next steps for finalizing the document Close


Annex 1: Detailed overview of review tasks to guide the multi-stakeholder review

Section 1: Executive Summary Review tasks ⋅ Highlights of the policy analysis.

Review/update as required.

Questions to guide the review ⋅ Does it capture the main highlights? Sources ⋅ Body of the document Section 2: Introduction Review tasks ⋅ Methods and background (see Annex 2 for the general method).

Review/update as required.

Questions to guide the review ⋅ Should the introduction provide any other background details? Tables/Figures ⋅ Map (design provided) Sources ⋅ Local staff involved with development of the first draft. ⋅ Stakeholder inputs. Section 3: Country context for health and development Review tasks ⋅ Brief narrative on country context, including: Geography; History as relevant to health status;

Population size and dynamics; Ethnicity; Language; Religion; Status of women; Economy – GDP per capita PPP; Inequalities; Revenue, trade and labour participation; HDI rank. Review/update as required.

Questions to guide the review ⋅ Are the data accurate? Tables/Figures ⋅ Table 1. Key country indicators (table provided) Sources ⋅ World Development Indicators ⋅ Agreed national data sources Section 4: Key trends, timelines and challenges Review tasks ⋅ Graphical overview of Trends on MMR, NMR, U5MR, TFR and nutrition provided.


Review and update as required ⋅ Timeline – highlight key reform phases in health and other sectors ⋅ Review and update key challenges to progress Questions to guide the review ⋅ Are data used valid and reliable? ⋅ Are additional impact data needed to summarize current RMNCH status? ⋅ Are additional coverage data needed to summarize current RMNCH status? ⋅ Are the policies and programmes highlighted those that have had the most impact in improving the

coverage of health services in contributing to reductions in maternal and child mortality? ⋅ Are there examples of success integration / harmonisation between disease-specific programming and

RMNCH? ⋅ What is the history of health sector reform? ⋅ What are the key reforms in other sectors. Tables/Figures ⋅ Figure 1. Trends on MMR, NMR , U5MR and TFR, Nutrition ⋅ Figure 2. Timelines with key policy inputs - health and sectors outside of health ⋅ Table 2: RMNCH coverage indicators Sources ⋅ WD Indicators (World Bank); GHO (WHO); CD to 2015 Country Profiles; DHS/MICS surveys; other large-

sample population-based surveys. ⋅ Agreed national data sources ⋅ RMNCH program staff; national statistics office; HMIS staff. Section 5: Health sector initiatives and investments Review Tasks ⋅ Based on existing reviews/reports and inputs, synthesize the 3-5 most important inputs possibly

associated with mortality reduction and improved coverage ⋅ Review related trends and coverage levels using robust evidence, to substantiate these inputs/

examples ⋅ For each example, construct a policy story based on the evidence and develop the plausible pathway.

Please cover all elements of the policy analysis framework. Questions to guide the review ⋅ Are the most important improvements in RMNCH intervention coverage established? ⋅ How did the key reform phases identified in the timeline Section 4 influence or play a role in the

changes? ⋅ What systems and program inputs led to improved coverage – what was done and how? ⋅ Is there evidence that the policies adopted have led to implementation changes consistent with those

policies? ⋅ Are data on systems and programme inputs accurate and available, and do additional data need to be

added? ⋅ What or who were the catalysts for change? What happened? What did they do? What was their role? ⋅ Was there a champion for a key policy or intervention area? What did they do that made a difference? ⋅ Were there specific innovations that made a difference?


⋅ Are there lessons learned from less successful system or programme activities?

Tables/Figures ⋅ Table 2 - Key RMNCH Coverage indicators Sources ⋅ Agreed national data sources ⋅ RMNCH coverage data. ⋅ Annex 5 – Review of systems and programmes inputs checklist is used to review inputs for each

initiative or investment area being considered. ⋅ RMNCH Policy compendium: health sector policies - Policy Compendium ⋅ Data on activities and policies inputs in system and programme areas– technical reports, program

documents, journal articles. ⋅ Interviews with staff in RMNCH or main systems areas. Section 6: Initiatives/ investments in sectors outside of health (such as education, WASH, nutrition, rural/urban infrastructure, labour sector, science and innovation, overcoming gender bias etc) Review Tasks ⋅ Based on existing reviews/reports and inputs, highlight the 3-5 most important inputs outside the

health sector possibly associated with mortality reduction and improved coverage ⋅ Review related trends and coverage levels using robust evidence, to substantiate these inputs/

examples ⋅ For each example, construct a policy story based on the evidence and develop the plausible pathway.

Please cover all elements of the policy analysis framework. Questions to guide the review ⋅ Are the most important inputs outside of the health sector identified? ⋅ Has the most important non-health sector programme or policy highlight been included? ⋅ Are data on non-health system factors accurate, and do additional data need to be added? ⋅ Is there evidence that the policies adopted have led to implementation changes consistent with those

policies? ⋅ Are there key sectors outside of health that made more of a difference? ⋅ How did the key reform phases identified in the timeline Section 4 influence or play a role in the

changes? ⋅ Were there specific innovations that made a difference? ⋅ Are/were there linkages across sectors that made a difference? ⋅ What or who were the catalysts for change? What did they do? What was their role? ⋅ Are there any lessons learned from less successful activities or inputs? ⋅ How has migration, internal and/or external, affected men and women’s roles and labor force

participation ⋅ Consider changes in women’s labour force participation and whether/how that has affected women’s

and children’s health Sources


⋅ Agreed national data sources ⋅ RMNCH Policy Compendium: Policies outside of the health sector ⋅ Annex 5 – Review of systems and programmes inputs checklist is used to review inputs for each

initiative or investment area being considered. ⋅ Data on activities conducted and policy inputs – policy and strategy documents, technical reports,

program documents, journal articles. ⋅ Interviews with staff in other sectors or systems areas outside of health. Section 7: Key actors and political economy Review tasks ⋅ Key actors and political factors which influenced and promoted successful adoption and

implementation of key policies Add a section on actors and political economy factors that have influenced strategic and policy decisions

Questions to guide the review ⋅ What place do health policies occupy in the political process? ⋅ What political factors drive adoption of health policy change? ⋅ How are health policies implemented? Does implementation generally respond to a general need of

the population (for example, nation-wide programmes of vaccination) or more to a specific clientele (for example, a hospital in a constituency where an MP seeks re-election)?

⋅ Within health policies, how are reproductive, maternal, newborn and child health (RMNCH) issues prioritized and framed?

⋅ To what extent did public pressure for improved health services help policy makers prioritize health in public spending and public policy?

⋅ Are there policy-makers that have displayed a consistent interest in terms of electoral or otherwise consent in prioritizing health policies over a long time period?

⋅ What are the influences of "non-state actors" such as the media, academia, professional organizations, communities and NGOs in framing public discourse on health?

⋅ What is the role of human rights and other global initiatives in driving country level change? ⋅ What has been the role of donors in this sector in promoting or funding change? ⋅ Did the media or social networks influence change or adoption of new interventions or behaviours?

Sources ⋅ Additional data on policy inputs – policy and strategy documents, technical reports, program

documents, journal articles. ⋅ Interviews with staff in RMNCH or sectors outside of health.

Section 8: Governance and leadership Review tasks ⋅ Overall coordination and commitment, as well as specific to the health sector and sectors outside

health. Review/update as required.

Questions to guide the review ⋅ What national/sub-national organization or structural factors promote governance and accountability? ⋅ What mechanisms are in place for ensuring joint planning, coordination and sharing of resources? ⋅ What, if any, was the role of donors?


⋅ How was/is local leadership and accountability promoted? ⋅ How often turnover in either high level political leadership and/or mid-level bureaucracy occur and did

this affect leadership, continuity of policies etc. Tables / figures ⋅ Consider a Figure or box summarizing different mechanisms that promote governance, coordination

and leadership both within the health sector and with sectors outside of health Sources ⋅ Additional data on mechanisms of governance – policy and strategy documents, technical reports,

program documents, journal articles. ⋅ Interviews with staff in RMNCH or systems areas and in sectors outside of health. Section 9: Lessons learned and future priorities Review tasks ⋅ Highlight the lessons learned in terms of key factors, catalysts and contexts. ⋅ What are the implications you would draw for post-2015 eg. in terms of goals, targets, approaches.

Questions to guide the review ⋅ What are the general lessons from the Success Factor analysis? ⋅ How should country targets be set? Are global targets and data used for estimates of RMNCH mortality

and coverage consistent with national targets and estimates? ⋅ What differences have you observed? Has this caused any problems with national planning or

programming? ⋅ How should findings from the policy analysis inform the post-2015 agenda? ⋅ Do you agree with the post-2015 recommendations from the study? If not, what are the changes you

would make? Sources ⋅ Revised Success Factor Policy Analysis ⋅ Success Factor overview document ⋅ Stakeholder interviews Section 10: References Review tasks ⋅ Cite all key references. Please also list the partner organizations and the individual partners who

contributed to the finalization of the country policy analyses, with their contact details. Questions to guide the review ⋅ Are all references and contributing stakeholders included? Is the WHO referencing style used?

Tables / figures ⋅ Consider a box summarizing post-2015 agenda issues

Sources ⋅ Policy and strategy documents, technical and program documents, journal articles; WHO style guide.


Annex 2: Methods used to develop the country policy analyses

The selection and development of the 10 country policy summaries is part of a multi-method, multi-country study undertaken by the PMNCH, WHO, World Bank, Johns Hopkins University, London School of Hygiene and Tropical Medicine and other partners to identify factors associated with high-performing countries’ reduction of maternal and child mortality.4 The 10 country policy summaries describe how policies and programmes across health, multi-sector and cross-cutting areas have helped shape progress on MDG 4 (reduce by two-thirds, between 1990 and 2015, the under-five mortality rate) and 5a (reduce by three quarters between 1990 and 2015, the maternal mortality ratio). MDG 5b (to achieve universal access to reproductive health by 2015) is another relevant target for improving maternal health. The methods employed to develop the country policy summaries included:

a) A literature review based on peer-reviewed and grey literature, policy documents, programme evaluations and sector strategies and plans. There were existing detailed case studies available for China, Egypt and Nepal which informed the development of these particular summaries. (see below for further information)

b) Extraction of quantitative data to develop key statistics and continuum of care tables as well as MDG 4 and 5a graphs

c) Key expert reviews to inform and help validate findings

Extraction of quantitative data In the overall success factors study, to facilitate comparisons across countries and across methods, (e.g. between the 10 country summaries and with the statistical analysis of 144 low and middle income countries for 20 years), we sourced quantitative data from global data sources such as the Countdown 2013 report on key maternal, newborn and child health coverage indicators. The World Bank DataBank and other databases were used to obtain data on demographic, economic, socio-political, infrastructure, health workforce and good governance indicators for all countries. Data from the World Bank DataBank were also used to develop charts highlighting trends over time for maternal, child and newborn mortality as well as fertility rates for each country. To finalize the individual country policy summaries, country teams would use national or global data sources that the national teams consider have the most recent and valid data. Key expert reviews Key experts who contributed to the development of each country summary were identified through existing stakeholder networks, WHO focal points in each country and by reaching out to informants directly through email. Key experts were from academia, government, UN and civil society organizations. A draft of each country summary was sent to government, WHO and other key experts in each country to review and help validate the findings.

4 These 10 countries were selected based on their being ‘on-track’ for MDGs4 and 5a in 2012: 7 of the 10 countries - Bangladesh, Cambodia, China, Egypt, Lao PDR, Nepal and Viet Nam are ‘on track’ to meet MDG 4 and 5a based on annual rates of change and absolute reductions. The remaining three countries, Ethiopia, Peru and Rwanda are ‘on-track’ to achieve MDG 4 and 5a as measured through rates of absolute reductions.


Literature Review The literature search strategy comprised an initial standardized search and then a tailored search depending on the relevance of results generated for each country. The search results across all countries have been documented in the flowchart below. The initial standardized search consisted of the following approaches: Overall success factors study literature review and synthesis by Frost et al (2012)5 . This

literature review method and search strategy was then adapted for the individual country summaries as described below.

Targeted search of PMNCH portal6, Measure/DHS, WHO, UNICEF, and UNFPA websites using key words: “child mortality,” “infant mortality,” “maternal mortality,” “trends,” “progress,” “success,” “decline”, “<<country>>”.

A literature search using PubMed and Google Scholar databases. Search strings for the databases were developed based on the literature review conducted by Frost et al (2012) and the specific requirements of the country summary reports. Variations of search strings were run in the different databases to identify which combinations would bring up the most relevant results across each country. Depending on the results emerging from this initial search, a more tailored search for the country was conducted to generate more relevant findings and/or to identify context specific policies and programmes. This involved looking at government and organizational websites, as well as conducting bibliographical searches from retrieved articles. Search strings for the tailored searches have been documented for each country. A two phase process was undertaken to assess abstracts and full-text articles for review based on the following exclusion criteria: • Outcomes: The document does not explain how reductions were achieved for at least one of the

following study outcomes: a) change in rates of under-5 mortality (birth to 59 months); b) change in rates of neonatal or neonatal mortality (birth to 1 month); c) change in rates of infant mortality (1 month – 12 months); d) change in rates of post-neonatal mortality (1 month-59 months); e) change in rates of maternal mortality (death of a woman while pregnant or within 42 days of termination of pregnancy, from any cause related to or aggravated by the pregnancy or its management).

• Country focus: The document mentions the country name but does not provide further MNH information

• Sub-national: The document has a subnational (i.e. district level) focus only with no national impact. • Timeline: The document refers to information pre-1990 • Duplicate: The document is a duplicate version of another report or study that is published

elsewhere. • Commentary: The document is conceptual, an advocacy piece, a commentary, or a textbook that

does not include national-level empirical data. • Inaccessible: Books or reports that are not publically accessible for screening within the study’s time

period. 5Frost, A and Pratt, A (2012).A Qualitative Evidence Synthesis of Factors Contributing to the Reductions of Maternal and Child Mortality in Low-Income and Middle-Income Countries.Global Health Insights. http://www.who.int/pmnch/knowledge/publications/qualitative_evidence_synthesis.pdf 6http://search.who.int/search?q=rwanda&ie=utf8&site=default_collection&client=pmnch_en&proxystylesheet=pmnch_en&output=xml_no_dtd&oe=UTF-8&sitesearch=http://www.who.int/entity/pmnch&ip=58.169.184.103&access=p&sort=date:D:L:d1&entqr=3&ud=1&proxycustom=%3CADVANCED/%3E

http://search.who.int/search?q=rwanda&ie=utf8&site=default_collection&client=pmnch_en&proxystylesheet=pmnch_en&output=xml_no_dtd&oe=UTF-8&sitesearch=http://www.who.int/entity/pmnch&ip=58.169.184.103&access=p&sort=date:D:L:d1&entqr=3&ud=1&proxycustom=%3CADVANCED/%3E





Annex 3: RMNCH Success Factors Analysis for the Health Sector – Plausibility Pathway

surveys rvesurvsurveysacility surveys

MICS

DHS

Improved survival and health (impact) • What trends have been observed? • Are all key data included? • What sources have been used – data valid and reliable?

Increased population-based coverage of effective interventions • Have data for all interventions expected to have an

impact on mortality along the continuum of care been included? Valid and reliable?

• Which have shown positive trends or met targets? • Which have not improved significantly or have no data

available

Results of activities in main policy, systems and program areas Do data support national improvements in? • Increased availability of/access to services • Improved quality of services • Increased demand for services/care • Improved information to/knowledge of caretakers and

families

Program activities conducted in policy, systems or program areas In which areas have inputs been made? What and how? Systems and program areas - • Planning and coordination • Financing • Human resources: pre-service and in-service • Clinical care – quality of care, supervision, referral • Supply chain - medicines, equipment and supplies • Community mobilization • Health information Policies, standards and guidelines

Measurement method MICS DHS International data-bases Countdown 2015 MICS DHS Large-sample surveys Countdown 2015 International data-bases Review of program reports and documents Hospital assessments Health facility surveys Household surveys Human resource assessments Review of program reports, documents, policies and guidelines, journal articles, interviews with RMNCH technical and program staff


Annex 4: Draft ToR for National Consultant

Terms of Reference for a short-term local consultant

Success Factors for Women’s and Child’s Health

Country multistakeholder review to finalize policy analysis and lessons learned for the post-2015 agenda

1. Background: There has been significant progress worldwide towards Millennium Development Goals (MDGs) 4 and 5a—to reduce child and maternal deaths. Since 1990, maternal and child mortality rates have both reduced by around 50%, but the numbers of preventable deaths are still unacceptably high. We know about the leading causes of preventable maternal and child mortality around the world and effective interventions to prevent them. But we know less about why some countries do better than others, despite similar economic contexts. Developing this understanding is important to help draw lessons from high-performing countries in order to accelerate progress towards reducing preventable maternal and child mortality. Knowledge about the high performing countries is also important to inform the development of related goals and measures of progress, including for the post-2015 development agenda. To contribute to ongoing efforts to understand the factors that influence reductions in preventable maternal and child mortality, the ‘Success Factors’ studies, a three-year collaborative effort, seeks to answer two key questions: - What factors statistically distinguish high-performing countries from countries that did not perform as well in reducing maternal and child mortality, given similar levels of economic development? - How have high-performing countries optimized performance in reducing maternal and child mortality, including in policies and programmes related to the identified high performance factors? [Country] is one among 10 identified high-performing countries. A country policy analysis has been drafted which describes key policies and programmes that helped accelerate progress in women’s and children’s health. The draft country policy analysis “Success Factors in Women’s and Children’s Health – Mapping Pathways to Progress: [country]” will be validated as part of a country multi-stakeholder review process. Findings of the analysis will be incorporated within an overall report on “Success Factors” in the ten studied countries. The final report on the Success Factors for Women’s and Children’s Health, including the country policy analyses will be launched at the 2014 Partners’ Forum in June, co-hosted by the Government of Republic of South Africa, PMNCH, Countdown to 2015, A Promise Renewed and the independent Expert Review Group (iERG).

http://www.who.int/woman_child_accountability/ierg/en/


2. Objectives of the country multi-stakeholder review:

• To review and finalise the country policy analysis “Success Factors in Women’s and Children’s Health – Mapping Pathways to Progress: [Country]”

• To validate the draft findings for the overall study and to propose recommendations for the post-2015 development agenda.

• To contribute to a publication on Success Factors for Women’s and Children’s Health, to be launched at the Partner’s forum in June 2014.

3. Tasks of the Local consultant The Consultant will provide technical support to the MoH and to WHO/[country] to facilitate the review process and finalise the country policy analysis. The review will take place between February and March 2014, with the final document to be produced by end-April 2014. The Consultant will report to the [xxxxx] and will work closely with WHO/MCH teams, the WHO/HQ external consultant and the MoH. In particular, the consultant will take responsibility for the following tasks: 1. Review technical content of the draft [country] Success Factor Analysis document

• Ensure that all data used in the analysis are valid, reliable and use the most up-to-date

sources; add any additional data where needed. (review of DHS, MICS and other population-based surveys; review of international data sources (WHO, UNICEF, World Bank); review of peer reviewed journal studies and articles – review has been done and articles are available).

• Review success factors in major health sector and multi-sector areas; ensure that data

support the inclusion of these factors with an emphasis on establishing how selected factors contributed to improved intervention coverage and mortality decline; add additional success factors if necessary; obtain additional data to support existing or additional factors as needed (consult with staff involved with the first draft of the analysis; arrange one-on-one meetings with key stakeholders in the MOH, development partners and NGOs to review content). Key content areas: human resources; financing; policies, planning and coordination; medicines and commodities; service delivery)

• Incorporate feedback and recommendations from all interviews and data review into the country draft and circulate it to informants for validation (in collaboration with the international consultant supporting the review).

2. Prepare for and help conduct a one-day stakeholder review and consensus workshop on the

Success Factor analysis

Work with [xxxxxx], WHO teams, and the WHO/HQ external consultant to: • Organize the one-day workshop during the third week of March (incl. identifying participants,

sending invitations, completing the revised success factors analysis summary and making it


available to participants prior to the meeting, finalizing the timetable, arranging the venue and materials).

• Work with the WHO/HQ consultant - who will come in the days around the workshop - to finalize the draft document, support the MoH to conduct the workshop and contribute as facilitator.

• Prepare a report on the multi-stakeholder review process, in collaboration with the WHO/HQ consultant, on what worked well, what did not and lessons learned.

4. Develop final draft of the Success Factor Analysis

Work with the WHO/HQ external consultant, [xxxxxx], and WHO teams to: • Revise the country policy analysis based on key informants’ feedback in the workshop

• Find additional data and conduct key informant interviews as needed to complete proposed

edits, additions or changes

• Consolidate and finalize the policy analysis and submit it to the MoH and to WHO/PMNCH by end-April 2014.

5. Required education and experiences Qualifications: Essential: Medical doctor or Bachelor degree with master in Public Health, Desirable: Post graduate qualification on health policy Experience: Essential: Minimum 3 years’ working experience in health policy; report writing in English; Desirable: Experience in the area of reproductive, maternal, newborn and child health policy and services; experience in organization of high level workshops and conferences Competencies: Excellent communication and inter-personal skills, ability to communicate effectively within different levels of governmental institutions, international agencies and NGOs, ability to interpret and evaluate research data, excellent team work and commitment to contributing to improved health in [country]. Language requirement: Excellent spoken and written English.


Annex 5: Review of systems and programme inputs in the health Sector and initiatives / investments in sectors outside of health that have affected delivery of RMNCH Interventions

Intervention or Intervention package OR initiative/investment outside of health to be reviewed :_____________

National Strategic Choices Progress in Program implementation

Preparatory Early Mature Planning, coordination to ensure access and availability

Are implementation/scale-up plans made? Is a coordinating body working? Are key stakeholders involved?

Is the package included in national & district workplans? Have partner roles been established? Are criteria for selecting early implementation areas decided?

Has implementation begun in pilot areas? Is implementation by government sector or other partner support? Is there regular coordination and sharing lessons learned?

Is geographic expansion underway (% districts receiving the intervention)? Are underserved populations being reached? Is the approach institutionalized at all levels?

Financing

Have funding sources been identified for activities (Gov and development partners)? Are resources available at sub-national levels when they are needed?

Are national and sub-national budgets and financing plans for implementation available?

Does budgeting and mobilization of resources happen regularly at all levels? Are resources available where and when they are needed?

Are resources allocated to all levels where they are needed each year? Is the source of funding secure and without interruption?

Workforce

Has the availability of staff to provide services been reviewed - has a training needs assessment been conducted? Is there a plan to recruit and train additional staff/fill gaps available? Does this plan include strategies to improve retention and motivation? Have accreditation standards been established, if relevant?

Have training materials and curriculae been developed? Are master trainers or facilitators available? Has a training plan been developed and costed – pre and in-service? Has a plan to improve availability and retention of staff been implemented?

What proportion of workers in the sector have been trained? Is the quality of training monitored? Is availability, retention of staff and use of incentives monitored? Is availability of accredited institutions and/or workers monitored?

Is the training mainly supported by the relevant government ministry? Is training ongoing? Are workers motivated and retained? What do data on staff numbers, allocation and coverage show?

Supporting quality services in institutions and community

Have implementation approaches been selected – institutions and community? Is there an implementation plan available? Are materials/guidelines to support implementation available? Does implementation involve a forming a quality team to oversee the process?

Does supervision system/checklist include: review of key functions; observation, feedback and problem solving? Is a plan to improve referrals or problem solve gaps in place? Are methods and measures to track quality of services identified?

Is regular supportive supervision conducted in implementation areas? Is the referral system or problem solving system working? Is quality of service delivery regularly measured and are data used?

Do districts conduct regular quality improvement activities for this area of service delivery? Do data show improvements in the service?

Supply chain

Has an assessment including logistics of estimation, procurement, distribution and ordering of supplies been conducted?

Are equipment and supplies procured + available where and when needed?

Are equipment/supplies tracked? Are actions taken to improve availability?

Are all districts equipped and have essential supplies? Are stock-outs rare? Do data support improved supply?

Community mobilization

Has a community mobilization (CM) strategy been developed? Are key community groups and decision-makers included?

Have key CM materials been developed and tested?; Are facilitators/trainers for CM identified and trained?

Have CM approaches been implemented? What evidence is there to support this?

Have CM approaches been established in all districts - run and overseen by communities?

Advocacy, behavior change, social marketing

Is formative research available? Are behavior change strategy (s) and plan developed and available for the activity or intervention? Has an implementation scale-up plan been made?

Have key behaviors been selected for focus? Are methods and channels for reaching target audience defined? Are messages and materials developed and field tested?

Has the behavior change approach been implemented?

Are behavior change activities included in national and sub-national workplans and are funds available regularly?

Information systems

Is situation analysis completed? Are indicators and targets developed – for both surveys and routine information systems?

Are data collection tools revised?

Are data collected, analyzed and used in implementation areas?

Are key indicators in routine data systems tracked?


Hypothetical example of 4 step process applied to a Health intervention success Observed impact: Country X has achieved a significant downward trend in maternal mortality since 2000 as measured through the maternal mortality verbal autopsy module of the Demographic and Health Survey (DHS) such that they can claim to have achieved MDG 5a. The decline in maternal mortality was entirely because of reductions in direct obstetric deaths. Declines in maternal deaths were noted for hemorrhage (35% decline), eclampsia (50% decline), abortion (85% decline), and obstructed labor (26% decline). The TFR fell from 3.2 to 2.5 between 2001 and 2010, a 22% decline. The fall in fertility has resulted in a decline in births to older mothers and high parity births, both of which are associated with higher maternal mortality. Step 1: Identify coverage indicators (and supporting data) that may have contributed to the observed impact. Facility deliveries. The proportion of women delivering at a health facility rose from 9% in 2001 to 23% in 2010. Much of this increase was from a rise in private sector facility deliveries (2.7% to 11.3%), although public sector facility deliveries also rose (5.8% to 10%). A high proportion of women continue to deliver at home. Delivery with a Skilled Birth Attendant (SBA). Delivery with a SBA doubled from 12.2% in 2001 to 26.5% in 2010. Most deliveries with a SBA took place at a health facility (4% of SBA deliveries were conducted at home). Cesarean section (c-section) rate. A five-fold increase in the c-section rate was noted from 2.6% in 2001 to 12.2% in 2010. Half of all facility deliveries in 2010 were by c-section, 71% of these were conducted in private facilities and 35% in public facilities. Most mothers receiving a c-section reported delivery complications; 9.4% reported no complications, and may have received the procedure unnecessarily. Contraceptive prevalence rate. The rate for any method rose from 54% in 2000 to 62% in 2010, and the rate for a modern method from 43% to 53% in 2010. Step 2: What were the system or programme inputs that contributed to coverage improvements? How did the country achieve this? Treatment seeking for complications. More women who reported obstetric complications sought care, rising from 53% in 2001 to 68% in 2010. This care included seeking home-based care, purchasing medicines from pharmacies, and seeking facility-based care. Seeking treatment from a facility for a complication rose from 16% to 29%. Twelve percent of women purchased medicine from a pharmacy, suggesting that they self-administered antibiotics and other medicines. Improved access to health services. By 2000, 59 district hospitals and 60 health centers offered comprehensive EmONC services. By 2010, an additional 132 health centers had been upgraded to provide comprehensive emergency obstetric care and 1,500 clinics to provide basic EmONC. Qualitative data suggest that other factors may also have increased use of services, including more widespread use of mobile phones to contact providers and improved road infrastructure in some areas.


Improve use of contraception. A national community-based awareness campaign has been running since 2005 using local drama and community education meetings. Household survey data indicate that 72% of women of reproductive age have been exposed to messages. Improved female education and awareness. The proportion of mothers with no education halved between 2001 and 2010, and the proportion of women with some secondary education or greater increased by 18%. Fewer uneducated women are giving birth, and their care behaviors have improved over time. Care-seeking for obstetric complications from facility-based providers rose from 8.6% to 16.9% among women with no education between 2001 and 2010. C-section rates rose five-fold for women in both the poorest and wealthiest wealth quintiles. Since 1995 the number of trained school teachers has doubled; and the number of schools in rural areas has increased by one third. Improved economic conditions. Gross national income per capita rose from US$350 in 2001 to US$550 in 2008. Improved wealth should improve quality of housing, water, and sanitation and provide more funds for medical emergencies and facility-based care. Step 3: What were the policy inputs? Are these likely to be responsible for observed programme and system changes? Health Sector policy; 20% of national health budget allocated to RMNCH since 2000. SBA expansion policy. Training schools have expanded training numbers by 200 per year since 2002, and most upgraded facilities now have SBAs deployed and available. A policy to develop a new catehory of SBAs – community SBAs has been adopted. Community SBAs are able to provide skilled delivery, timely referral, and immediate postpartum care at home or in facilities. However, training did not begin until 2008 and the first cohort was deployed in 2009. This policy has therefore not played a role in improving skilled birth attendants. EmONC upgrading policy and plan – led to an increase in the availability of EmONC services country-wide, beginning in 2001. Education for all policy – leading to increases in numbers of school teachers, schools, and increased budget allocation to public education and a widespread public awareness campaign. Implementation began in 1995. Step 4: Synthesis of findings Maternal mortality declined 40% between 2001 and 2010, to 194 deaths per 100,000 live births.

The main reasons for the mortality decline were:

− Fertility reductions that reduced the proportion of high-risk, high parity births − Increases in facility deliveries (9% to 23%), deliveries by SBAs (12% to 27%) and care-seeking for

complications from a facility-based provider (16% to 29%)

Improvements are associated with improved access to care, a reduction in financial barriers to seeking care, and increased awareness of the importance of care-seeking. Improvements in female education and economic conditions have played a role.