Landscape Study of the Cost, Impact, and Efficiency of Above Service Delivery Activities in HIV and Other Global Health Programs

Jack Clift, Daniel Arias,

Michael Chaitkin, Meghan O'Connell,

Arjun Vasan, Teresa Guthrie,

Stephen Resch, and Robert Hecht

Results for Development Institute

July 2016

Results for Development Institute (R4D) is a nonprofit organization

whose mission is to unlock solutions to tough development challenges

that prevent people in low- and middle-income countries from realizing

their full potential. Using varied approaches in multiple sectors, including

global health, global education, governance, and market dynamics, R4D

supports the discovery and implementation of new ideas for reducing

poverty and improving lives around the world.

This report was authored by a team at R4D led by Jack Clift under the

general direction of Robert Hecht and including Daniel Arias, Michael

Chaitkin, Meghan O'Connell, and Arjun Vasan, with additional support

from Teresa Guthrie and Stephen Resch. For more information, please

contact Jack Clift at jclift@r4d.org.

Copyright © 2016 Results for Development Institute 1111 19th Street, N.W., Suite 700, Washington, DC 20036

Acknowledgements

The authors would like to thank the Bill and Melinda Gates Foundation

(BMGF) for supporting this study, in particular Kate Harris, Marty Gross,

Damian Walker, and Sarah Hamm Rush. The report also benefited from

the review and insights provided by BMGF staff, including Logan Brenzel,

Win Brown, Yvette Gerrans, Bruno Moonen, Ellen Piwoz, and Raja Rao.

The team would like to express its gratitude to the experts and

stakeholders interviewed for this report, who contributed their time and

input to the study. The names of experts and stakeholders interviewed

for this report are listed in Appendix 1. In addition, a number of Results for

Development Institute (R4D) staff provided support and peer review in the

preparation of this report: the authors wish to thank Cheryl Cashin, Mary

D’Alimonte, and Shan Soe-Lin for their contributions.

Landscape Study of the Cost, Impact, and Efficiency of Above Service Delivery Activities in HIV and Other Global Health Programs i

Table of Contents

Introduction 1

Overarching Themes and Taxonomies of Costs 2

Definitions and working taxonomy 2

Overarching themes: ASD costs 4

Overarching themes: ASD impact/efficiency 8

Above Service Delivery Costs in HIV Programs 9

What is the current level of publicly available data for HIV-related ASD costs? 9

What are ASD activities and what share of program spending do they comprise? 10

What are current, ongoing, or planned efforts to improve the knowledge base of HIV ASD costs, impact, and efficiency? 15

What are the key gaps in the HIV ASD knowledge base? 16

Above Service Delivery Costs in Other Programs 18

Immunization 18

Tuberculosis 25

Malaria 29

Nutrition 34

Family Planning 40

Cross-Program Focus Topics in Above Service Delivery Costs 44

Supply chain and procurement 44

Lab operations 50

Above Service Delivery Costs Related to Aid Architecture 56

Overview of aid architecture 56

Above-national costs of international aid organizations 57

Overhead costs of implementing partners 59

In-country expenses: Expatriate labor and external consultants 59

Priorities and Opportunities for Future Research and Programming: Recommendations 61

Recommendations regarding future research, data gathering, and harmonization 61

Recommendations regarding future programming 64

Appendices 66

Appendix 1: Experts and stakeholders interviewed 66

Appendix 2: Literature search protocol 67

Appendix 3: Challenges with using historic National Health Account data 68

References 69

ii

Figures

Figure 1. Illustrative taxonomy of costs incurred in the delivery of health services 3

Figure 2. EPIC costs categories by activity and level 5

Figure 3. PEPFAR taxonomy of program areas and cost categories 6

Figure 4. High variation in ASD estimates across countries within programs 7

Figure 5. Select NASAs: Percentage of spending above the point of service delivery 10

Figure 6. NASA Program Management costs disaggregated (percentage) 11

Figure 7. PEPFAR expenditures by high level cost category (percentage) 12

Figure 8. PEPFAR HSS expenditures in South Africa, 2012 to 2014 (ZAR millions), by detailed category 13

Figure 9. PEPFAR HSS spending in South Africa, 2012 to 2014 (ZAR millions), by category and geographical location 14

Figure 10. Percent of spending above the point of service delivery, reported in selected NASAs against 12-month ART retention rates 16

Figure 11. Prototypical cMYP results presentation, indicating the breakdown of cost data by input rather than by activities 19

Figure 12. Immunization program cost per dose in EPIC study countries (USD, 2011) 20

Figure 13. Distribution of immunization program costs by organizational level in EPIC study countries (USD millions, 2011) 20

Figure 14. Percent of total immunization program costs incurred at the district, region, and central levels in EPIC study countries 21

Figure 15. Breakdown of immunization program costs by activity for four sub-Saharan African EPIC countries, including vaccine and injection supplies 21

Figure 16. Share of each program activity cost incurred above the facility level in 4 sub-Saharan African EPIC countries 22

Figure 17. Trends of completeness for JRF reported data, as reported by the WHO IVB; trend data highlights gaps and inconsistencies in reporting over time 23

Figure 18. WHO tracked TB expenditure data (USD 2010- 2014), disaggregated by program activity/category 26

Figure 19. WHO tracked TB expenditure data indicating the percent of total country ASD-related expenditure plotted against TB treatment success rates (percent of new cases, 2012) 28

Figure 20. Average annual program management costs for malaria by program phase and territory, from Sabot et al. 30

Figure 21. Breakdown of projected malaria program financial need for five sub-Saharan African countries, disaggregated by program activity/category (USD, 2014) 31

Figure 22. WHO tracked malaria financing data (USD, 2014), disaggregated by program activity/category 32

Figure 23. Governance costs by activity for selected SUN costed national plans, indicating the heavy proportional costs of system capacity building as a share of total governance costs 36

Figure 24. Program costs by activity category for SUN costed national plans for 11 countries in sub-Saharan Africa, with nutrition-sensitive costs excluded 38

Figure 25. Budgeted nutrition allocations in Nepal span six ministries (NPR thousands, 2013-2014) 39

Figure 26. Allocation of projected FP spending needs for ASD activities 41

Figure 27. Costs of meeting demand for modern contraception under current coverage, improved care, and universal coverage scenarios 42

Figure 28. Total immunization program costs by component and by routine vs. SIA (2011-2020) 46

Figure 29. Laboratory spending as a share of total facility-level expenditure across the five MATCH study countries 50

Figure 30. WHO TB expenditure data for laboratory services in seven sub-Saharan countries (2010-2014), as a percentage of total country TB expenditure 51

Figure 31. SARA 2010 results regarding the availability of standard laboratory tests, per district, among 17 districts in Zambia 53

Figure 32. Conceptual model for understanding ASD activities associated with Aid Architecture 56

Figure 33. PMI funding by fiscal year (USD, FY 2006-2014) 58

Tables

Table 1. Health system components of the most current cMYP analysis framework 19

Table 2. Percent of total health expenditures for TB identified by NHA analysis for governance and administration 27

Table 3. Percent of total health expenditure for malaria identified by NHA analysis for governance and administration 31

Table 4. A sample NHA profile of expenditures for nutritional deficiencies by use function (Niger) 37

Table 5. NHAs that include expenditure tracking for nutrition rarely include tracking for ASD activities (e.g., "Governance, health system, and financing administration") 37

Table 6. Availability of Costed Implementation Plans and underlying costing models 40

Table 7. Categories of projected indirect costs for FP in Adding It Up 42

Table 8. Total estimates of supply chain costs, by tier, for Nigerian contraceptives logistics management system (USD) 45

Table 9. EPIC study data for four sub-Saharan African countries for cold chain maintenance and vaccine collection, distribution, and storage 47

Table 10. USG agencies receiving PEPFAR Technical Oversight and Management (TOM) funds 57

Table 11. Percent of PEPFAR funding by fiscal year approved for Technical Oversight and Management 58

Table 12. Share of PMI funding by fiscal year allocated to Headquarters 58

Table 13. Share of spending identified by PEPFAR expenditure analysis for external consultants 60

Table 14. Consolidated data from the health sub-accounts of 7 sub-Saharan African countries 68

Landscape Study of the Cost, Impact, and Efficiency of Above Service Delivery Activities in HIV and Other Global Health Programs 1

Introduction

Costs incurred by health programs for activities

conducted above the front-line facility or community

setting, referred to in this report as costs above the

point of service delivery (ASD), constitute a substantial

share of health program spending: for example, for

2014, ASD expenditures for the President's Emergency

Plan for AIDS Relief (PEPFAR) reported through the

PEPFAR dashboards amounted to over 1.5 billion USD, or 45 percent of the total expenditures reported.

Despite the vast sums spent above the point of service

delivery, far less is known about ASD costs than costs

at the point of service delivery.

Understanding the costs, impact, and efficiency of

ASD activities should be a priority for all stakeholders,

as policy-makers and funders seek their expenditures

on large health programs to be more efficient and

have a greater impact, and as beneficiary countries

assume responsibility for a larger share of spending on

programs as they transition away from donor funding.

In instances in which spending on ASD activities is

inefficient or wasteful, the funds could be reallocated

to fund front-line services. Conversely, ASD activities

that have a strong impact on the quality or efficiency

of service delivery could be supported further to

maximize scarce resources.

To better understand and assess the potential

for future improvements in this area, the Bill and

Melinda Gates Foundation (BMGF) asked Results for

Development Institute (R4D) to assess the current

state of knowledge and practice regarding ASD costs

for HIV and other health programs (immunization,

tuberculosis, malaria, nutrition, and family planning).

From July 2015 to April 2016, R4D conducted desk

research and expert interviews and facilitated a

consultation between BMGF, experts, and other

stakeholders.1 This report represents the final output of

this research.

The report examines the following critical questions

relevant to three specific groups of stakeholders:

For ministries of health: How much money are

different countries and disease programs spending

on ASD activities? How important are investments

in these activities and systems? In which might it be

most feasible to improve technical efficiency?

For donors: What roles do ASD activities play in

driving value for money in global health programs?

As countries transition from donor funding, how can

donors better evaluate their own ASD spending and

provide transitioning countries with a clear rationale to

fund these activities?

For foundations, NGOs, and the research community: What are the critical gaps in knowledge

surrounding ASD activities, and how can they be filled?

What actions should stakeholders take today based on

the knowledge that already exists?

In this report, to answer these questions, we examine

in detail publicly available data about ASD costs,

discuss the current state of knowledge and ongoing

efforts to understand the impact and efficiency of

ASD activities, and offer recommendations based on

our preliminary findings and the recommendations

of experts and stakeholders that we consulted

throughout the process.

In the first chapter of the report, we provide an overall

taxonomy of ASD activities and provide an overview

of the costs, impact, and efficiency of ASD activities.

The second chapter focuses on ASD activities in the

context of HIV programs, the primary programmatic

area of interest in the report. The third chapter

summarizes our findings in additional programmatic

areas of interest to ministries of health and other

stakeholders that provide additional context for HIV:

immunization, tuberculosis (TB), malaria, nutrition, and

family planning. The fourth chapter explores in more

detail two cross-cutting topics of particular interest:

Supply Chain and Procurement, and Laboratory

Operations. The fifth chapter explores costs related

to Aid Architecture. The sixth chapter summarizes the

recommendations of the report. We provide additional

background on our methodology and the sources

that we consulted in the appendices.

1 We provide further details on the experts who we interviewed and the literature that we reviewed in the appendices to this report.

2

Overarching Themes and Taxonomies of Costs

level. In addition, we consider as point of service

activities those activities that are performed in

communities (outreach or ‘below facility’ activities),

such as community screening or vaccination drives.

Second, we do not include in our taxonomy all of

the activities that are associated with the national

(or international) response to a given health priority.

In particular, we do not address non-health-related

activities, such as social protection for Orphans

and Vulnerable Children, that often are included in

national assessments of spending on HIV programs,

but which have an indirect impact on health

outcomes, and for which the health benefit is only

one of several potential justifications for the societal

resources invested.

Third, the different health programs discussed in this

report have different emphases and use different

practices to estimate and account for costs. As

such, there is no single accepted taxonomy of costs

across (or even within) programs, and we have not

attempted to reshape existing literature and data

to match a detailed unified framework. Insofar as

major health programs differ significantly in their

key cost drivers, it may be logical to retain different

taxonomies: when it comes to collecting data for

analysis, it is most important that country programs

capture costs comprehensively, and in a way that

enables researchers to compare them with those

of similar health programs in other countries. These

comparisons are particularly relevant for donors

funding programs in multiple countries, and also for

health officials who seek guidelines to plan programs.

Comparability across programs could benefit health

officials, but is less likely to yield immediately helpful

benchmarks unless significant adjustments are made

for the nature of the different health areas.

For the purpose of this work and to frame and

ground further discussion, we have created an

illustrative taxonomy of costs by activity and level,

designed to be broadly applicable across multiple

disease areas, provided in Figure 1.

Definitions and working taxonomy

Above the point of service delivery (ASD) activities

include procurement and supply chain activities,

in which critical commodities and equipment

are supplied to service providers, and demand

generation activities like mass media campaigns at

the district, regional, or national level, which seek to

maximize the numbers of patients who visit service

providers. Some ASD activities are systemic, such as

activities that support the laboratory system or health

information infrastructure, while other ASD activities

are programmatic and take place at district, regional,

national, or above-national level. Such programmatic

activities are designed to ensure the proper operation

of the program and the quality of front-line services,

and include program management, monitoring and

evaluation, supervision, and surveillance.

Several caveats are important to mention at the

outset. First, while some activities, such as procuring

antiretroviral drugs (ARV), are performed above

the point of service delivery, many of the activities

mentioned above also are performed by front-

line facilities or have clear corollaries at the facility

level. For example, to provide a vaccine to a child,

supply chain activities above the facility level are a

prerequisite, including cold chain maintenance and

properly storing the vaccines at all times, but these

activities must also occur at the facility level. The

facility must have proper storage equipment and

processes.

In this report, we examine the costs, impact,

and efficiency of ASD activities in HIV and other

programs. To do so, we provide an overall taxonomy

of ASD activities. Specifically, in this report, we focus

generally on the costs incurred by global health

programs away from facilities, as they are less well

accounted-for and are more opaque. As such, we

have not included in our definition of “above the

point of service delivery” activities that are performed

at the facility level even if they are conceptually

similar to activities that take place above the facility

Landscape Study of the Cost, Impact, and Efficiency of Above Service Delivery Activities in HIV and Other Global Health Programs 3

Additional dimensions could be added to this

taxonomy. It is common in costing studies/analyses

to break down costs into categories representing

specific types of resources, sometimes referred to

as ‘inputs’ and/or by categories representing specific

functions or activities. In some cases (such as the

EPIC studies), researchers distribute costs across a

two-dimensional input-activity matrix. For simplicity,

we have not included this ‘input’ line-item lens in this

taxonomy, but we will refer to specific line items later

in this report where relevant.

Finally, we did not make two additional distinctions in

this taxonomy that may be relevant to policy-makers.

The first is the distinction between recurrent costs

(i.e., costs that represent inputs that are ‘used up’ in a

given year—including employee time—and are likely

to recur in the next year) and capital or investment

costs, which typically reflect some investment that

provides benefits over multiple years. The second

is the distinction between ongoing costs (expected

to occur each year to maintain a current level of

service) and set-up costs (costs that are associated

with the introduction of a new program or with

the scaling up of an existing program, but are not

expected to be incurred after the set-up period).

These distinctions underline the importance of

the specific country’s and program’s context to

make comparisons: newly introduced programs,

or programs that are rebuilt in conflict-affected

countries, are likely to have quite different cost

structures from mature programs that operate in

stable environments.

Figure 1. Illustrative taxonomy of costs incurred in the delivery of health services

Below-facility level

Facility level District levelRegional

levelNational

levelAbove-

national level

Routine facility-based service delivery

Treatment

Prevention

Outreach service delivery

Treatment

Prevention

Procurement, collection, distribution and storage of medicines and commodities

Social mobilization and demand generation

Laboratory system

HMIS and record-keeping

Program management

Monitoring and evaluation

Supervision

Surveillance

Training

Core above-service delivery costs

4

Overarching themes: ASD costs

In this section, we introduce and discuss several

overarching themes that emerged regarding

ASD costs in health programs addressing HIV,

immunization, TB, malaria, nutrition, and family

planning.

Cost theme 1: Data on ASD costs are limited in availability and quality

Based on our review of published and grey literature,

and interviews with cost experts, we found little

high quality data on costs above the point of

service delivery. The vast majority of cost studies

that we reviewed concerned costs at the facility

level. Few accounted for any activities above facility

level, and almost none addressed ASD costs in a

comprehensive fashion. For illustration, in a review

of literature on the costs of antiretroviral therapy

(ART) and prevention of mother-to-child transmission

(PMTCT) interventions in LMICs, Galárraga et al.,

(2011) found 29 relevant articles that met their

inclusion criteria, only 3 of which mentioned

“programmatic” costs, and these did not provide

sufficient detail for analysis.2 We believe this may

reflect that some researchers perceive costs above

facility level to be fixed, or effort above facility level

to be only indirectly tied to facility-level activities.

Researchers also may find it too challenging to study

costs above the facility level. Studying costs at the

facility level may be more tractable analytically, and

can be the basis to effect meaningful change at the

facility level, but neglecting ASD costs has limited

the improvements that stakeholders, donors, and

implementing organizations can make to health

programs more broadly.

In cases in which ASD cost data are available, we

found that they may not have been gathered in

ways that enable us to draw comparisons between

programs with confidence. To assess comparability

between ASD costs in HIV programs in different

countries, for example, the relevant studies first

must similarly define whether a cost is incurred ASD

or at the point of service, and within the category

of ASD costs, the studies should define similar

categories of activities that either directly correspond

with each other or can be easily manipulated to

create comparable categories. To date, the lack of

standardization makes it challenging to compare

results from different studies, with the exception

of those studies (such as the EPIC studies in

immunization) that were prospectively designed to

use a consistent methodology across countries.

In addition, even when researchers made a good faith

effort to collect ASD cost data, we found that they

encountered methodological challenges in doing so.

ASD activities often involve resources that are shared

by more than one health program, e.g., staff members

in a district health office, medical warehouses to

store drugs and commodities, or laboratory systems

that service multiple diseases. Researchers often

have difficulty accurately teasing apart how much of

a shared resource is allocated to different programs

because managers within programs/health systems

do not typically account for this distinction in the

course of day-to-day operations. To focus primarily

on ASD costs, researchers could use methods like

time-motion or work sampling techniques to achieve

fairly rigorous estimates of the time allocations

across different tasks, but generally in studies, ASD

costs are just a small piece of a larger costing effort.

Accordingly, researchers may rely on self-reported

time allocations or on reports from managers that

describe how they think their staff split their time

across activities.

We did not find any studies in the literature review

for this report that provide comprehensive ASD

costing using the most rigorous (but time-intensive)

research techniques. We anticipate that more

precise estimates may be of interest to researchers

and donors: the former, because investigating the

impact of these activities might be confounded by

poor measurement of the activity itself; the latter, to

better understand the extent to which staff funded

by donor programs are providing additional benefits

to other disease programs or the primary healthcare

system more generally. However, given the high cost

of producing these estimates, the value to different

actors must be carefully considered—both in the

decision to commit resources for this type of study,

2 “We intended to analyze all relevant cost components, including program–level (above-facility-level) activities and expenses including managerial overheads, administration, monitoring and evaluation and training borne at district, province and national levels. Only three ART studies however provided such programmatic costs, without sufficient description of the relevant activities or cost items to analyze the determinants of these potentially significant contributions to overall delivery cost.” Galárraga, O., Wirtz, V. J., Figueroa-Lara, A., Santa-Ana-Tellez, Y., Coulibaly, I., Viisainen, K., ... & Korenromp, E. L. (2011). Unit costs for delivery of antiretroviral treatment and prevention of mother-to-child transmission of HIV. Pharmacoeconomics, 29(7), 579-599.

Landscape Study of the Cost, Impact, and Efficiency of Above Service Delivery Activities in HIV and Other Global Health Programs 5

and in how the studies are designed to maximize

usefulness to decision-makers.

Cost theme 2: Among the largest relevant datasets, taxonomies vary significantly, reflecting broader differences in health area and program priorities

As mentioned in previous sections, there is no single

unified taxonomy of program activities that can be

applied across all relevant health programs; and

even within the same program area, taxonomies

are not fully consistent with each other. The three

taxonomies that we find most relevant to this report

are the EPIC study cost taxonomy in immunization,

the HIV expenditure taxonomies in the National

AIDS Spending Assessments (NASAs), and the United

States President’s Emergency Plan for AIDS Relief

(PEPFAR) Expenditure Analyses (PEPFAR EAs).

The EPIC study provides the most useful example for

this work. The study tracks costs by activity type, level

at which the cost is incurred, and cost line item. The

comprehensive costs of the programs are captured

and allocated to the correct place in a three-

dimensional matrix, of which the two-dimensional

activity/level matrix is presented below in Figure 2.

The matrix in Figure 2 below reflects the particular

interests of immunization programs, e.g., creating

separate categories for the related activities, “Cold

chain maintenance” and “Vaccine collection,

distribution, storage,” and, given the nature of service,

does not include laboratory costs as a category.

Despite the high value of these studies for elucidating

ASD costs, it is important to note that in some cases

these cost estimates were determined by higher

level staff providing their opinions on how staff

time and other resources were used, rather than by

researchers collecting primary data directly. As with

many cost studies, these studies are intended as one-

off snapshots of the costs, rather than as multi-year

efforts to examine trends in costs over time.

The taxonomies used in the two major sources of

data on HIV spending, the NASAs and PEPFAR EAs,

differ significantly from the EPIC study.

The NASAs track a detailed set of activities, but

unlike in the EPIC study, they do not report on the

multiple levels at which these activities take place.

Conceptually, the NASAs aggregate expenditures into

the following eight categories:

1. Prevention

2. Care and treatment

3. Orphans and vulnerable children

4. Program management and administration

5. Human resources

6. Social protections and social services

7. Enabling environment

8. Research

Within each of these large categories, a series of

more detailed activities is tracked. Although the

NASAs do not break out the level at which the

expenditures occur, the activity definitions do allow

some separation of service delivery and ASD. For

Figure 2. EPIC costs categories by activity and level

Ro

uti

ne

fac

ility

-bas

ed

se

rvic

e d

eliv

ery

Ou

tre

ach

se

rvic

e

de

live

ry

Soci

al m

ob

iliza

tio

n a

nd

ad

voca

cy

Trai

nin

g

Co

ld c

hai

n m

ain

ten

ance

Vac

cin

e c

olle

ctio

n,

dis

trib

uti

on

, sto

rag

e

Re

cord

-ke

ep

ing

an

d

HM

IS

Sup

erv

isio

n

Surv

eill

ance

Pro

gra

m m

anag

em

en

t

Facility level

District level

Regional level

National/Central level

6

example, the UNAIDS technical guidance describing

category 4 above explains that “Programme

expenditures are defined as expenses incurred at

administrative levels outside the point of health care delivery” (emphasis added).

The PEPFAR Expenditure Analyses track spending by

PEPFAR-defined program area and by cost category,

as shown in Figure 3, PEPFAR taxonomy of program

areas and cost categories. This taxonomy is not fully

intuitive, as Strategic Information and Health System

Strengthening are hybrid classifications that serve as

both program area and cost category.

In comparing the HIV taxonomies to the EPIC study,

we found that the HIV taxonomies emphasize

understanding exactly what types of distinct

treatment and prevention interventions programs

are carrying out. Researchers have focused less on

understanding the ‘level’ at which the expenditures

occur; accordingly, these data have not been

included in publicly available datasets.

Cost theme 3: Where high level ASD costs or expenditures are estimated for a program area, very high variation is observed across countries, and is unlikely to reflect true variation in delivery models and/or efficiency within the program area

Overall, in cases in which we found data sources

with which to make high level estimates of ASD

costs/expenditures at a country-program level,

estimates within the same health program varied very

widely across countries, to an extent that we do not

believe can be plausibly explained by differences in

delivery model or efficiency between countries. This

is less of a concern in the EPIC studies, for which the

range is more compressed, but is a larger concern

in expenditure analyses and in exercises estimating

projected financial needs.

In addition to the EPIC, NASA, and PEPFAR

EA reports, which deal with observed costs or

expenditures, large exercises in family planning and

nutrition attempt to estimate the projected cost of

fulfilling health programs’ goals in those countries.

While these are not ‘actual’ costs/expenditures, they

Figure 3. PEPFAR taxonomy of program areas and cost categories

Recurrent InvestmentStrategic

InformationHealth System Strengthening

Program Management

Facility-based care, treatment and support (FBCTS)

Community-based care, treatment, and support (CBCTS)

Prevention of mother-to-child transmission (PMTCT)

HIV testing and counseling (HTC)

Post-exposure prophylaxis (PEP)

Blood safety (BS)

Laboratory (LAB)

Orphans and vulnerable children (OVC)

Sexual and other risk prevention-general population (SORP-GP)

Sexual and other risk prevention-key populations (SORP-KPs)

Voluntary medical male circumcision (VMMC)

Infection control (IC)

Strategic Information (SI)

Surveillance

Health System Strengthening (HSS)

Landscape Study of the Cost, Impact, and Efficiency of Above Service Delivery Activities in HIV and Other Global Health Programs 7

may help illuminate the balance between ASD and

point of service costs in these health areas.

As can be seen in Figure 4 below, which compares

the percentage of ‘cost’ allocated to ASD activities,3

there is significant variation across countries within

health areas. Variation between health areas can

also be observed in this chart, but the methodology

for estimating ASD share differs substantially across

health areas, and can only be considered moderately

consistent within a health area.

As noted above, this high variation within

programmatic areas may slightly call into question

the accuracy of data on ASD activities, or, at a

minimum, the consistency/standardization of data

collection. We recommend that cross-program

comparisons be made with extreme caution with

these data; nevertheless, it is noteworthy that the

EPIC data appear to have the smallest spread, which

is consistent with our finding that these are the most

carefully gathered data on ASD costs.

The PEPFAR EAs have a smaller spread than the

NASAs, and a similar spread to the TB data. The

nutrition and family planning data have wide spreads,

which may be understandable given the more

hypothetical nature of those exercises. The malaria

data have the widest variation, some of which

may reflect countries in different stages of malaria

elimination deploying a different set of activities.

3 With the exception of the EPIC studies, none of the other sources are strictly cost estimates, and the organizational level at which spending occurs is not explicitly reported. As such, the data summary here is based on some high level assumptions we applied to the data as reported. To calculate %ASD share, we applied the following rules:

• NASAs: %ASD = (Program management + HR)/(Total costs – OVC costs – Enabling Environment – Social Protection – Research)• PEPFAR EAs: %ASD = (Health system strengthening + program management + strategic information)/(Total costs – OVC costs)• Family planning (FP2020 Costed Implementation Plans): %ASD = (Demand creation + Contraceptive security + Policy and enabling environment +

Financing + Stewardship, management, accountability, monitoring, and coordination)/(Total costs)• Nutrition (SUN Costed Implementation Plans): %ASD = Governance costs/(Governance costs + nutrition-specific intervention costs)• WHO TB: %ASD = (Laboratory operations + Operational research + National-level staff)/(Total expenditure)• WHO Malaria: %ASD = (Monitoring and evaluation + Human resources & technical assistance + Management and other costs)/(Total expenditure)• EPIC: %ASD = Percent of costs incurred above facility level (with cost of commodities defined to be part of facility-level service delivery regardless of

where they were recorded)

We discuss these data sources further in the relevant sections of this report.

Figure 4. High variation in ASD estimates across countries within programs

Select data sources with nominally consistent methodology within programs across countries for estimating costs or expenditures

0

10

20

30

40

50

60

70

80

90

% A

SD

Ethiopia

Costing approach

Costs included

Immunization:EPIC costing

studies

Full economiccosting

All publicsources

HIV:UNAIDS NASA

reports

Kenya

Malawi

Nigeria

Swaziland

Zambia

Expendituretracking

All publicsources

HIV:PEPFAR EAs

Swaziland

ZambiaNigeria

Kenya

EthiopiaMalawi

Expendituretracking

PEPFARfunding

TB:WHO TBdatabase

Malawi

ZambiaEthiopia

SwazilandNigeria

Kenya

Expendituretracking

All publicsources

Malaria:WHO malaria

database

Zambia

Ethiopia

Swaziland

NigeriaKenya

Expendituretracking

All publicsources

Family planning:Costed Impl.

Plans

Malawi

Niger

Uganda Togo

Burkina Faso Guinea

Benin Zambia

Mali

Nigeria

Mauritania

Projected financialneed

All publicsources

Nutrition:Scaling Up

Nutrition plans

Projected financialneed

All publicsources

Mozambique

Kenya

Tanzania

Senegal

Madagascar

Burkina Faso

Sierra Leone

Rwanda

Benin

Malawi

Niger

Gambia

Uganda

Ghana

Mali

Niger

Uganda

ZambiaHonduras

Ghana

Moldova Benin

8

Overarching themes: ASD impact/efficiency

To evaluate whether current ASD spending is optimal,

we considered the impact that ASD activities have

on service delivery and outcomes, and determined

whether there are opportunities for programs to

achieve greater impact or become more efficient.

While some limited data are available on ASD costs,

we found that data on the impact of ASD activities

are almost non-existent. Given the nature (and

location) of many ASD activities, few researchers

have attempted to draw a direct causal link between

ASD costs and impact at the point of service delivery,

and none have sought to do so in a comprehensive

fashion across multiple ASD activities.

One ASD activity that could have an impact of

service delivery outcomes is the supervision of front

line facilities and workers. For example, more regular

supervision could reduce absenteeism among health

workers and improve the quality of treatment that

they provide. While we did not find any completed

studies on this topic in global health in the literature

review, there are ongoing research efforts on HIV

prevention in Mexico, and on immunization delivery

in Ethiopia, that may shed light on this topic.4

In some cases it may be possible to make significant

gains in program performance and efficiency without

linking ASD activities directly to service delivery. The

performance and efficiency of some other ASD

activities have been evaluated by program staff and

researchers using metrics for intermediate outcomes

rather than for direct health outcomes or quality of

service received by patients. For example, a supply

chain’s functionality can be evaluated by using

a battery of intermediate outcomes such as the

number of stockouts of key commodities, where

basic logic would hold that service delivery is harmed

by stockouts even if this outcome for end users is

not being measured. Similarly, procurement systems

can be judged by metrics such as the difference

between average price paid by the health system for

a commodity over a period of time and a reference

international price over the same period. Although

the systems may not impact service delivery,

procuring goods at a lower cost can benefit the

system as a whole, assuming that the money saved

in procurement is redeployed to other worthwhile

activities. In both examples, if managers were to

focus exclusively on a single indicator or a narrow

set of indicators, this could have unintended negative

consequences, particularly if staff were evaluated or

had significant incentives tied to these indicators; as

such, caution is needed when using intermediate

metrics rather than ultimate health outcomes to

evaluate policy options.5 Nonetheless, thanks to

the existence of useful intermediate indicators,

supply chain and procurement are two areas of ASD

activity in which greater strides have been made in

improving performance and efficiency (described in

more detail in a later chapter).

Based on the interviews that we conducted for this

report, experts and stakeholders agree that (a) ASD

activities are critical elements of a health program and

prerequisites of successful service delivery, (b) ASD

spending likely is not currently allocated in an optimal

way, but (c) policy makers do not have sufficient data

to identify the areas in which additional investments in

ASD activities will have the most impact and increase

efficiency. As donors' budgets tighten, and it becomes

increasingly important to justify the impact on health

outcomes from any given investment, understanding

the link between ASD activities, costs, and impacts is

important for the future of national and global health

programs.

4 See the HIV and Immunization chapters for further information.5 For example, it is possible to minimize stockouts by dramatically increasing inventory, but this could easily lead to significant increases in wastage and

costs that might outweigh the benefits of the reduced stockouts. Similarly, focusing narrowly on minimizing the difference between procurement cost of drugs and an international reference price for drugs could lead to suboptimal decisions (e.g., buying in inappropriate quantities).

Landscape Study of the Cost, Impact, and Efficiency of Above Service Delivery Activities in HIV and Other Global Health Programs 9

Above Service Delivery Costs in HIV Programs

consultants who completed the NASAs included

all PEPFAR-financed staff costs in the Program

Management category, regardless of what these

staff were being paid to do and where. Given these

examples, even when NASAs’ data may provide a good

estimate of the total amount of a program’s spending,

any granular analysis at the activity level must be

undertaken with caution. On an additional practical

note, to review the detailed NASAs’ data requires

manual extraction from individual country reports and

databases, sometimes with a limited number of years,

and with some differences in reporting sub-categories

from year to year. The database that is publicly

available on the UNAIDS website provides the more

aggregated country spending by two dimensions:

source of funding and the activity.

The PEPFAR EAs, available on the PEPFAR dashboard,

have several key strengths. Collectively, they

comprise the most easily accessible large dataset

on PEPFAR HIV spending, including both up-to-

date data and some historic data for each country.

By providing the data by activity and cost line

item, these data also provide detailed information

enabling researchers to understand the factors

driving the costs within a particular activity. The

greatest limitation that researchers encounter in

using the PEPFAR data to analyze levels and variation

in spending above the point of service delivery is

that the data were not intended to include non-

PEPFAR spending. Because PEPFAR may fund

different activities in different countries depending

both on what the country wishes to fund from

national sources and on the other support that the

country receives from international actors, patterns

of PEPFAR spending within and across countries

may be difficult to interpret. Additionally, the publicly

available PEPFAR EA datasets do not include more

granular breakdowns of high level activity categories

such as Health Systems Strengthening, Program

Management, and Strategic Information, nor any

geographic breakdown within countries, currently.

Data that are currently publicly available from the

National Health Accounts (NHAs) produced by the

System of Health Accounts (SHA) provide the least

What is the current level of publicly available data for HIV-related ASD costs?

We found very little data on the comprehensive

costs of HIV programs that include both service

level costs and costs incurred above the point of

service delivery. In the absence of comprehensive

cost data, expenditure data are the most helpful to

understand activities and costs above the point of

service delivery. Large, publicly available data sources

that shed some light on HIV program expenditures

include the National AIDS Spending Assessments

(NASAs), the United States President’s Emergency

Plan for AIDS Relief (PEPFAR) Expenditure Analyses

(PEPFAR EAs), and System of Health Accounts (SHA).

While these publicly available data sources typically

do not provide a lot of granularity on the location

of spending (at point of service vs. above point of

service delivery), they enabled us to perform some

high-level analyses of activities, from which we can

draw insights for ASD costs.

The different datasets have different strengths

and weaknesses to understand HIV program

expenditures above the point of service delivery.

The National AIDS Spending Assessments (NASAs)

provide a comprehensive assessment of spending

on HIV/AIDS from all funding sources, and provide a

reasonably detailed breakdown of their large “Program

management” category into a series of constituent

activities (e.g., planning, coordination, and program

management; administration and transaction costs

in managing and disbursing funds; monitoring and

evaluation). In principle, the NASAs’ methodology

is meant to disaggregate the spending by line item

costs within each activity (such as labor costs vs.

commodities vs. travel/transportation); however, this

has not always been executed consistently with the

NASAs’ guidelines. For example, while the “Program

Management” costs are defined to be non-site costs,

informants suggest that some NASAs’ reports include

site-level program management and administration

in this cost category, leading to inflated numbers.

Similarly, in a small number of NASAs’ reports, the

10

granular breakdown of spending of the three large

data sources. To perform a detailed analysis of recent

years of spending, we had to manually extract data

from country level reports. The publicly available

global database on the WHO website provides the

compiled data from National Health Accounts. Based

on our preliminary analysis of these data using the

latest reports from select countries of interest from

sub-Saharan Africa, we found a number of data

anomalies, and variations within and across countries

that we thought could not result from actual

variation in program costs.6 Because we consider

these data to be suspect, we do not present results

from this source in cases in which better data are

available. We understand that there is ongoing work

to refine and improve the methodology and data

quality; when this process is complete, researchers

may find it useful to reanalyze the date to determine

service costs across programs and countries.

In addition to the large public datasets, we found a

small number of academic papers that also provided

high level estimates of costs incurred above the

point of service delivery in HIV programs. On the

treatment side, Marseille et al. (2012) examined the

comprehensive costs of scaling up ART provision in

Zambia, incorporating both costs incurred at facility

level and “off-site” costs (primarily costs incurred at

the national level in Lusaka).

On the prevention side,

Chandrashekar et al. (2014)

examined the costs of scaling

up the Avahan project,

distinguishing between

service-level costs of local

NGOs, and above-service level

costs incurred both by state-

level implementing partners

and by the national program

level office.

We did not find any other

papers that comprehensively

captured costs incurred by

HIV programs above the point

of service delivery.

What are ASD activities and what share of program spending do they comprise?In this section, we focus on ASD activities and costs

within Kenya, Malawi, Ethiopia, Zambia, Nigeria and

Swaziland, and present data from both the NASAs

and PEPFAR EAs before performing a more detailed

analysis of PEPFAR EA data from South Africa.

ASD activities in the publicly available NASAs

As described in the previous section, we conducted

granular analysis of NASAs’ data, including on

activity level expenditures, recognizing the source’s

limitations. Site-level activity categories are clustered

under Treatment and Prevention overarching

categories, while the overarching categories

pertaining to above-service delivery spending are the

Program Management category and the (typically

much smaller) HR capacity building category.7

At a high level, as seen in Figure 5, the NASAs

of Kenya, Malawi, Ethiopia, Zambia, Nigeria and

6 See Appendix 3 for further details.7 For the purposes of our landscape assessment of ASD activities, costs, and expenditures, we exclude spending for Orphans and Vulnerable Children

(OVC), social protection, enabling environment, and research activities in our assessment of NASAs’ data; these activities, though supportive of service delivery, are not within the scope of this project and, thus, are not classified as ASD activities for the purposes of our analysis.

Figure 5. Select NASAs: Percentage of spending above the point of service delivery

Kenya(2011)

Malawi(2008/09)

Ethiopia(2011)

Zambia(2012)

Nigeria(2011/12)

Swaziland(2007/10)

30%

40% 41%

51%

59%

12%

Source: NASA, UNAIDS

Landscape Study of the Cost, Impact, and Efficiency of Above Service Delivery Activities in HIV and Other Global Health Programs 11

Swaziland vary widely in the percentages of spending

above the point of service delivery, ranging from 12

percent in Kenya (year) to 59 percent in Swaziland

(year), with the four other countries grouped

between 30 percent and 51 percent (the NASA

reports relate to various years). The range expressed

here is surprisingly wide. As mentioned previously,

there may be an over-estimate of program

management expenditures if site-level management

has been included in the program-level management

expenses. It is also possible that some of the variance

in these percentages reflects differing impacts of ART

on the NASAs’ expenditure calculations: in countries

reporting low ART spending, the share of treatment

costs at facility level will be lower and costs above

the point of service will be higher. Among the

countries examined here, Swaziland’s NASAs’ reports

only 12 percent of total costs going towards ART

compared to an average of more than 20 percent in

the other countries reporting this line item separately.

Looking at the more granular data, the Program

Management (PM) category is defined as expenses

incurred at administrative levels outside the point

of healthcare delivery and includes a range of

management/supervision activities, systems

spending, and infrastructure investments. The

following categories constitute the largest

expenditures in these countries:

• Planning, coordination, and program management

• Administration and transaction costs in managing

and disbursing funds

• Upgrading and construction of infrastructure8

• Monitoring and evaluation (M&E)

The PM “Other” category includes smaller amounts

spent on operations research, serological-

surveillance (serosurveillance), HIV drug-resistance

surveillance, drug supply systems, information

technology, patient tracking, and PM not

disaggregated.

As seen in Figure 6 below, the largest component of

the broader Program Management category is the

Planning, coordination and program management

category, ranging from 34 percent to 59 percent,

with Administration and transaction costs in

managing and disbursing funds also accounting for a

Figure 6. NASA Program Management costs disaggregated (percentage)

Ethiopia Malawi Zambia Nigeria Swaziland

Other

Monitoring and evaluation

Upgrading andconstructionof infrastructure

Administration andtransaction costs in managing and disbursing funds

Planning coordinationand program management

14%

15%

19%

18%

34%

16%

13%

5%

30%

36%

15%

5%3%

29%

48%

13%

2%4%

27%

54%

5%3%

20%

13%

59%

Source: NASA, UNAIDS

8 It is unclear within the NASAs’ data how much of this category would include infrastructure expenditures at point of service. However, this category is only a significant proportion of program management in Ethiopia and Swaziland, and most of these infrastructure expenditures in recent NASAs would be expected to be above-facility investments, e.g., in laboratory upgrades.

12

substantial proportion in each country (ranging from

13 percent to 30 percent). Monitoring and evaluation,

a category that might be expected to have an impact

on quality of service provision, varies significantly

from Ethiopia at the top (15 percent of Program

Management) to Swaziland at the bottom (3 percent

of Program Management). A common benchmark

figure cited for M&E expenditure is three to five

percent of total program budget,9 which Ethiopia and

Malawi would be meeting while the other countries

fall short.

ASD activities in the publicly available PEPFAR EAs

In its technical guidance on expenditure analyses,

PEPFAR distinguishes between “site-level support”

recurrent and investment expenditures, and “above

site-level”/“system-level” support. The above site-

level costs are divided into three major categories:

Program Management (PM); Strategic Information

(SI); and Health System Strengthening (HSS). While

we treat these categories as the best proxy for

spending above the point of service delivery, it is

worth noting that the PEPFAR categories appear to

be designed with greater focus on the activity rather

than the precise location of spending; as such, it

is possible that some site-level expenditures (e.g.,

general facility-level management or facility-level

IT systems) might be reported in the system-level

support categories.

As seen in Figure 7 below, approximately 45 percent

of PEPFAR’s expenditures in the focus countries was

above site-level in 2014, ranging from approximately

34 percent in Kenya to approximately 67 percent

in Swaziland. Less money typically is spent on SI

(approximately 5 percent of total expenditures),

with HSS and PM roughly equal in this sample of

countries. As described in an earlier section of this

report, the composition of PEPFAR spending likely

varies significantly across countries based on the

particular need of the country, so a wide variation

here is not unexpected. In particular, variation in

the needs that countries have for support to meet

the costs of ARV drugs has a significant impact on

the proportion of PEPFAR funding spent on point-

of-service rather than above-service level. In Kenya,

Nigeria, and Zambia, 11 to 15 percent of PEPFAR’s

9 See, e.g., Blue, R., Clapp-Wincek, C., & Benner, H. (2009, May). Beyond success stories: Monitoring & evaluation for foreign assistance results. Retrieved from https://www.usaid.gov/sites/default/files/documents/1868/060909_beyond_success_stories.pdf.

Figure 7. PEPFAR expenditures by high level cost category (percentage)

Investment

Recurrent

SI

PM

HSS

Grand total Kenya Zambia Malawi EthiopiaNigeria Swaziland

9 8 9 10 12 11 10

2115

2424

2523

21

6 6

9

4758 48 45

31 3124

18 16 15 1726 29

37

4

3

4 4

Source: PEPFAR, 2014

Landscape Study of the Cost, Impact, and Efficiency of Above Service Delivery Activities in HIV and Other Global Health Programs 13

total expenditures was accounted for by ARV

drugs, while in Malawi, Ethiopia, and Swaziland, the

proportion was less than 0.1 percent.

PEPFAR’s publicly available expenditure analyses do

not break down more granular activities within PM,

SI, and HSS. However, in collecting the data, PEPFAR

asks the implementing partners to estimate how

much of this above-site spending directly benefits

each of the specific program areas defined by

PEPFAR.10 All of the above-site spending is allocated

to one of the program areas, with the majority of this

spending typically estimated to directly benefit either

the facility based care and treatment program, or the

PMTCT program.

Further breakdown of ASD activities in the South Africa PEPFAR EA

In addition to the high level data that are publicly

available through the PEPFAR dashboard, we have

worked with more granular data in the context

of South Africa, which provide a more detailed

breakdown of spending within HSS.

As shown in Figure 8 below, South Africa’s largest

expenditures from 2012 to 2014 were on HSS,

and specifically for institutional and organizational

development of government institutions (425

million ZAR, approximately 45 million USD), and the

development of health information systems (349

million ZAR, approximately 37 million USD).

10 The full list of program areas is: Facility-based Care, Treatment and Support; Community-based care, treatment, and support; Prevention of mother-to-child transmission; HIV testing and counseling; Post-exposure prophylaxis; Blood safety; Laboratory; Orphans and vulnerable children; Sexual and other risk prevention-general population; Sexual and other risk prevention-key populations; Voluntary medical male circumcision; Infection control; Strategic Information; Surveillance; Health System Strengthening.

Figure 8. PEPFAR HSS expenditures in South Africa, 2012 to 2014 (ZAR millions), by detailed category

2,028

103

303

74

134

170

106

145

425

349

96

89

35

Governance: General Policy and other governance

Governance: Technical-area specific guidelines, tools and policy

HR: Curriculum development

HR: HR management and retention

HR: Pre-service training

HR: Training of trainers

Institutional and Org. Development: Civil society and non-governmental organizations

Institutional and Org. Development: Government institutions

Systems Development: Health information systems

Systems Development: Laboratory strengthening

Systems Development: Supply chain systems

Finance

Source: PEPFAR

14

Figure 9, PEPFAR HSS spending in South Africa, 2012

to 2014 (ZAR millions), by category and geographical

location, above, illustrates that the South Africa EA

data enable analysis by geography. We found that

the majority of PEPFAR’s HSS spending in South

Africa occurs at the provincial or sub-provincial level,

approximately 61 percent, and roughly 34 percent

is spent at the national level, and approximately 5

percent is spent above the national level.

Unfortunately, public data and analysis of South

Africa’s national budget and expenditures do

not categorize health system strengthening in a

similar way to PEPFAR’s data. Therefore, we were

unable to compare PEPFAR’s spending to South

Africa’s national public spending. To place this

HSS breakdown in context, PEPFAR’s total HIV

expenditures in 2013 to 2014 were roughly a quarter

of the size of South Africa’s own government

expenditures on HIV.11 For the two-year period from

2013 to 2014, HSS represented roughly a fifth of

PEPFAR’s funding to South Africa.12

ASD costs in the academic literature

In addition to what we gleaned from NASAs and

PEPFAR EAs, we found two relevant academic

studies that provided insights into ASD costs for HIV.

Marseille et al. (2012) studied the costs of scaling-up

ART in Zambia, and estimated that roughly a third

of the cost was in “off-site” activities above the point

of service delivery. The division of costs was by cost

category rather than by activity, so there is some

ambiguity in what the off-site costs were designed to

do, but 18 percent of the total cost of the program

supported off-site personnel in Lusaka (of which

6 percent represented salaries for expatriates, and

12 percent for Zambian nationals), and a further

15 percent was incurred in procuring goods and

services in Lusaka.

Chandrashekar et al. (2014) studied the Avahan

prevention program in India, and found costs above

the point of service delivery to be around 65 percent

Figure 9. PEPFAR HSS spending in South Africa, 2012 to 2014 (ZAR millions), by category and geographical location

Governance

HR

Institutional andorganizational development

Systems development

Finance

Total by geography

Above-National

National

17

8

9

16

45

158

153

189

178

94 694

15

Provincialand below

230

323

372

286

1,240

29

Source: PEPFAR

11 Guthrie et al. 2015. South African Expenditure on HIV and TB in 2011/12-2013/14. Investment Case.12 PEPFAR funding data extracted from PEPFAR Dashboards, available at https://data.pepfar.net/.

Landscape Study of the Cost, Impact, and Efficiency of Above Service Delivery Activities in HIV and Other Global Health Programs 15

of total costs, of which approximately 35 percent

were costs incurred at the national program level,

and approximately 30 percent were costs incurred

by the state-level lead implementing partners. As

such, only 35 percent of costs directly supported the

NGOs working to deliver the prevention program. By

comparison with the full programs discussed in NASAs

and PEPFAR, this appears to be a high proportion of

above service delivery spending, but there are a few

differentiating factors here. As a prevention program

rather than a treatment program, the program did not

pay for high cost ARVs, which reduced the relative

cost of service delivery versus above-service delivery.

In addition, the rapid scale-up nature of the program,

and extensive capacity-building required, may have

both contributed to higher above-service costs in the

early years of the program.

Additional views on ASD costs

To gain an additional view on ASD costs at a global

level, we talked with a source at the Clinton Health

Access Initiative (CHAI) who summarized a high-

level triangulation exercise which estimated that

approximately 40 percent of global HIV spending

can be attributed to ARVs and other direct treatment

and prevention activity costs at facility level. The

remaining 60 percent of global spending is spent

on activities above the facility level. Based on what

donors and implementing partners have shared with

CHAI, roughly $1.50 is spent above the facility level

for every dollar spent at facility level. The source

also reported that based on CHAI’s experience, ARV

drugs make up roughly 50 percent of costs incurred

at facility level, and spending on ARVs globally is

estimated to be $1.7 billion, out of a total of around

$10 billion spent in HIV prevention and treatment

worldwide. Building up from this number implies

a facility/above facility split of approximately 35

percent to 65 percent.

What are current, ongoing, or planned efforts to improve the knowledge base of HIV ASD costs, impact, and efficiency?

Among the health areas that we reviewed in this

report, HIV is an area in which increased efforts are

underway to better understand the costs and the

impact and efficiency of ASD activities.

As PEPFAR continues to maximize the impact of

dollars that it spends on the HIV response, its country

teams are beginning to work more closely with

countries to demonstrate the impact of strategic

investments in ASD activities, either directly on

service provision or on upstream measures that

demonstrably impact service provision. PEPFAR is

performing detailed reviews of its expenditures in

countries, scrutinizing whether ASD investments

are made at appropriate levels and in the right

geographies, and recommending reallocations

where appropriate. Working with countries to

understand how ASD activities lead to improved

coverage, performance, efficiency, and health

outcomes will be critical for PEPFAR’s optimization of

funding allocations.

To better understand the amount that countries

should request for investments in broader health

systems rather than in narrow program activities,

the Global Fund has been in discussions with WHO

regarding assessments of the efficiency of health

systems, with a view to performing systematic

evaluations across countries. In addition, the Global

Fund and WHO are jointly pursuing health facility

assessments; while these do not directly capture ASD

costs and activities, they do capture data on whether

facilities have the supplies and providers that they

need to deliver standard service. These data in turn

may shed light on whether facilities are receiving the

necessary support from activities and services above

facility level.

As part of its ongoing efforts in this space, UNAIDS

continues to develop its resource tracking and

analysis. Through this process, UNAIDS hopes to

clearly align inputs (including above the facility

level) with outputs and eventually outcomes to

better assess the impact and efficiency of activities.

16

In discussions with countries regarding their current

national programs, UNAIDS already distinguishes

between activities that have been shown to have a

strong impact on health outcomes and those for

which the evidence is inconclusive. Assessments

of this type will be enhanced if data provide a more

complete picture of how ASD activities interact with

service delivery activities and health outcomes.

At a more specific research level, the ongoing

Optimizing the Response in Prevention: HIV

Efficiency in Africa (ORPHEA) study will provide

interesting insights into the impact of one ASD

activity: among its various research questions

concerning site-level service delivery, the study

will track the number of supervisory visits each

facility receives from district-level supervisors, and,

therefore, will enable researchers to examine the

relationship between supervision and service delivery.

What are the key gaps in the HIV ASD knowledge base?

Overall, researchers need higher quality descriptive

data on ASD costs to identify areas that warrant

additional investment or should be targets for

improvements in efficiency. As described in this

chapter, we have found that the best data available

to understand ASD activities comes from analyses

of large expenditures whose primary goals are to

track how much money is spent on HIV activities.

Given the purpose of these analyses, and the fact

that a number of major ASD categories may involve

cross-program activities and resources that are

difficult to allocate, the estimates from expenditure

tracking are unlikely to provide a precise estimate of

true costs. Comprehensive efforts to determine costs

(ideally, that estimate ASD costs accurately) would

13 %ASD is defined here as (Program management + HR)/(Total costs – OVC costs – Enabling Environment – Social Protection – Research)

Figure 10. Percent of spending above the point of service delivery, reported in selected NASAs against 12-month ART retention rates

12%

Kenya(2011)

Malawi(2008/09)

Ethiopia(2011)

Zambia(2012)

Nigeria(2011/12)

Swaziland(2007/10)

30%

40% 41%

51%

59%

75%

80%

73%

80%78%

87%

0

10

20

30

40

50

60

70

80

90

100 12 month retention(year of NASA data)

% ASD (NASA)

Source: UNAIDS

Landscape Study of the Cost, Impact, and Efficiency of Above Service Delivery Activities in HIV and Other Global Health Programs 17

significantly enhance current levels of understanding

of ASD activities. Moreover, these data, if collected in

a standardized fashion across multiple countries (or

across multiple geographies within a country), would

enable researchers to make useful comparisons that

could form the basis for future resource allocation.

To fully optimize resource allocation to (and across)

ASD activities, researchers and stakeholders must

understand the link between ASD activities and

impact on service delivery. Collectively, our current

level of understanding is limited. As a first step,

researchers should collect high quality ASD cost data

and map it against HIV prevention and treatment

performance indicators to enable researchers to

provide helpful insights and generate hypotheses

as to the impact and efficiency of ASD activities.

An illustrative example of the type of analysis that

could be conducted and refined is provided above in

Figure 10 on the preceding page. The figure displays

NASAs’ reported HIV spending above the point of

service delivery against one possible performance

indicator: the percentage of adults and children

with HIV known to be on treatment 12 months

after initiation of antiretroviral therapy.13 This simple

(and non-causal) analysis appears to suggest a

moderate relationship between percentage of ASD

spending and 12-month ART retention rates. With

higher quality, robust and disaggregated cost data,

these types of simple analyses could yield useful

observations and generate important hypotheses

regarding the relationship between (levels and

composition of) ASD spending and the performance

of HIV programs.

In the longer term, researchers should aim to

generate plausible causal analyses between ASD

activity and impact, to provide the most robust

evidence for policy-makers to make decisions in this

space. With high quality data collection, longitudinal

data exploiting natural experiments could provide

some insights into how ASD activities have an impact

on service delivery; in the event of planned scale-

ups or drawing down of donor support in particular

HIV programs, a randomized staggered design

could provide an alternative strategy to enhance

understanding.

Several more specific questions may also be

of particular relevance to understanding ASD

expenditures in HIV programs, and the implications

of these expenditures for the future. First it is

important to understand potential economies

of scale and scope in ASD activities, and the

implications for how ASD costs might be expected to

change as countries attempt to broaden coverage to

more difficult-to-reach populations, and for how they

may change as HIV treatment increasingly moves

towards a chronic care model. Second, given the

necessity of optimizing the impact of dollars that are

spent on HIV response, it is important to understand

whether and the extent to which investments made

through national and donor HIV programs have a

broader impact or benefit on other health programs

or the health system at large. Investments in systems

such as laboratories or blood safety are absolutely

critical for an effective HIV response; intuitively, these

investments have clear benefits in fighting other

communicable diseases that should be factored into

any societal view of the impact of these investments.

In cases in which only a certain level of investment

can be justified by reference to HIV outcomes

alone, additional investment (whether financed from

HIV programs, or from other sources) may well be

justified in light of other benefits.

Finally, given the rapid expansion of HIV programs

throughout the last decade and the increasing

amounts of donor money to support them,

HIV programs are at risk of high organizational

overhead, above-national costs, and over-utilization

of expatriate labor and international technical

assistance. (Further discussion of this topic is

contained in the Aid Architecture chapter of this

report.) The urgency of the global epidemic and

the imperative for rapid treatment and prevention

expansion may have justifiably led to a greater

emphasis on impact than on efficiency during the

scale-up period, but as countries and donors move

towards more mature programs, the role of high-

cost support activities should be evaluated and the

spending levels should be made transparent.

18

Above Service Delivery Costs in Other Programs

Immunization

What is the current level of publicly available data for immunization-related ASD costs?

The most current publicly available data on the

economic costs of national immunization programs in

low- and middle-income countries was generated by

the six-country Expanded Program on Immunization

Costing and Financing (EPIC) study, supported by

the Bill & Melinda Gates Foundation (BMGF). The first

phase of the EPIC study was conducted between 2012

and 2014, using a common methodological approach

to estimate costs for program year 2011 in Benin,

Ghana, Honduras, Moldova, Uganda, and Zambia. Data

from this study—and associated materials, including

data documentation, data collection instruments,

publications, and presentations of analytical results—

reside in a DataVerse data repository.14 The EPIC data

repository is one of the few sources that provides a

sense of ASD costs across immunization activities and

operational levels. We discuss the results from the EPIC

study’s first phase of data collection in greater detail in

the following section.

Additional cost data are emerging from the PAHO

ProVac Initiative, which is currently providing technical

support to immunization cost studies in Latin American

countries. The methodology of these studies is very

similar to the methodology used in the EPIC study

(Brenzel, 2014). Current studies in progress include

those in Brazil (data collection complete) and Costa

Rica (data anticipated 2016). These studies use the

ProVac “CostVac” Toolkit, which includes Excel tools

and guidance documents to design and implement

the data collection and analysis. CostVac’s paper-based

survey instruments capture ASD activities, such as

supply chain management and administrative/central

level activities. Researchers enter these data into the

tool’s spreadsheets, which define resource use at

three different levels: the central level, intermediate

administrative levels (department and municipality),

and the level of service delivery. Costs at each level

are disaggregated across six major cost categories

using the CostVac tool: “vaccines and supplies (only

captured at the central level), personnel, cold chain,

vehicles, buildings, and other costs (only captured at

the central level)” (Castañeda-Orjuela et al., 2013).

The WHO Immunization, Vaccines and Biologicals

Department (WHO IVB), in conjunction with UNICEF,

also has developed a tool and guidelines for

comprehensive multiyear strategic planning (cMYP)

for national immunization programs.15 More than

75 countries have used the cMYP tool for budget

planning. To receive financing from Gavi, applicants

must develop strategic plans; many countries have

used the cMYP tool for this purpose. Cost related data

in the cMYP are primarily financial data used to plan

national immunization program budgets for a forward-

looking five-year time horizon. The cMYP includes

special modules to assist countries in planning for

new vaccine introductions. The WHO IVB stores data

from all cMYP analyses; researchers have used these

data to conduct a number of cross-country analyses

on the costs and financing of national immunization

programs (Brenzel, 2015; Brenzel & Politi, 2012).

The cMYP data provide a general indication of ASD

costs for immunization programs, to the extent

that ASD costs can be mapped to a subset of the

cMYP components. Earlier iterations of the cMYP

analyses were organized around the following seven

immunization program components:

1. Immunization services delivery;

2. Program management;

3. Human resource management;

4. Costs and financing;

5. Vaccine cold-chain and logistics;

6. Surveillance and reporting; and,

7. Demand generation, communication, and

advocacy.

14 Available at http://www.immunizationcosting.org.15 The Excel based cMYP Tool and corresponding guidance documents are available at http://www.who.int/immunization/programmes_systems/financing/

tools/cmyp/en/.

Landscape Study of the Cost, Impact, and Efficiency of Above Service Delivery Activities in HIV and Other Global Health Programs 19

The current version of the cMYP is aligned with the

WHO Health Systems Framework consisting of six

‘building blocks,’ which is presented in Table 1, Health

system components of the most current cMYP

analysis framework, above.

The cMYP is organized primarily around inputs

rather than activities, which makes it challenging

for researchers to map ASD costs to health system

components. The standard summary data show

how the costs of an immunization program are

disaggregated by different inputs, many of which

do not clearly fall into either service delivery or ASD

cost, as illustrated below in Figure 11.

Table 1. Health system components of the most current cMYP analysis framework

Health system components Inputs Activities

Leadership & governanceProgram management, computers and office equipment

Meetings, planning, research, data management, expanded program on immunization (EPI) reviews, cold chain assessment, etc.

Health workforce Human resources/salaries, outreach per diems Supervision, training, workshops, etc.

Finance Financial resources Budgeting and monitoring expenditures

Medical product and technologyVaccines, auto-disable (AD) syringes, safety boxes, other injection supplies, cold-chain equipment vaccines, cold chain and logistics

Vaccine procurement and storage; monitoring; vaccine stock management activities

Service deliveryTransport, operational cost for routine immunization and campaigns

Operations for immunization delivery

InformationInformation, education and communication (IEC) materials, such as posters, etc.; surveillance and laboratory equipment

Social mobilization, IEC, development of advocacy and communication plan, surveillance

Table extracted from cMYP: A Tool and User Guide for cMYP Costing and Financing: Update 2014. WHO/IVB/14.06.

Figure 11. Prototypical cMYP results presentation, indicating the breakdown of cost data by input rather than by activities

Underused vaccines, $1,573,161

Personnel, $1,458,877

Other routine recurrent costs, $470,778

Traditional vaccines, $406,079

Injection supplies, $118,223

Transportation, $58,847

Campaigns, $57,201

Other capital equipment, $7,983

Cold chain equipment, $0

Vehicles, $0

New vaccines, $0

38%

35%

11%

11%

3%

0%1%

2%

0%

Figure extracted from cMYP: A Tool and User Guide for cMYP Costing and Financing: Update 2014. WHO/IVB/14.06.

20

What are immunization ASD activities and for what share of program spending do they account?

We examined the breakdown

of immunization costs for six

countries by input (resource

type), program activity, and

organizational level using data

from the EPIC study. Some

ASD activities of immunization

could occur in health facilities; however, in our analysis we

used organizational level as

a proxy for service delivery.

In other words, we focused

on “above the facility-level”

costs. We considered any

costs incurred in facilities to be part of service

delivery. We also defined the cost of vaccines and

injection supplies to be part of service delivery

regardless of the organizational level at which they

were accounted for in a particular country.

The six countries of the EPIC included four Gavi-

recipient countries in sub-Saharan Africa, Benin,

Ghana, Uganda, and Zambia, and two Gavi-

transitioning countries in other regions, Moldova and

Honduras. The total amount spent on immunization

programs ranged from 10 million USD to 54 million

USD for 2011. The cost of immunization programs

Figure 12. Immunization program cost per dose in EPIC study countries (USD, 2011)

$3.53

Benin Uganda Ghana Zambia Honduras Moldova

$3.93

$5.65

$7.86

$12.18

$14.48

Source: EPIC studies

Figure 13. Distribution of immunization program costs by organizational level in EPIC study countries (USD millions, 2011)

Source: EPIC studies

Vaccines andinjection supplies

Facility

District

Region

Central

Uganda Zambia Honduras Benin MoldovaGhana

15.36.35

5.39

7.99

10.1 1.06

10.1

37.0

4.81

16.7

24.8

19.1

3.322.08

8.123.35

0.93 0.7 0.94 0.684.61 0.39 0.23

0.210.54 0.14

3.71

Landscape Study of the Cost, Impact, and Efficiency of Above Service Delivery Activities in HIV and Other Global Health Programs 21

Figure 13 illustrates the

distribution costs of the

EPIC country immunization

programs across organization

level, while Figure 14 displays

the share of total costs

occurring above the facility

level. Across all 6 countries,

13 percent of the costs

(minimum of 6 percent

in Benin, maximum of 20

percent in Uganda) occurred

outside of health facilities

at sub-national levels (e.g.,

districts, provinces, regions)

and the central level.

Figure 15, below, illustrates

the distribution of program

costs by program activity in

the four sub-Saharan African

EPIC countries. Routine

vaccination in facilities and outreach vaccination

collectively made up 59 percent of program costs

on average; the major expenditures were vaccines,

injection supplies (i.e., syringes), and health worker

labor. Costs for the other activities included primarily

labor, cold chain equipment, and vehicles.

Figure 15. Breakdown of immunization program costs by activity for four sub-Saharan African EPIC countries, including vaccine and injection supplies

0

20

40

60

80

100

10

30

50

70

90

ZambiaUgandaBenin Ghana

Service delivery

Other

Social mobilization and advocacy

Record-keeping and HMIS

Surveillance

Training

Supervision

Program management

Vaccine collection,distribution and storage

Cold chain maintenance

Source: EPIC studies

Figure 14. Percent of total immunization program costs incurred at the district, region, and central levels in EPIC study countries

20%

Uganda Zambia Honduras Ghana Moldova Benin

18%17%

12%

7%6%

Source: EPIC studies

varied in part due to scale (number of infants), scope

(number of vaccines in routine schedule), and price

levels for non-tradable inputs (e.g., wages). The

average cost of delivering a dose of vaccine ranged

from 3.53 USD to 14.48 USD, including the cost of

the vaccine and injection supplies, as illustrated in

Figure 12.

22

We also looked at the program activities and parsed

them according to whether their costs were incurred

at the health facility level or at a higher organizational

level, as displayed in Figure 16, above. In the four

sub-Saharan African countries, the activities that

occurred above the facility level included vaccine

collection, storage, and distribution (21 percent),

program management (40 percent), supervision (44

percent), training (44 percent), and surveillance (32

percent).

To what extent are regularly reported ASD expenditure data available for immunization?

Since 2006, WHO member states have reported their

government’s immunization expenditures annually

via a Joint Reporting Form (JRF). More recently,

the JRF has been used to monitor progress on the

Global Vaccine Action Plan (GVAP), approved by

World Health Assembly in 2012.16

Though over 100 countries regularly report figures

for government expenditure on vaccines used for

routine immunization campaigns through the JRF,

there are noticeable gaps and inconsistencies in

reported data across multiple indicators. Figure

17 shows trends of completeness with respect to

countries’ reporting of six of the JRF expenditure

tracking indicators.17 The WHO IVB, which conducted

an analysis of the extent to which countries reported

JRF data from 2006 to 2012, noted that the following

challenges frustrated countries’ abilities to collect

accurate and complete JRF data:

• lack of information in countries with poor financial

management information systems;

• lack of accounting and financial skills within

Expanded Immunization Program (EPI) staff to

record, track and report expenditure data;

Figure 16. Share of each program activity cost incurred above the facility level in 4 sub-Saharan African EPIC countries

0% 20% 40% 60% 80% 100%

Benin

Ghana

Uganda

Zambia

Social mobilization and advocacy

Record-keeping and HMIS

Surveillance

Training

Supervision

Program management

Vaccine collection,distribution and storage

Cold chain maintenance

Source: EPIC studies

16 JRF data is publicly available through the WHO’s website, http://www.who.int/immunization/programmes_systems/financing/data_indicators/en/.17 The following are six of the JRF indicators:

• What amount of government funds was spent on vaccines used in routine immunization?

• What is the total expenditure (from all sources) on vaccines used in routine immunization?

• What percentage of all spending on vaccines was financed using government funds?

• What amount of government funds was spent on routine immunization?

• What is the total expenditure (from all sources) on routine immunization?

• What percentage of all spending on routine immunization was financed using government funds?

Landscape Study of the Cost, Impact, and Efficiency of Above Service Delivery Activities in HIV and Other Global Health Programs 23

• difficulty in clearly identifying what is included and

what is excluded in routine immunization;

• degree of EPI integration within the various service

delivery platforms (e.g., outreach, facility-based),

making it difficult to quantify shared inputs and

related costs; [and]

• lack of incentive to estimate routine immunization

expenditures. (WHO, 2014).

Researchers at the Sabin Institute recently analyzed

immunization financing data reported via the JRF

(Nader et al., 2014), which included a comparison

of reported JRF data to data from cMYP analyses.

The study found that “on average, mean JRF

immunization expenditures were 81 percent of the

corresponding cMYP baseline year immunization

costs,” suggesting “some degree of consistency in

how countries capture and report their health and

immunization expenditures.” (Nader et al., 2014).

We found that the JRF data are useful to analyze

expenditures, but JRF reporting is very high-level.

At present, none of the JRF’s financing indicators

provide data on the service delivery and above

service delivery expenditure split. The lack of

specificity in the indicators and the corresponding

data limit our ability to analyze these data to gain

insight into ASD expenditures. In addition, JRF data

do not include expenditures on shared resources

or on capital expenses, which further limits

comparisons with cMYP and EPIC cost data.

What is known about the impact/efficiency of immunization ASD activities?

We found little empirical research examining

the impact of ASD spending on the efficiency of

immunization activities. The EPIC team currently

is analyzing technical efficiency using data that it

collected in its first phase.

Some of our informants noted that the study

of technical efficiency in ASD activities such as

program management is not as a high a priority for

immunization programs as for other health programs

(e.g., HIV/AIDS). Our informants emphasized that this

lower prioritization does not reflect a lack of concern

or attentiveness for ASD efficiency, but rather the

view that immunization programs are fairly ‘lean’

and spend most resources on activities that directly

contribute to service delivery. Programs might save

the most resources by improving technical efficiency

at the facility level and reducing the access prices of

vaccines.

Figure 17. Trends of completeness for JRF reported data, as reported by the WHO IVB; trend data highlights gaps and inconsistencies in reporting over time

0

10

20

30

40

50

60

70

80

90

100Reported

Estimates

Inconsistencies

Missing

2006 2007 2008 2009 2010 2011 2012

Source: WHO, 2014

24

We found that the clearest relationships between

ASD activities and service delivery impact within

immunization programs are demonstrated by supply

chain and procurement. Vaccines are one of the

primary drivers of total immunization program

spending, thus, by maximizing efficiencies in the

supply chain, programs could incur substantial

savings by reducing waste and improving

performance. Poor supply chains result in programs

overstocking, allowing vaccines to expire, and

wasting procured vaccines. By improving logistics

and data collection and by using transport loops,

level jumping, informed push, and dedicated

personnel, programs can reduce these inefficiencies.

We discuss the supply chain and its impact on

service delivery in greater length in a separate

chapter of this report.

What current, on-going, or planned efforts improve the knowledge base of immunization-related ASD costs and their efficiency?

A second phase of the EPIC study is currently

underway, and a public dissemination workshop is

planned for May 2016 at the project’s end. EPIC II

aims to analyze the determinants of program costs

using data pooled across countries. EPIC II will

seek to disseminate lessons learned and technical

guidance for conducting, interpreting, and making

use of costing data in the context of national

immunization programs.

Additional research is also being conducted on

the impact and effectiveness of supervision. In

Ethiopia, an ongoing study seeks to document the

costs and outputs for interventions that will address

program management and supervision by the local

health office. The project is expected to terminate

in early 2016, with the results from the study to be

disseminated later in the year.

What are the key gaps in the immunization ASD knowledge base?

Data on the costs of immunization programs are

more robust than analogous data for other health

programs, including many of those presented in

this report. Recent and ongoing costing studies—

chiefly, those of the EPIC study—have generated a

foundation of granular costing data, which provide

insights into the costs of ASD activities and the

related share of program costs they comprise.

Although the EPIC studies have generated a strong

assessment of ASD costs for six countries, we found

that there is a paucity of baseline and longitudinal

cost data for immunization programs. The EPIC

study provides an estimate of costs for program year

2011, establishing a baseline by which to compare

future spending within the six original EPIC countries;

additional research is needed to understand the

relationship between shifts in program scope and

scale and changes in cost over time. Most countries,

however, lack these baseline data, and thus would

be candidates for future iterations of EPIC-like

assessments of immunization costs.

Our informants considered the methodologies

of EPIC studies, and stated that EPIC’s ASD

questionnaires to allocate health worker time to

immunization and across immunization activities

have not been robustly validated. It is possible

that EPIC’s costing analysis inaccurately captured

estimated personnel costs for immunization. We

anticipate that future studies may improve upon this

area of data collection. In addition, the EPIC studies

do not explicitly link cost data to outcome or process

metrics, which limits researchers’ abilities to estimate

ASD spending’s impact on service delivery.

Although regular immunization expenditure data

are reported through the JRF by a large number of

countries, the JRF financing indicators do not collect

the granular data that researchers require to analyze

ASD spending. Further, the JRF reporting documents

countries’ self-reported expenditures, rather than

costs. Consequently, the JRF also does not provide

reliable longitudinal cost data, which would enable

policy makers and analysts to study the relationship

between ASD spending and various other indicators—

most critically, coverage and quality of service delivery.

Finally, our informants observed that as country

immunization programs become more mature

and successful in providing high coverage of initial

vaccines, most national programs fail to adequately

administer booster campaigns, whose strategies

may differ from initial vaccination campaigns.

Immunization programs still are developing delivery

strategies for booster campaigns; these campaigns

may have implications for ASD programming and

spending, though the scope and scale of their impact

remains to be seen.

Landscape Study of the Cost, Impact, and Efficiency of Above Service Delivery Activities in HIV and Other Global Health Programs 25

Tuberculosis

What is the current level of publicly available data for ASD costs?

Among the six global health areas that we surveyed

in this landscape analysis, we found a scarcity of data

on ASD activities and costs related to tuberculosis

(TB) programming. We found little publicly available

data for costs of TB prevention, diagnosis, and

treatment programs in general, extending from ASD

activities to unit costs for site-level activities.

We found no studies on TB ASD activities, and no

information on the costs thereof in our literature

review. The TB Modelling and Analysis Consortium—a

collaboration of health modelers, analysts,

researchers, and epidemiologists working in TB—

anticipates conducting more detailed analyses on

costs and priority setting in the future, but these

studies have yet to be initiated.

Data on TB allocations through Global Fund grants

are recorded in individual program grant agreements,

which we discuss in the subsequent section on

malaria ASD. The Global Fund program grant

agreement budgets are available on the Global Fund

website as scanned PDFs, but these PDFs are not

machine readable. To determine the representative

size of budget ASD allocations within the Global

Fund’s TB portfolio based on these data, we would

have to analyze each grant’s budget individually in

order to extract the relevant data.

With respect to budgeting tools and costing

models, the WHO’s Global TB Programme currently

is overseeing and implementing a transition

between two tools for countries’ programs and

policymakers. The WHO’s Budgeting and Planning

Tool was developed in 2006—just as the seven-year

Global Plan to Stop TB was being implemented

(2006-2015)—to provide countries with support in

developing budgets for both domestic and donor

sources to use to mobilize TB resources. An Excel

based tool, the Budgeting and Planning Tool,

enables users to generate a wide number of budget

estimates, including for ASD activities (such as

monitoring and evaluation). However, the tool is not

pre-populated with respect to these activities, and,

thus, requires users to provide substantial country-

specific inputs to generate ASD budget estimates.

The WHO currently is encouraging health program

and policy planners to use the newly developed

OneHealth software tool, developed by Avenir

Health, and launched in mid-2011. This tool provides

users with a unified framework to analyze costs and

finances for all major disease and health system

areas. The analytic framework is more horizontal

than vertical. The OneHealth tool is similar to the

Budgeting and Planning Tool in that it is capable

of estimating ASD activity budgets; however, the

OneHealth tool does not have built-in assumptions

about ASD costs, and also requires substantial user-

provided inputs to generate these estimates. Due to

the number of inputs needed to calculate ASD line

allocations, these tools require that users have robust

knowledge of country-specific costs to generate ASD

budget estimates for tuberculosis programming.

The WHO National Health Account (NHA) repository

also includes TB expenditures for some countries,

which we describe in more detail in the following

section. See Appendix 3 for additional discussion

of these data and the challenges of using them to

identify ASD costs.

What are ASD activities and for what share of program spending do they account?

Our informants expressed that ASD spending could

be an important component of TB expenditures,

particularly with regard to the introduction of

new technologies, such as new drugs or new

diagnostics. Many of our informants noted that these

new technologies require large training efforts to

ensure that healthcare practitioners implement the

technologies effectively, prescribe and apply new

drugs correctly, and use new diagnostic machines

correctly and in adherence to clinical guidelines.

Since 2002, the WHO has monitored government

and international donor financing for TB. Each

year, the WHO asks countries to complete a

questionnaire, which includes questions on their

estimated spending for TB programs. The survey

yields expenditure data disaggregated across the

following categories:

• Laboratory infrastructure, equipment, and

supplies;

• National TB Programme staff (central unit staff and

subnational TB staff);

26

• Drug-susceptible TB: drugs;

• Drug-susceptible TB: programme costs;18

• Drug-resistant TB: drugs;

• Drug-resistant TB: programme costs;

• Collaborative TB/HIV activities;

• Patient support;

• Operational research and surveys; and

• All other budget lines.

The WHO’s global TB database stores the data

from this spending survey. The expenditure tracking

data undergo a month-long validation process

between WHO and TB endemic countries. The

data ultimately inform the financing section of the

Global Tuberculosis Report. Although the data are

accessible upon request, they are not released by

the WHO as a publicly accessible data repository.

Aggregated financing figures, which report total

program expenditure (without any disaggregation by

the budget lines mentioned above), are accessible on

the WHO’s website, but do not provide any indication

of ASD activities or spending.

We requested and obtained TB expenditure data

from the WHO’s global database. Specifically, the

data we obtained provide self-reported expenditure

estimates for 47 countries, between 2010 and 2014,

as illustrated in Figure 18 below. The data provided

by the WHO were disaggregated by eight categories;

three categories—national-level TB program staff,

operational research, and laboratory operations—

were identifiable as ASD-related expenditures. The

available data indicate pronounced variation in the

relative size of ASD-related expenditures against total

program spending, ranging from roughly 74 percent

of spending (Benin) to just under 3 percent (Uganda).

On average, the TB expenditure data indicate that

Figure 18. WHO tracked TB expenditure data (USD, 2010-2014), disaggregated by program activity/category

Other

First-line TB drugs

Second-line TB drugs

MDR-TB management

TB/HIV

Laboratory operations

Operational research

National-level sta�

0.0

0.2

0.4

0.6

0.8

1.0

0.1

0.3

0.5

0.7

0.9

Benin

Gam

bia

Equa

toria

l Gui

nea

Keny

aRw

anda

Gha

na

Cen

tral A

frica

n Rep

ublic

Gui

nea-

Bissau

Mau

ritan

ia T

anza

nia

Côt

e d’

Ivoi

reN

amib

iaSe

nega

l

Sout

h Su

dan

São

Tom

é an

d Pr

ínci

peLi

beria

Togo

Nig

eria

Swaz

iland

Cha

dBur

undi

Ethi

opia

Mad

agas

car

Zam

bia

Burki

na F

aso

Gui

nea

Nig

er

Zim

babw

e

Cab

o Ve

rde

Mal

awi

Botsw

ana

Gab

on Mal

iD

RCC

amer

oon

Eritr

eaLe

soth

o

Moz

ambi

que

Con

goU

gand

aAv

erag

e

Source: WHO

18 Including program management, supervision, training, policy development, office equipment/vehicles, construction of buildings, surveillance, advocacy and communication, public-private mix activities, community engagement, active case-finding, infection control, procurement and distribution of TB drugs, and other line items

Landscape Study of the Cost, Impact, and Efficiency of Above Service Delivery Activities in HIV and Other Global Health Programs 27

countries spend roughly 43 percent of total program

spending on ASD activities/costs.

Besides the WHO’s TB country spending data,

the only other publicly available source of ASD

expenditure data is the National Heath Account

module on TB expenditures. As in other NHA

modules, TB expenditures are reported for

different activities, including for “governance and

administration,” as illustrated in Table 2.

NHA data for tuberculosis expenditures are not

reported across all country NHAs. We found in

the sample of countries that we examined in this

study that the reported values range from 0 percent

(Ghana, 2012) to 60 percent (Niger, 2013), when

countries reported the percent of program spending

directed to governance and administration. Prima

facie, it is unlikely that this wide variance reflects the

true differences between countries’ expenditures

on governance and administration to support TB

programs. Rather, the data call into question how

consistently the data have been gathered in different

countries.

What is known about the impact/efficiency of TB ASD activities?

In spite of the scope and size of TB programs

worldwide, we found a lack of analysis on the impact

and efficiency of ASD within TB programming.

Our informants speculated that ASD activities

were necessary for the successful operation of TB

treatment and prevention initiatives, and for the

success of the Global Plan to Stop TB overall. Many

commented, however, that researchers have barely

studied these activities and their costs, impact, and

efficiency.

We found it difficult to evaluate the efficiency of ASD

activities in TB without better costing data. Some

of our informants speculated that the dearth of TB

costing data could be attributed to a large funding

vacuum within TB. One informant noted that donors

do not support TB costing work to the full extent

needed, notwithstanding the funded activities of the

TB Modelling and Analysis Consortium (TB-MAC).

Informants noted that the lack of available funding

for broader costing studies has a chilling effect on

the interest for TB cost research.

TB is a health area that collects a number of

treatment outcome indicators that can serve as

overall performance proxies for national responses

to TB, which researchers can use to examine

the impact of ASD activities. In this regard, with

better data on costs/expenditures, we anticipate

that researchers could examine how changes in

ASD activities within a country relate to changes

in performance. At a high level, researchers could

measure impact and efficiency by comparing the

high-level WHO country-reported spending data

with TB treatment outcome indicators. For example,

in Figure 19, we plotted the percent of a country’s

total expenditures dedicated to ASD activities against

the country’s TB treatment success rates for select

countries.

The data that we present in Figure 19 display no

significant correlation. It is important to note,

however, that the self-reported TB expenditure

data are incomplete for various cost categories and

across various years, and, thus, may not present

a comprehensive assessment of ASD spending.

More robust spending data could inform additional

analyses against treatment success indicators; such

data collected over time could inform longitudinal

analyses, capable of being disaggregated by individual

Table 2. Percent of total health expenditures for TB identified by NHA analysis for

governance and administration

Percent of expenditures for governance and administration

Kenya (2012-13) 18%

Tanzania (2009-2010) No data

Malawi (2011-2012) No data

Benin (2012) 16%

Burkina Faso (2013) 32%

Ghana (2012) 0%

Niger (2013) 60%

DRC (2013) 12%

Source: SHA, WHO

28

ASD cost components, providing initial indications of

the impact of (changes in) ASD spending.

What current, on-going, or planned efforts improve the knowledge base of ASD costs and their efficiency?

The TB-MAC recently received funding to conduct

detailed analyses to provide greater analytic data on

costs and priority setting. These analyses will focus on

site level and patient costs; our informants indicated

that ASD cost data may be part of the analysis.

Our informants indicated that tracking expenditures—

particularly by using NHAs—may help to capture

both ASD and site level costs in a systematic

and comprehensive way. As more NHAs are

conducted using the WHO’s new System of Health

Accounts framework, it is likely that additional

NHAs will provide new data on TB governance and

administration costs. However, we do not know

whether these data will eventually provide any

greater granularity of focus within these categories.

What are the key gaps in the TB ASD knowledge base?

We found a lack of comprehensive data

encompassing both service delivery costs and ASD

costs for TB interventions, which hindered our

ability to analyze above service delivery spending

and efficiency. This was the case in several of the

other health areas we examined in this study as well.

Our informants noted that research on cost and

efficiency of ASD activities in TB is frustrated by a lack

of data for basic unit costs even at the site-level; this

differs from some other health areas, in which site-

level unit costs are highly researched. Without good

site-level data, we found it to be impossible to judge

the impact of ASD activities on site-level costs or

performance. A recent survey of cost data availability

for TB indicated that there is a range of data on

treatment costs, but much of these data are out of

date. Furthermore, there are fewer data available on

the costs of diagnosis and detection, and little to no

data on the costs of improving TB service efficiency

(Lawrence & Baena, 2015). A stronger foundation of

cost data would provide researchers with a clearer

sense of the costs of ASD activities for TB, and

more readily accessible and reliable expenditure

Figure 19. WHO tracked TB expenditure data indicating the percent of total country ASD-related expenditure plotted against TB treatment success rates (percent of new cases, 2012)

0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1.0 Tuberculosis treatmentsuccess rate (percent of new cases)

ASD spending as a percent of total TB spending

Tanz

ania

Côte d

'Ivoire

Nam

ibia

Sene

gal

Sout

h Su

dan

São T

omé

and

Prín

cipe

Libe

riaTo

goN

iger

iaSw

azila

ndCha

dBur

undi

Ethi

opia

Mad

agas

car

Zam

bia

Burki

na F

aso

Gui

nea

Nig

erZi

mba

bwe

Cabo V

erde

Mal

awi

Botsw

ana

Gab

onM

ali

DRC

Camer

oon

Source: World Bank

Landscape Study of the Cost, Impact, and Efficiency of Above Service Delivery Activities in HIV and Other Global Health Programs 29

data would enable program staff and researchers to

monitor the impact of activity spending on programs’

performance.

Another emerging issue within TB is the current

effort to increase the use of PCR assays, such as

the Xpert automated cartridge-based machine. The

WHO currently is engaged in an initiative to promote

the procurement and distribution of Xpert machines,

which, though more expensive than conventional

microscopy diagnostics, are more sensitive and

reliable. Numerous modeling studies have estimated

that the implementation of Xpert MTB/RIF is cost-

effective for the diagnosis of TB and multidrug-

resistant TB in counties with a high burden (Langley

et al., 2014; Pantoja et al., 2013; Theron et al., 2012;

Vassall et al., 2011). However, many of these studies

differ in assumptions regarding rates of disease

transmission and the sensitivity of baseline laboratory

testing protocols (Dheda, Theron, & Welte, 2014).

Our informants noted that the success of Xpert

integration depends on training laboratory staff

successfully so that they are able to use the new

equipment that is introduced. These trainings—in

addition to the costs of new equipment—may

contribute to TB ASD spending. However, the

extent to which they contribute to overall spending

is unclear as training costs are not included in the

cost-effectiveness analysis of Xpert integration

consistently. Other considerations related to

laboratory operations are discussed in greater detail

in a subsequent chapter.

Malaria

What is the current level of publicly available data for malaria-related ASD costs?

Although malaria is one of the largest global

health areas that we considered in this landscape

assessment, we found limited high-quality, publicly

available data for ASD costs across its three

programmatic stages: control, elimination, and post-

elimination.

In our literature review we found a single academic

study on malaria ASD activities (Sabot et al., 2010).

This study’s authors conducted a literature review

to estimate the costs incurred by malaria programs

across the years in which the programs’ phases

shifted, from controlled-low endemic malaria (CLM),

to elimination, to prevention of reintroduction (POR).

The five case studies provided estimated costs

over time for malaria programs in China’s Hainan

and Jiangsu provinces, Mauritius, Swaziland, and

Tanzania’s Zanzibar archipelago.

Consistent with the findings of our literature review,

Sabot et al. found a paucity of contemporary

published data on the costs of malaria elimination

programs. The authors note that the most recent

robust analysis of program costs in malaria

“examined the 5-year expenditures of most countries

participating in the GMEP,” the Global Malaria

Eradication Programme, which was conducted

between 1955 and 1969. Although data from this

era may provide researchers with some helpful

insights into current costs, the GMEP cost data

do not describe the program’s components or

epidemiological context, and, thus, do not help

researchers ascertain the costs of ASD associated

activities.

The country case studies described by Sabot et al.—

constructed from national health accounts, donor

proposals, and informant interviews—indicated that

program management costs tend to rise as countries

advance from controlled-low endemic malaria

(CLM) to elimination,19 in both absolute amount

and as a share of total annual costs. The trend in

costs described by Sabot et al. is illustrated in Figure

20 below. The average share of total annual costs

dedicated to program management was 23 percent,

with a minimum of 9 percent (Zanzibar, CLM 2009-

13) and a maximum of 40 percent (Jiangsu, China,

elimination 2010-2014).

In Figure 20, we display the average annual malaria

program management costs as a percent of total

annual cost (primary axis) and in absolute value

(secondary axis) for the four regions that we identify.

Lightly colored bars indicate percent of total cost in

CLM phase; darkly colored bars indicate percent of

total cost in elimination phase.

19 The Sabot et al. case study for Mauritius examines the transition between elimination and prevention of reintroduction from 1983 to 2008, and is thus not included in the analysis.

30

In addition to the data on these few countries

provided by Sabot et al, data for a larger group of

countries and time periods are available on Global

Fund grant budgets, which are documented across

the Global Fund’s program grant agreements. These

budget data are disaggregated by the following

expenditure categories:

• Human resources

• Technical assistance

• Training

• Health products and health equipment

• Medicines and pharmaceutical products

• Procurement and supply chain management costs

• Infrastructure and other equipment

• Communication materials

• Monitoring and evaluation

• Living support to clients/target population

• Planning and administration

• Overhead

• Other

Although several of these categories include ASD

activities (e.g., monitoring and evaluation, training,

planning and administration, etc.), the Global Fund

does not publish grant budget data via a repository

or in a machine readable format. The program grant

agreements that collect these data are available on

the Global Fund website as scanned PDFs; however,

without Excel compatible data on grant budgets

and expenditures, we cannot ascertain budget ASD

allocations within the Global Fund’s malaria portfolio

without engaging in a time-consuming grant-by-

grant review.

The WHO National Health Account (NHA) repository

also includes malaria expenditures for some

countries, which we describe in more detail in the

following section. See Appendix 3 for additional

discussion of these data and the challenges of using

them to identify ASD costs.

Additional data on malaria ASD costs are available

through costed national malaria strategic plans.

Government officials—under the Roll Back Malaria

Initiative—have developed a number of strategic

plans that include costed components of varying

detail and disaggregation; those for Namibia, Nigeria,

Malawi, Sierra Leone, and Uganda provide sufficient

detail for comparison.

Figure 20. Average annual program management costs for malaria by program phase and territory, from Sabot et al.

5%

10%

15%

20%

25%

30%

35%

40%

0%

45%

$0

$1

$2

$3

$4

$5

$6

$7

Mill

ion

s

Average annual cost

Percent of total annual cost (CLM)

Percent of totalannual cost (elimination)

2007-09 2010-14

China-Hainan

2007-09 2010-14

China-Jiangsu

2004-08 2009-13

Swaziland

2009-13 2010-19

Tanzania-Zanzibar

27%

31%

37%

23%

9%

13%

34%

40%

Source: Sabot et al. 2010

Landscape Study of the Cost, Impact, and Efficiency of Above Service Delivery Activities in HIV and Other Global Health Programs 31

In Figure 21, we present the breakdown of total

program costs associated with national malaria

strategic plans for the year 2014. The costed national

plans each contain different cost categories and

were not conducted using a standardized approach;

in order to present comparative cost data, we

cross-walked cost categories across the various

national plans into standard categories. These

categories echo those used in the EPIC studies for

immunization program costs and the Global Fund’s

malaria grant budget data.

Across these five plans, the average share of total

projected costs for 2014 dedicated to ASD activities

was 22 percent, with a minimum of 5 percent

(Malawi) and a maximum of 46 percent (Namibia).

What are malaria ASD activities and what share of program spending do they comprise?

ASD activities with malaria are similar to those of other

communicable disease programs, like HIV/AIDS and

TB, and may include surveillance, monitoring and

evaluation, HMIS, training, administration, and supply

chain. We were unable to estimate the relative share

of program spending that these activities comprise

given the lack of granular data on ASD spending.

With regard to expenditure data on malaria

programming, we found only one publicly accessible

database, the WHO National Health Account

(NHA) repository, which has a module dedicated to

capturing malaria expenditures. This information is

presented in Table 3 below.

Table 3. Percent of total health expenditure for malaria identified by NHA analysis

for governance and administration

Percent of malaria spending for governance and administration

Kenya (2012-13) 16%

Tanzania (2009-2010) 1.4%

Malawi (2011-2012) 13.4%

Benin (2012) 16%

Burkina Faso (2013) 19%

Ghana (2012) 0%

Niger (2013) 30%

DRC (2013) 5%

Source: SHA, WHO

In cases in which NHA data on ASD activities are

available for malaria expenditures, we have captured

these data under the category “Governance and

health system and financing administration.”20 Across

different country NHAs, this cost category varies

as a share of total program cost, from 0 percent

(Ghana, 2012) to 30 percent (Niger, 2013). Among

the countries that have available data, the average

share of malaria related expenditures dedicated to

governance and administration was 13 percent of

total program expenditures.

Figure 21. Breakdown of projected malaria program financial need for five sub-Saharan African countries, disaggregated by program activity/category (USD, 2014)

0%

20%

40%

60%

80%

100%

10%

30%

50%

70%

90%

Service delivery

Other

Social mobilization and advocacy

Record-keeping and HMIS

Surveillance

Training & HR

Supervision

M&E

Program management

Procurement, supply chain, and transport

Source: Roll Back Malaria costed national plans

20 See Appendix 3.

32

The NHA data do not provide granular insights into

how governance and administration expenses may

be disaggregated by activity or functional level within

the health system. Data from the President’s Malaria

Initiative and from the World Malaria Report similarly

do not include granular expenditure data on ASD

activities.21 The World Malaria Report does include

country profile data on financing by intervention, which

includes financing for monitoring and evaluation; this

information is presented through graphical read-outs,

however, and does not include the source data.22

Using the graphical read-outs from the World Malaria

Report’s country profiles, we were able to manually

extract data on financing by intervention for twenty-

nine sub-Saharan African countries. These data are

presented in Figure 22.

The data obtained from the World Malaria Report

reflect self-reported financing estimates for

2014, disaggregated by seven categories. Three

categories—monitoring and evaluation, human

resources and technical assistance, and management

and other costs—were identified as ASD-related

activities for the purposes of our analysis.

The available data indicate substantial variation in

ASD-related financing as a share of total program

financing, ranging from 95 percent (Comoros) to 0

percent (Uganda). Excluding countries for which data

was not reported for 2014, the malaria financing data

indicate that countries in sub-Saharan Africa direct

roughly 48 percent of funding for malaria to ASD

activities. This contrasts with the data presented earlier

in this chapter extracted from the Roll Back Malaria

costed national plans, which found a significantly

lower share of costs for ASD activities. These data

are derived in a very different way and cannot easily

be compared; costed plans refer to planned costs

rather than actual financial flows, and the categories

used are not identical. It is also possible that some

costs are included in the costed plans but excluded

in the financial flows, or vice versa, given the different

perspective of these analyses.23

21 Data on the President’s Malaria Initiative funding for FY 2006-FY2013 can be accessed in Appendix I of PMI’s Eighth Annual Report to Congress, available at http://www.pmi.gov/docs/default-source/default-document-library/pmi-reports/pmireport_final.pdf?sfvrsn=14. Data from the World Malaria Report 2015 may be accessed at http://www.who.int/malaria/publications/world-malaria-report-2015/report/en/.

22 The WHO team is considering publishing this data in future annexes of the World Malaria Report, but were unable to release these data to us on request.23 As an example of how different perspectives can lead to different activities being captured: a report of financial flows for malaria may exclude the costs

of some healthcare workers who are paid for as part of the general primary healthcare system (as these workers are not part of a ‘malaria budget’), while the labor of these workers may be included in a costed national plan (as the time and effort that these workers dedicate to malaria response constitutes a drain on the resources of the healthcare system, ultimately requiring either that additional workers are hired (at some financial cost) or that workers are diverted from other activities (with some associated opportunity cost)).

Figure 22. WHO tracked malaria financing data (USD, 2014), disaggregated by program activity/category

Insecticides & spraymaterials

ITNs

Diagnostic testing

Antimalarial medicines

Monitoring andevaluation

Human resources & technical assistance

Management andother costs

Sout

h Af

rica

Mad

agas

car

Gab

on

Eritr

ea

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Gam

bia

Keny

a

Gha

na

Gui

nea-

Bissau

Tan

zani

a (Z

anzi

bar)

Tan

zani

a (M

ainl

and)

Nam

ibia

Sene

gal

São

Tom

é an

d Pr

ínci

pe

Com

oros

Libe

ria

Nig

eria

Swaz

iland

Alge

ria

Burun

di

Ethi

opia

Zam

bia

Sier

ra L

eone

Burki

na F

aso

Nig

er

Cab

o Ve

rde

Mal

i

DRC

Moz

ambi

que

Uga

nda

Aver

age

Aver

age

Reported confirmed cases exceeds250 cases per 10,000 population

Reported confirmed cases is lower than250 cases per 10,000 population

Source: WHO

Landscape Study of the Cost, Impact, and Efficiency of Above Service Delivery Activities in HIV and Other Global Health Programs 33

Our analysis further indicated that malaria financing in

countries where the prevalence of malaria exceeded

250 cases per 10,000 primarily went to service

delivery activities—on average, only a third of financing

went to ASD activities. By contrast, ASD-related

malaria financing in countries with prevalence rates

below 250 cases per 10,000 comprised, on average,

63 percent of total program financing. A simple

regression analysis indicates a statistically significant

negative relationship between prevalence and the

portion of financing dedicated to ASD activities.24

Although these data reflect differences across

countries rather than across time, and cannot be

interpreted causally, these findings are consistent with

the idea that as prevalence declines, a lesser share of

malaria resources may be dedicated to broad vector

control efforts and treatment, and a greater share may

be dedicated to ASD activities such as surveillance.

What is known about the impact/efficiency of malaria ASD activities?

As discussed above, we found little research addressing

the impact or efficiency of ASD activities in malaria

programs in the published or grey literature. Our

interviews with experts corroborated our finding.

However, our informants stated that ASD activities were

likely to be critically important as malaria programs

shifted their focus from control to elimination and,

finally, prevention of reintroduction. In countries in

which there is consistently high prevalence of malaria,

pushing prevention and treatment services to all areas

of the affected country may be a reasonable (and

politically palatable) policy, but as prevalence decreases

and outbreaks become more sporadic, a more

differential approach may be needed (particularly as

budget allocations for malaria decline). Our informants

expressed that better surveillance systems for malaria

enable more strategic targeting and prioritization of

efforts, which results in more efficient deployment

of resources. Just as elimination campaigns require

a “heavy push” and strong surveillance services, our

informants aver that post-elimination malaria programs

likely will require strong surveillance, monitoring, and

evaluation throughout at-risk areas.

Our informants also suggested that as malaria

prevalence declines within a country many malaria

programs increasingly rely on a horizontal approach.

When malaria is highly endemic, malaria programs

often use a vertical approach for prevention activities

and commodity supply that run parallel to the primary

healthcare system.

As prevalence declines, however, and as countries

pursue elimination as a goal, malaria programs require

increased coordination and horizontal distribution

across the health system. In this scenario, central level

planning and administration may again contribute

to a higher relative share of ASD spending, although

researchers have yet to examine this.

What current, on-going, or planned efforts improve the knowledge base of malaria ASD costs and their efficiency?

At present, CHAI is attempting to gain a

comprehensive estimate of the costs associated with

malaria prevention, particularly for indoor residual

spraying (IRS) campaigns. This analysis is meant

to provide a thorough understanding of IRS costs

incurred at every level of the campaign; CHAI has not

yet released results of this analysis.

Focusing on the costs of preventing malaria, as CHAI

is, may be more tractable for researchers than broader

malaria costing. From a conceptual standpoint,

calculating the comprehensive vertical costs of malaria,

which incorporate treatment as well as prevention, and

costs at every level, is complicated by the nature of

detecting cases of malaria. One of the most common

symptoms of malaria infection is fever; patients who

present with malaria-like fevers enter the primary

healthcare system, which acts as a catchment for cases

of malaria and other fever related illness. It is, thus,

unclear to what extent researchers should consider the

primary healthcare system within the context of malaria

program costs; as one our informants noted, it is often

difficult to assess where primary healthcare systems

end and malaria programs begin.

What are the key gaps in the malaria ASD knowledge base?

We found that comprehensive costing data for

malaria interventions, prevention, and treatment are

24 The regression of [percent of financing to ASD activities] against [Natural logarithm of: confirmed malaria cases per 10,000 population] is statistically significant at p<0.05, with a coefficient that suggests that when prevalence doubles, the share of finance for ASD activities decreases by 2.5 percentage points. This negative relationship is more pronounced if analysis is restricted to countries with at least 1 case per 10,000 (i.e., excluding Algeria and Cabo Verde).

34

missing from the knowledge base. If researchers

were able to perform a robust analysis of programs’

costs, based on up-to-date data from countries

that are implementing malaria programs in varying

stages, researchers would have greater insight into

the relative size and share of programs’ costs for

which the above service delivery activities account.

Our informants noted that understanding what

specific surveillance activities are required in different

circumstances, and what these activities should cost,

should be a particular priority for further investigation.

With regard to expenditure tracking, we found

that more granular financial data on spending

towards individual ASD activities, such as a program

management and surveillance, could be used to

empirically study the relationship between prevalence

and ASD spending. By linking expenditure data to

programs' outcomes related to prevalence and

coverage, researchers would be able to estimate the

impact and efficiency of ASD activities for countries for

which granular data are available.

Finally, as countries reduce the prevalence of malaria,

their health systems must increasingly work to maintain

the cost-effectiveness and quality of malaria programs

across larger coverage areas that have declining

prevalence and more sporadic incidence of malaria,

and hence less predictable demand for medicines.

Ensuring that affected individuals have access to

antimalarial medicines may be a challenge for the

system, particularly given the sharp growth in global

demand for artemisinin-based combination therapy

(ACT) since its implementation as the standard first-line

treatment for malaria (WHO, 2011). Creative solutions

above the point of service delivery—including rotating

buffer stock and non-rotating emergency buffers—are

needed to avoid stockouts, expiry, and supply chain

disruptions in access to ACT (Shretta & Yadav, 2012).

Nutrition

What is the current level of publicly available data for nutrition ASD costs?

Of the six global health areas that we analyzed in this

report, nutrition is among the most varied in both

focus and interventions. Nutrition is a broad field,

encompassing multiple indicators and dimensions of

illness. Nutrition programs may target undernutrition—

evidenced in stunting, wasting, and vitamin deficiency—

or overnutrition, as in the case of obesity. These

conditions have unique interventions, which may

further be characterized by whether the interventions

directly target the immediate causes (“nutrition-specific

interventions”) or the underlying determinants of the

conditions (“nutrition-sensitive interventions”).

In this chapter, we focus on nutrition-specific

interventions, which are more analogous to the targeted

interventions described in other health areas in this

report, but we refer to the broader definition of nutrition

(including nutrition-sensitive interventions) where

appropriate. Nutrition-specific interventions include

the distribution of supplementary foods, community-

based management of acute malnutrition (CMAM),

biofortification of staple foods, exclusive breastfeeding

promotion, and vitamin A supplementation for

infants and pregnant women. Nutrition-sensitive

interventions, by comparison, encompass a wide range

of program areas, including agriculture; education;

social welfare; water, sanitation, and hygiene; and

women’s empowerment. Efforts by researchers to

cost and finance these programs have been especially

challenging, because of this broad inclusion criteria for

nutrition sensitive programs.

Researchers have found it difficult to capture cost

data that are standardized across all facets of nutrition

programming; comprehensive basic cost data have yet

to be gathered for a number of nutrition interventions.

Data on ASD costs are similarly limited; there are few

studies or repositories that capture these costs in any

systematic way. In this section, we examine four sources

of cost data: academic studies published on the costs

of CMAM, the World Bank’s 2010 report Scaling Up

Nutrition: What Will It Cost?, the Lives Saves Tool (List),

and the Scaling Up Nutrition (SUN) Movement cost

national nutrition plans.

In our literature review, we identified only two relevant

academic studies published since 2010 that discussed

ASD activities and costs within the context of nutrition

programming, both of which focused on the costs

of a CMAM intervention. CMAM is a community

mobilization program that requires training, supervising,

and monitoring a large number of community health

workers (CHWs). These studies—one in Malawi and

one in Bangladesh—reported that ASD costs related to

management and overhead comprised 51 to 53 percent

of program costs, respectively (Puet et al., 2012; Wilford

et al., 2010). Although this expenditure split may suggest

that these CMAM programs were management-heavy,

the high ASD costs associated with the community

Landscape Study of the Cost, Impact, and Efficiency of Above Service Delivery Activities in HIV and Other Global Health Programs 35

case management of severe acute malnutrition (SAM) in

these two studies can be understood as a consequence

of the high initial start-up costs of launching the CMAM

programs. Generally, the start-up ASD costs are higher

for CMAM programs due to the volume of training

and field-level management necessary to operate the

programs effectively. As CMAM programs mature,

costs decline, as new staff require less training, and the

programs utilize more cost-effective CHWs and create

new monitoring systems.

We found in the grey literature on nutrition costs

that the leading costing analysis is the World Bank’s

2010 report Scaling Up Nutrition: What Will It Cost?

(Horton et al., 2010). This flagship paper provides costs

for a range of recommended nutrition interventions

and analyzes the resources necessary to scale up

nutrition programming worldwide. Nonetheless,

the report contains relatively little discussion of ASD

activities needed to support the programs’ delivery.

While ASD costs are not directly considered within the

analysis, the authors estimate that $200 million USD

of the $11.8 billion USD needed to scale up nutrition

globally is necessary for operations research, technical

assistance, and monitoring and evaluation of large-

scale programs. Within each intervention, ASD activities

such as supply chain and procurement are discussed

as delivery costs within the analysis. These costs are

estimated to account for varying shares of different total

intervention costs, ranging from less than 5 percent for

iron fortification and salt iodization to 96 percent for

vitamin A supplementation. The report estimated that

for complementary feeding delivery costs comprised

12 percent of the total intervention costs; the authors

note that this allocation is “probably an underestimate;

further research is needed” (Horton et al., 2010). The

varying costs can be understood, in part, as a function

of commodity costs (e.g., vitamin A supplementation

commodity costs are far lower than those for iron

fortification and salt iodization); however, the report

does not directly address this variance in delivery and

ASD costs among interventions.

The World Bank currently is engaged in a second global

nutrition scale-up costing project (in partnership with

R4D and 1,000 Days, with the support of the Children’s

Investment Fund Foundation and BMGF), whose results

are anticipated to be released in August 2016. The

costing methodology for the ongoing project estimates

the costs of ASD activities by increasing the cost of the

intervention by 12 percent, pursuant to the estimate by

Horton et al. (2010). In the emerging analysis, policy

development (9 percent), monitoring and evaluation

(2 percent), and capacity strengthening (1 percent)

contribute to the estimated 12 percent for ASD activities.

The leading impact model for nutrition interventions,

the Lives Saved Tool (LiST), contains a costing

module that includes intervention-specific costing

estimates. The tool provides users flexibility in

accounting for ASD activities under its indirect costs

category for outpatient and inpatient days. This

category captures “support service costs like central

support/management staff, international consultants,

maintenance workers, and supervision of staff, as

well as insurance, utilities (telephone, electricity,

etc.), publicity and other promotional activities, office

furniture, other equipment such as autoclaves and

typewriters, vehicle maintenance, other electronic

maintenance, and monitoring and evaluation” (LiST,

2015). By default, however, these inputs are not

populated, and users must estimate and account for

these costs in the analysis.

The most comprehensive data available on the costs of

countries’ plans for nutrition has been assembled by the

Scaling Up Nutrition (SUN) Movement, a member-led

consortium of countries and networks of civil society,

business, and UN organizations working to improve

nutritional outcomes worldwide (SUN Secretariat,

2014). The SUN Movement, under its Synthesis Report

(“Planning and costing for the acceleration of actions for

nutrition: experiences of countries in the Movement for

Scaling Up Nutrition”) presents costed national plans for

20 countries.25

The analytical framework that classifies nutrition

costs for the country plans' separate costs into three

sub-categories: nutrition-specific, nutrition-sensitive,

and governance. ASD activity costs considered in

these plans generally fall under the governance

sub-category, which is defined by the Synthesis

Report as including “coordination and information

management, systems and capacity building, and

policy development, advocacy and capacity building”

(SUN Secretariat, 2014). Among the 13 sub-Saharan

African countries whose national nutrition plans

were analyzed under the SUN Synthesis Report,

governance costs reflected an average of 25 percent

of total program cost (when nutrition-sensitive

costs were excluded), ranging from 3 percent

(Mozambique) to 62 percent (Uganda).

25 This guidance documents can be accessed at http://scalingupnutrition.org/wp-content/uploads/2015/06/CRF-TOOL-Guidance-Notes.pdf.

36

26 The CRF Planning Tool can be accessed at http://scalingupnutrition.org/resources-archive/financial-tracking-resource-mobilization/aggregated-planning-tool.27 This guidance documents can be accessed at http://scalingupnutrition.org/wp-content/uploads/2015/06/CRF-TOOL-Guidance-Notes.pdf.

The SUN Movement also hosts the Common

Results Framework (CRF) Planning Tool, an Excel

database that houses disaggregated costing data for

country nutrition plans.26 The CRF Planning Tool is

managed by Maximising the Quality of Scaling Up

Nutrition Programmes (MSQUN), an association of

technical experts. Countries participating in the SUN

Movement are invited to share their plans with the

SUN Secretariat; MQSUN then reviews the costed

national plans and inputs the data into the CRF

Planning Tool.

We found that the cost categories presented in the

SUN Synthesis Report and the CRF Planning Tool

captured a wide range of program activities (SUN

Secretariat, 2014). The CRF Planning Tool presents

disaggregated costs for the following governance

activities, as represented in Figure 23 for a selection

of sub-Saharan African countries:

• System Capacity Building

• Surveillance

• Research

• Policy Development

• Monitoring and Evaluation

• Information Management

• Governance

• Coordination and partnership

• Communication

• Advocacy

Across many of these plans, the costs for system

capacity building activities are the largest within the

governance sub-heading. These activities are defined

by the CRF Planning Tool guidance document as

activities “which are system-wide, i.e., they support

the systems and functionality of all nutrition activities

and services” (SUN Secretariat, 2015).27

What are ASD activities and for what share of program spending do they account?

With regard to actual nutrition spending, one of the

few sources of data that we identified is the WHO

National Health Account repository (NHA), which

Figure 23. Governance costs by activity for selected SUN costed national plans, indicating the heavy proportional costs of system capacity building as a share of total governance costs

Advocacy

Communication

Coordination and partnership

Information management

Surveillance

Monitoring and evaluation

Research

Policy development

System capacity building

Nam

ibia

(2014

-2016

)

Yem

en (2

015-2

019)

Burki

na F

aso

(2010

-2015

)

Mad

agas

car (

2013

-2015

)

Sier

ra L

eone

(2013

-2017

)

Gam

bia

(2011

-2015

)

Bangl

ades

h (2

011-2

016)

Uga

nda

(2012

-2016

)

Mal

awi (

2009-

2011

)

Tanz

ania

(2012

-2016

)

Nep

al (2

013-2

017)

Indo

nesia

(2011

-2017

)

Cha

d (2

014-2

018)

Hai

ti (2

013-2

017)

Benin

(2012

-2015

)

Gua

tem

ala

(2012

-2015

)

Nig

er (2

012-2

015)

0

20

40

60

80

100

Source: SUN Movement CRF Planning Tool

Landscape Study of the Cost, Impact, and Efficiency of Above Service Delivery Activities in HIV and Other Global Health Programs 37

tracks expenditures within various program areas. A

sample NHA profile of expenditures for nutrition is

illustrated in Table 4.

We found that the quality of expenditure tracking data

for nutrition within the NHAs is inconsistent; indeed,

many NHAs do not include expenditure figures for

nutrition. Countries for which NHA data on nutrition

are available may not include sufficient granularity to

indicate an expenditure split between ASD and service

delivery costs, as demonstrated by the data in Table

5. The implementation of the WHO’s 2011 System of

Health Accounts aims to standardize reporting such

that these granular data are available for all countries'

NHAs moving forward (OECD et al., 2011).

In instances in which NHA data on ASD activities

are available, ASD expenditures that are captured

are included within the category, “Governance and

health system and financing administration.” Across

different countries' NHA reports this cost category

varies substantially as a share of total program cost,

from 1 percent in Burkina Faso (2011) to 60 percent

in Niger (2011).

What is known about the impact/efficiency of nutrition ASD activities?

In our review of the relevant literature and in interviews

with informants, we found that little research has been

conducted to understand the relationship between

and impact of ASD spending and efficiency of nutrition

programs, and that there is a resulting lack of relevant

data. While we found some research on the impact of

nutrition-sensitive interventions within water, sanitation,

and hygiene (Dangour et al., 2013) and agriculture

(Berti, Krasevec, & Fitzgerald, 2004), these activities

generally are not considered to be ASD activities and

more aptly are considered ancillary activities (cf. HIV

supportive activities, like OVC and social protection).

We posit that the lack of analysis in this space is

explained by a number of factors. Our informants

28 Niger’s governance spending data are conspicuously high in 2011 (60 percent of total), especially given that governance spending only accounted for 28 percent of program costs in 2013. Prior to 2012, Niger’s nutrition NHA portfolio only reported expenditures across two lines, “Governance and health system and financing administration” and “Curative care.” In 2011, Niger joined the SUN Movement, and, in 2012, the government implemented a multi-sectoral initiative designed to provide a central coordinating body for nutrition planning and programming. As a result, Niger’s total expenditure for nutrition increased from 8 million USD in 2011 to 21 million USD in 2013, and the NHA expenditure tracking included spending data for 7 additional lines.

Table 4. A sample NHA profile of expenditures for nutritional

deficiencies by use function (Niger)

2013 2012 2011

Curative care 35% 38% 40%

Medical goods 28% 31%

IEC programs 0%

Immunization programs 1% 0%

Early disease detection programs

0%

Healthy condition monitoring programs

1% 2%

Epidemiologic surveillance and risk & disease control programs

5% 1%

Unspecified preventative care 2%

Governance and health system and financing admin

28% 28% 60%

TOTAL 100% 100% 100%

Source: SHA, WHO

Table 5. NHAs that include expenditure tracking for nutrition rarely include tracking for ASD activities (e.g., "Governance, health

system, and financing administration")

2010 2011 2012 2013

Benin

Burkina Faso 1% 3% 6%

Burundi 6%

Cambodia

Cameroon

Cote d’Ivoire

DRC 3% 6%

Ethiopia

Haiti

Kenya 22%

Mauritania

Myanmar

Niger 60% 28% 28%

Philippines

Seychelles 2%

Uganda

Source: SHA, WHO

38

indicated—and our desk research affirmed—that

nutrition costing and expenditure tracking remains

an incipient field, with a developing methodology

that is still coalescing around standardized practices

implemented at scale. Many of our informants noted

that it is difficult to conduct a rigorous analysis of

ASD costs when there is a profound lack of basic

nutrition cost and spending data. Within the nutrition

community, there is an ongoing effort to improve

costing data and expenditure tracking, which we

discuss further in the following section.

Our informants strongly held that ASD activities are

essential for strong nutrition planning, which echoes

advocacy at the global level. However, our informants

also noted that members of the nutrition programming

community think that reducing ASD inefficiencies may

not be as mission-critical for nutrition as for other areas

of programmatic spending. Informants expressed the

opinion that nutrition was not as “top heavy” as other

program areas—the field generally is under-resourced

and lacks the developed architecture of HIV/AIDS

and immunization programs—and, thus, the greatest

efficiency gains that could be leveraged to improve

service delivery likely are at the point of service delivery.

Many of our informants noted that the nutrition field

suffers from gross under-investment at present;

because the field

receives limited funding,

they think that it is

possible that there are

not substantial areas of

waste above the point

of service delivery.

Our informants also

observed that the

lack of visible data

may contribute to the

perceived unimportance

of nutrition-related

ASD costs, and that

it is possible that the

share of program

expenditures dedicated

to ASD activities is

underestimated and

underreported.

One observation on

the impact of nutrition

ASD activities that

emerged from both our

informant interviews and our desk research is worth

highlighting: given the multisectoral nature of nutrition

programs, programs’ accountability would improve

and redundancies in their efforts would decrease if

they were coordinated by a central office. Informants

pointed to Ethiopia’s nutrition programs, which are

run through the Prime Minister’s office, as an example

of centralized programming under the leadership

of a central authority. The SUN report on countries’

experiences echoed this observation: “Countries that

have clearly identified functions for coordination and

management of nutrition appear to have been better

positioned in identifying the required activities and

costs” (SUN Secretariat, 2014).

What current, on-going, or planned efforts will improve the knowledge base and efficiency of nutrition ASD costs?

Efforts to increase knowledge about nutrition costing

are in development; however, given the paucity of

data about basic nutrition costing, we do not expect

these efforts to focus on ASD costs, activities, and

efficiency. A technical working group convened by

Strengthening Partnerships, Results, and Innovations

in Nutrition Globally (SPRING), the SUN Movement,

Figure 24. Program costs by activity category for SUN costed national plans for 11 countries in sub-Saharan

Africa, with nutrition-sensitive costs excluded

Rwan

da (2

012)

Keny

a (2

013-2

017

Tanz

ania

(2012

-2016

)

Gam

bia

(2011

-2015

)

Sene

gal (

2013

-2017

)

Sier

ra L

eone

(2013

-2017

)

Uga

nda

(2012

-2016

)

Mad

agas

car (

2013

-2015

)

Nig

er (2

012-2

015)

Burki

na F

aso

(2010

-2015

)

Moz

ambi

que

(2011

-2015

)

Benin

(2012

-2015

)

Mal

awi (

2009-

2011

)

62%

42%

36%33%

27% 27%

21% 21%

16%

12%10%

7%

26%

Source: SUN Movement CRF Planning Tool

Landscape Study of the Cost, Impact, and Efficiency of Above Service Delivery Activities in HIV and Other Global Health Programs 39

and R4D is working to harmonize methods for

budget and expenditure analysis for nutrition. The

growth of the WHO NHA repository grows should

increase the data on public expenditures for nutrition

that are available. Public Expenditure Reviews (PERs)

for nutrition may also be forthcoming. In 2014,

Tanzania became the first country to execute a PER

for nutrition, opening the possibility for additional

PERs for nutrition in the future.

What are the key gaps in the nutrition ASD knowledge base?

We found that researchers need reliable,

comprehensive data to assess the unit costs of basic

nutrition interventions. The lack of availability of these

data limits policy makers’ and advocates’ abilities

to make compelling arguments for continuing and

increasing investment in nutrition.

Although researchers and policy makers have

considerable interest in the costs of ASD activities

associated with nutrition programs, a priority within

the field is understanding total nutrition expenditures.

In some countries, nutrition activities are coordinated

by ministries of health, agriculture, sanitation,

environmental affairs, finance, food, land, social welfare,

and infrastructure. In

instances in which

nutrition programs

are spread amongst

numerous ministries

and sectors, researchers

have difficulty obtaining

comprehensive

spending data. A recent

nutrition budget analysis

for FY 2014-15 for the

government of Nepal,

presented in Figure 25

below, indicated that

on-budget allocations

for nutrition spanned six

ministries, highlighting

the challenge faced by

researchers in obtaining

comprehensive data for

domestic financing for

nutrition.

Research studies on

the economies of

scale for nutrition interventions will aid health planners

in estimating the impact, costs, and efficiencies of

increasing the reach of nutrition interventions. In

addition, as nutrition interventions reach a saturation

point in certain areas, greater research will be needed

on the impact and efficiency of efforts to expand

coverage to hard-to-reach populations (so called “last

mile” efforts).

We found in the literature review that high-cost

commodities, especially those used to treat and

manage acute malnutrition, are among the main

cost drivers of nutrition programs. Within CMAM

programs, ready-to-use therapeutic foods (RUTFs)

often account for 25 to 40 percent of program

costs (Puet et al., 2012; Wilford et al., 2010; Horton

et al., 2010; Bachmann, 2009). We anticipate that

additional research in improving the efficiency of the

procurement and manufacture of common nutrition-

related commodities will indicate ways to improve

efficiencies and potentially lower the unit costs of

nutrition interventions and unlock savings at the point

of service delivery.

Overall, we found that a stronger empirical knowledge

base of ASD and service delivery costs in nutrition

alike is needed. Countries and donor organizations

should expand expenditure tracking for nutrition, such

Figure 25. Budgeted nutrition allocations in Nepal span six ministries (NPR thousands, 2013-2014)

Ministry ofFederal A�airs

and LocalDevelopment

Ministry ofAgricutural

Development

Ministry ofUrban

Development

Ministry ofHealth andPopulation

Ministry ofEducation

Ministry ofWomen,

Children andSocial Welfare

Total

7,775,829

958,662

733,103

938,647486,537 2,951 10,895,729

Source: SPRING, 2015

40

as through Public Expenditure Reviews or the WHO’s

NHA system. With expanded data, it may be possible

for researchers to fill the more specific knowledge gaps

of ASD activities. At present, if researchers consider

ASD costs in nutrition cost analyses, they often either

estimate or apportion costs by anecdotal experience.

To the extent that a more detailed sense of ASD and

service delivery expenditure split emerges, researchers

will be able to advance more refined analyses of the

costs, impact, and efficiency of ASD activities.

Family Planning

What data are publicly available for family planning ASD costs?

We found two major sources of ASD cost data

for family planning (FP) programs that are publicly

available. However, both of these sources provide the

incremental costs of meeting specific coverage and/or

outcomes targets, not the full cost to the health system.

As such, these data are not easily comparable with the

data in other program areas, and must be interpreted

particularly carefully.

First, since 2012, USAID’s Health Policy Project (HPP,

2015) and Futures Group (now Palladium Group) have

helped 15 countries to produce Costed Implementation

Plans (CIPs) to expand and improve FP services

(Zlatunich & Reeves, 2015) guided by national targets

tied to FP2020 (n.d.), a global partnership to promote

universal access to contraceptives. The CIPs contain

high-level breakdowns of costs for both service delivery

and ASD activities, while the underlying costing models

provide detailed ingredients-based costs for all activities.

As Table 6 indicates, 15 of the 16 plans are published

online, while full costing models are currently available

for only 4 countries: Ghana, Malawi, Uganda, and

Zambia.

Second, through its Adding It Up series, the Guttmacher

Institute29annually publishes global investment needs

to achieve universal coverage of essential sexual and

reproductive health services, including contraceptives

(Singh et al., 2014). The Guttmacher Institute includes

ASD activities in the estimates of “program and system

costs,” which we discuss in the next sub-section, but

AIU reports them only at the global level.

We also searched the peer-reviewed literature

and found minimal information about ASD costs

or spending for FP programs. We found only two

relevant studies published since 2010, neither of

which retrospectively examined FP-related ASD costs

or spending. One, a cost assessment of integrating

sexual and reproductive health and HIV services in

Kenya and Swaziland, focused exclusively on facility-

level spending (Warren et al., 2012). The second, a

facility-centric cost analysis of youth-friendly sexual

and reproductive health services in Moldova, used a

conventional ingredients-based approach to estimate

service delivery costs, which then underpinned

resource-needs estimates for scale-up (Kempers et al.,

2014). The authors excluded the largest facility when

they extrapolated scale-up needs because its budget

included a number of system-level services. In other

words, the study’s authors deliberately sidestepped

analyzing the costs of ASD activities.

Finally, we found no repositories of ASD spending

data for FP. Researchers might be able to estimate

such spending using National Health Accounts data.

However, FP-related activities are not neatly categorized

in the System of Health Accounts (OECD et al., 2011).

Analysts would have to extract data pertaining to non–

service delivery spending through multiple healthcare

Table 6. Availability of Costed Implementation Plans and underlying costing models

CountryYears covered

by CIPCIP

availableCosting model

available

Benin 2014–2018 X

Cameroon 2015–2020 X

Côte d’Ivoire 2015–2020 X

Ethiopia Under government review

Ghana 2016–2020 X X

Guinea 2014–2018 X

Malawi 2016–2020 X X

Mali 2014–2018 X

Mauritania 2014–2018 X

Myanmar 2014–2018 X

Niger 2012–2020 X

Nigeria 2014–2018 XForthcoming

(2016)

Togo 2013–2017 X

Uganda 2015–2020 X X

Zambia 2013–2020 X X

Sources: HPP (2015) and FP2020 (2015)

29 Accessible at https://www.guttmacher.org/.

Landscape Study of the Cost, Impact, and Efficiency of Above Service Delivery Activities in HIV and Other Global Health Programs 41

functions and providers, ranging from population-

level public health activities, like demand creation for

condoms, to FP counseling provided as part of routine

primary care, to complex obstetric services provided in

hospitals. The fact that FP services are often delivered

together with general primary healthcare, as part of

broader maternal and child health programs, or in

conjunction with programs addressing HIV and other

STIs, makes it difficult for analysts to allocate any ASD

spending (or non-variable service delivery spending)

specifically to FP. Additionally, we found no detailed

data on the expenditures of donor-funded programs

focused exclusively on FP interventions.

What are FP ASD activities and what share of program spending do they comprise?

There is no standard taxonomy of ASD activities for FP.

The FP CIPs all include slightly different activities

based on the respective country’s priorities and

needs. Most include at least a subset of the following

ASD cost categories:

• Demand creation

• Contraceptive security

• Policy and enabling environment

• Financing

• Health workforce

• Quality and safety

• Stewardship, management, and accountability

The CIPs also cost commodities and “service delivery

and access.” In some CIPs the latter includes both

service delivery and ASD activities, but they are not

easily disaggregated in the documents. For our

descriptive analysis of the data below, we consider

any costs labeled as “service delivery and access” to

be service delivery-related costs.

Figure 26 below depicts the allocation of projected

ASD and non-ASD (commodities and service delivery)

needs for the 13 CIPs published online. On average

the ASD share is 34 percent and varies considerably,

ranging from 19 percent in Niger to 56 percent in

Mauritania. We posit that the variation derives from

numerous factors. First, each country approaches

its planning from different initial coverage levels,

different capacities and baseline spending at both

service delivery and ASD levels, and with different

goals and timelines.

Second, although we

might expect that ASD

share for incremental

investments needed in

family planning is related

to the current level of

unmet need, we did

not find a consistent

relationship between

ASD share and unmet

need in the data. The

relationship between

modern contraceptive

prevalence rate (mCPR)

and ASD share is mildly

negative,30 which is

consistent with our

hypothesis that ASD

costs might increase

more slowly than service

delivery costs as FP

coverage expands (e.g.,

due to scale effects).

The data fit for this

Figure 26. Allocation of projected FP spending needs for ASD activities

Linear (Baseline mCPR)

ASD Activities

56%

48%

40%

36% 35% 34% 33%31% 31% 31%

26%24%

19%

26%25%

15%11% 15%

10% 9%7%

13%

40%42%

22%

16%

Gha

na (2

015)

Mal

awi (

2015

)

Mya

nmar

(2014

)

Togo

(201

3)

Mal

i (20

14)

Gui

nea

(2014

)

Nig

eria

(2014

)

Uga

nda

(2013

)

Zam

bia

(2013

)

Burki

na F

aso

(2010

-2015

)

Benin

(2013

)

Nig

er (2

012)

Uga

nda

(2012

-2016

)

Source: R4D analysis using data from the 13 CIPs posted to the Health Policy Project and FP2020 websites

30 Based on the overall pattern of these data, as baseline mCPR increases by three percentage points, ASD share decreases by one percentage point.

42

hypothesis is not strong, and we recognize that

alternative explanations for the observed effect are

possible.

Third, relative prices may vary from one country

to another. For example, a month of work by

an IT expert in Malawi (ASD) costs about 10,000

times what a single IUD costs (service delivery)

(Government of Malawi, 2015). In Ghana, however,

the ratio is around 4,000 to 1 (Government of Ghana,

2015). Given that these countries have similar ASD

shares, such price differences could indicate that

they are getting different amounts of ASD activities

for their money. Fourth, some CIPs may rely more

on external technical support than others, which

typically is more expensive than domestic expertise.

Ultimately, to explain the observed variation, we

would have to analyze intensively all 15 CIPs and

their underlying cost models, most of which are not

publicly available.

In contrast to the CIP cost taxonomy, Adding It Up

disaggregates costs into direct and indirect costs,

the latter of which includes a slew of “program

and system costs” that fall into several categories

(Weissman, 2013). Table 7 summarizes the

subcategories of indirect costs.

Adding it Up reports global estimates only for indirect

costs, not for individual program and system costs.

Figure 27 above depicts the distribution of costs across

contraceptive supplies, health worker salaries, and

program and system costs

under current coverage levels,

Adding it Up’s estimates for

improved care for those

currently covered, and

universal contraceptive

coverage. The global cost

of universal coverage is

estimated at 9.4 billion USD,

of which 6.0 billion USD (64

percent) covers program and

system costs.

Unfortunately, we found it

impossible to determine the

ASD share without data on

each of the activities listed in

Table 7 below. We found it

difficult to parse the methods

underpinning Adding it Up’s

estimates of indirect costs. Adding it Up relies on

regional indirect cost ratios produced by the United

Nations Population Fund (UNFPA, 2009), which in

turn are based on adapted estimation techniques

developed by the World Health Organization for

maternal and child health (WHO, 2005). On what basis

UNFPA determined the expected spending levels on

program and system costs remains opaque even to

Adding it Up’s authors; for instance, they may or may

not derive from real data on the costs of program

management and system investments. Even with

greater transparency, the reliance on regional averages

makes it unlikely that the cost estimates are precise at

the country level.

Table 7. Categories of projected indirect costs for FP in Adding It Up

Program Costs System Costs

Program management

Supervision

Health education

Advocacy

Monitoring and evaluation

Infrastructure

Transport and communication

Human resources (pre- and in-service training)

Logistics and supply chain

Health information systems

Leadership and governance

Health financing

Research, data, and policy analysis

Source: Weissman, 2013

Figure 27. Costs of meeting demand for modern contraception under current coverage, improved care, and universal coverage scenarios

Supplies

Health worker salaries

Program and systems costs

Current level of care

Improved care for current

users

100% of need for modern

methods met

1.3 0.7 2.1 $4.1

1.3 0.7 3.3 $5.4

2.3 1.2

Costs in 2014 USD (in billions)

6.0 $9.4

Source: Reproduction of Figure 2.3 in Singh et al., 2014

Landscape Study of the Cost, Impact, and Efficiency of Above Service Delivery Activities in HIV and Other Global Health Programs 43

What is the relationship between ASD spending and overall program efficiency?

We found little research into the relationship

between ASD spending and FP programs’ efficiency.

We found a few studies on the effectiveness of

specific non-service delivery interventions like social

and behavioral change campaigns, but neither

the literature nor interviewees pointed toward

any data that enable researchers to estimate how

much countries should be spending on various

ASD activities or on what the marginal impact of

increasing or decreasing expenditure would be.

Instead, there is some sense among experts that

service quality lacks in part because of severe

underinvestment in ASD activities like supervision

and M&E. If donors/programs work to track ASD

spending in FP programs, these data likely will

validate or correct this perception.

What current, on-going, or planned efforts will improve the knowledge base of ASD costs and their efficiency?

Along with country-level CIPs, HPP and FP2020 have

developed a CIP resource kit31 that includes tools

and resources for a rigorous three-phase approach

to planning, developing, and executing CIPs. These

entities are still developing a tool to monitor the

performance of CIPs for results and to better track

financing32 and outcomes. We recommend that

such a tool include ASD activities in its taxonomy for

expenditure tracking.

What investments in research and development should be prioritized to expand knowledge of ASD costs and improve the efficiency of ASD activities?

We found that there are several promising avenues

for FP-related investments. First, FP experts agreed

that more and better data on how much money is

spent on FP—at all levels of the health system—will

improve experts’ decision-making and research.

In general, they argued that the near-term priority

should be to address the paucity of cost and

expenditure data at the service delivery level rather

than ASD. However, they recognized that researchers

could design studies to collect both types of data

concurrently. At the very least, we recommend

that any existing cost models should be published

if possible, including those underpinning all of the

CIPs. Retrofitting those models for easier cross-

country comparability would also be valuable.

Second, FP lacks a broadly accepted taxonomy

for program costs. Unless leading institutions

in FP costing adopt a uniform taxonomy, data

collection and analysis will remain unstandardized

and researchers will have difficulty comparing

results across settings and over time. Consequently,

organizations such as FP2020 and the Guttmacher

Institute should initiate an effort to develop and

disseminate a widely applicable taxonomy that

clearly defines cost categories, and is aligned with

similar efforts in other program areas.

Third, researchers estimating ASD resource needs for

FP have focused mainly on scale-up costs rather than

recurrent costs. In fact, longer-term projections may

underestimate how much money will be required

to sustain ASD activities after basic capacity is built.

Consequently, we anticipate that costing studies in

countries that have more advanced FP programs

could better calibrate long-term resource planning

and advocacy globally.

Finally, given the substantial unmet service delivery

needs in FP in many countries (Singh et al., 2014),

experts felt that while additional data collection

and research are valuable, such efforts do not merit

diverting significant resources away from service

provision. Rather, experts prefer to see smaller

investments that yield decent data and insights into

ASD spending and its impact, rather than those to

that result in a perfect accounting of FP spending at

the ASD level.

31 Accessible at http://www.familyplanning2020.org/microsite/cip.32 USAID’s DELIVER Project already offers a tool to track contraceptive financing: http://deliver.jsi.com/dlvr_content/resources/allpubs/guidelines/

EnhaCSFin.pdf.

44

Cross-Program Focus Topics in Above Service Delivery Costs

et al., 2015). Both processes move essential drugs

and supplies from manufacturers to patients. To

disaggregate the costs of procurement and SCM

activities incurred at the facility from those above

the point of service creates an additional layer of

complexity, especially given the heterogeneity of

categorization in cost data.

We found, overall, that there is extensive literature

documenting actual health supply chain costs

(Mvundura et al., 2015; Shretta et al., 2015), as well

as modeled costs (Portney et al., 2015) in a variety of

settings. However, the level of publicly available SCM

data that are disaggregated by above and below site

costs is low. A substantial amount of procurement

cost data is available, primarily because a large share

of procurement is the cost of commodities (generally

categorized as facility level costs), which are often

known and public. Data on individual procurement

costs are available through the Global Fund’s price

and quality reporting system and voluntary pooled

procurement system, the PAHO Revolving Fund, and

from UNICEF Supplies and Logistics.

Ultimately, we conclude that an accurate idea of

ASD-SCM and procurement costs requires national

cost data disaggregated by service delivery level and

activity, and a clear understanding of when costs

for these two mechanisms are disaggregated from

each other or combined. We found four platforms

that report data of this nature, each of which has

advantages and limitations: (1) System of Health

Accounts (SHA); (2) peer-reviewed costing studies; (3)

published reports of donor projects; and (4) modeled

data. We discuss the four platforms in turn below.

National data sets

We have described National Health Accounts data

in other chapters of this report in reference to the

sub-accounts for different health program areas. An

additional feature of the System of Health Accounts

2011 (OECD et al., 2011) methodology is relevant

to SCM costs: the categorization of expenditures

by financing agent, the entities that manage and

In this section, we discuss two cross-program topics:

Supply chain and procurement, and Laboratory

operations. The topics are of particular interest due

to their substantial cost, the emphasis placed upon

them by major donor organizations, the tangibility of

their performance metrics, clear logical connection

with quality of care, and demonstrated methods for

improvement. In our interviews with stakeholders,

interviewees regularly raised these topics as the most

relevant activities above the point of service delivery

to evaluate for potential performance improvement

and cross-program learning.

Supply chain and procurement

What data on ASD costs in supply chain and procurement are publicly available?

We found that it is challenging for researchers

to develop complete and accurate costs of

pharmaceutical procurement and supply chain

management activities due to the large scale

and complex nature of most of these systems.

Pharmaceutical procurement is defined as the

country-level process of ordering drugs and/or

commodities. Costs related to procurement include

both (a) the costs of goods procured (e.g., drugs,

supplies, and equipment), which are often consumed

at the point of service delivery, and (b) the costs of

procuring those goods (e.g., needs assessments,

product selection, issuing tenders, quality assurance,

and monitoring the procurement process), which are

typically incurred at a level above the point of service

delivery. Supply chain management (SCM) is the

mechanism by which these products are delivered

to healthcare facilities. Costs for supply chain include

(a) transportation (including amortization for vehicles,

fuel) (b) storage (including warehousing/storage

and amortization for storage equipment, inventory

management, maintenance, energy costs and (c)

labor (including per diem and labor costs) (Portney

Landscape Study of the Cost, Impact, and Efficiency of Above Service Delivery Activities in HIV and Other Global Health Programs 45

channel funds provided by financing sources and use

those funds to pay healthcare providers or purchase

healthcare activities.

NHAs using the SHA 2011 methodology provide

information on the financing source and amount

allocated to supply chain financing agents, for

example, the Pharmaceuticals Fund and Supply

Agency (PFSA) in Ethiopia. The PFSA was formed

specifically to address timely procurement and

distribution of pharmaceuticals to health facilities

in Ethiopia. NHA data provide information on total

financial flows to agents such as the PFSA, and

indicate the direction of expenditures to providers

and functions. In this sense, NHA data offer a relevant

starting point to calculate ASD costs. However,

these expenditures are often tracked as aggregated

disbursements to government administrations of

health providers for general administrative activities.

There is not sufficient granularity within the provider

categories to enable researchers to disaggregate

data by service delivery level, or within the function

categories to disaggregate supply chain-specific

activities from general health administration activities.

The data are often insufficient for researchers to

determine specific ASD-supply chain expenditures.

Lastly, actual expenditures like the SHA 2011 are

crucial to understand resources within a country;

however, costed projections are critical to

understand the gap between actual resources and

the ideal needs/demands of a fully functional system.

Costing studies published in peer reviewed journals

We found recent literature on costs of SCM for HIV,

contraception, and immunization (among others), but

these studies generally did not disaggregate by ASD

and facility level costs. A 2014 systematic review by

Siapka et al., examined published and grey literature

from 1990 to 2013 on costing and efficiency data

for the six basic programs of the UNAIDS Strategic

Investment Framework. The review showed that

among 82 included studies, degree of reporting of

ASD costs varied and was always only partial. Most

studies included costs for training and supervision,

but they rarely included typical ASD costs such as the

maintenance of the drug supply chain, transportation,

and technical support (Siapka et al., 2014).

The EPI Costing and Financing studies (EPIC)

published in 2015 exemplify high quality publicly

available data on ASD costs. We describe the results

from the EPIC studies concerning SCM in greater

detail below.

Costing studies published in donor reports

We found that another rich source of supply chain

costing data is published donor reports, such as

the USAID-Deliver project and its costing of the

contraceptive logistic management system (CLMS) in

Nigeria. In Table 8 below we present deliver-project

supply chain costs, by tier, in Nigeria. The study

Total estimates of supply chain costs, by tier, for Nigerian contraceptives logistics management system (USD)

Function Central StateLocal government

areaService delivery

pointTotal

Procurement 67,952 940 23,578 102,241 194,710

Storage 47,999 60,972 118,188 863,490 1,090,649

Transportation -- 46,314 494,865 311,464 852,643

Management 2,057 81,317 343,562 350,315 777,251

Total systems costs 118,008 189,543 980,192 1,627,511 2,915,254

Source: Hasselback et al., 2012

46

included the following indicators (Hasselback et al,

2012):

• Supply chain costs as a percentage of the total

value of commodities

• Supply chain costs per USD value, volume, or

weight of commodities

• Costs by tiers and functions

We found that one limitation of these donor

sponsored costing studies is their vertical nature.

Outside of national data surveys like the SHA 2011,

supply chain activities and their financial analyses

often tracked are via parallel systems depending on

the commodity (vaccines, HIV-drugs, family planning

etc.) and the funder (donor, government, etc). Some

amount of segmentation may be necessary based

on differences in demand variability, criticality to

patients, costs, etc., of different products, which

influences forecasting, inventory management, and

logistics for the different chains. However, to get an

accurate picture of total ASD-costs for procurement

and supply chain, researchers must aggregate data

from each supply chain with all costs for national

procurement and distribution systems, such as

electronic logistic management information systems

(eLMIS). To date, researchers have not completed an

analysis of this nature.

Modeled savings

We found that research on modeled savings on

supply chain costs can support analyses of potential

future costs when actual data are not available.

However, in line with other published literature on

the topic, we found that many cost modeling studies

do not disaggregate ASD from facility level costs. A

2015 study by Dutta et al. that modeled the financial

gap resulting from scaling up ARV therapy in 97

countries from 2015 to 2020 excluded programs’

ASD costs, but cited the exclusion as one of the main

limitations of the study. Furthermore, the authors

defined the price of ARVs as “ex works” prices that

did not account for the costs of transportation or

storage in country (Dutta et al., 2015). Another large

modeling study by Portney et al., 2015 modeled

the costs of vaccine programs across 94 low- and

middle-income countries. The study examined

both procurement (including vaccine costs) and

supply chain as discrete cost categories, and

generated country specific cost models for current

and projected vaccine regimens using HERMES

(Highly Extensible Resource for Modeling Supply-

chains) for four reference countries (Benin, Niger,

Mozambique (Gaza Province), and India (State of

Bihar)). The authors then used data from the four

Figure 28. Total immunization program costs by component and by routine vs. SIA (2011-2020)

0

1,000

2,000

3,000

4,000

5,000

6,000

7,000

8,000

20

10 U

SD (

in m

illio

ns)

SIA operational costs

SIA vaccine costs

Routine service delivery costs

Routine supply chain costs

Routine vaccine costs

2011 2012 2013 2014 2015 2016 2017 2018 2019 2020

Source: Portney et al., 2015

Landscape Study of the Cost, Impact, and Efficiency of Above Service Delivery Activities in HIV and Other Global Health Programs 47

reference countries to extrapolate results to the

remaining 90 countries in the analysis. Figure 28

above provides the results on costs for components

for all 94 countries. The study differentiated between

service delivery costs and SCM, but did not explicitly

categorize costs by service delivery level. The

information provided is an appropriate starting place

to estimate ASD-SCM costs, but is not sufficient in-

and-of itself.

What are procurement and SCM ASD activities and what share of program spending do they comprise?

Activities

Supply chain activities include all activities needed to

ensure products are delivered to healthcare facilities.

By their very nature, the majority of SCM activities

occur above the point of service. While the level at

which healthcare programs allocate costs for these

activities is context specific and depends on the

organization of the health system, the EPIC study

identified drug collection, distribution, and storage as

ASD cost drivers at the regional and central level (Le

Gargasson et al., 2014). Procurement activities can

be more difficult to identify: researchers and officials

often bundle procurement commodity costs into

the unit cost of commodities at the facility level, and

other ASD procurement activities are not well defined.

Proportion of program spending

Regarding supply chain management, the USAID-

Deliver analysis in Nigeria cited storage and

transportation as supply chain system drivers at 37

percent and 29 percent of total supply chain costs

respectively (Hasselback et al., 2012). As a whole,

the EPIC initiative showed that facility level costs

for service delivery, specifically labor and vaccine

costs, and not ASD supply chain or procurement

costs, were the cost drivers of routine immunization

programs across six countries. On average, 85

percent of costs were accounted for at the facility

level, with 15 percent of costs incurred above the

point of service delivery (ranging from 6 percent

in Benin to 26 percent in Zambia). Moldova, which

had the highest immunization coverage, had ASD

costs of 18 percent. (Brenzel et al., 2015). We provide

country specific data for the following proportions in

Table 9 below:

• The percentage of activity costs that are incurred

above the point of service (percent of ASD activity

cost incurred);

• The proportion of total ASD costs accounted for

by each activity (ASD activity cost as percent of all

ASD program costs);

• The proportion of all costs (facility and ASD)

accounted for by each activity (ASD activity cost

as percent of all program costs).

The data in Table 9 illustrate that, on average

(using EPIC data from Benin, Ghana, Uganda,

and Zambia), 21 percent of vaccine collection,

distribution, and storage costs and 17 percent

of cold chain maintenance occurred above the

facility. Furthermore, the cost of vaccine collection,

Table 9. EPIC study data for four sub-Saharan African countries for cold chain maintenance and vaccine collection, distribution, and storage

Vaccine Collection, Distribution, and Storage Cold Chain Maintenance

Benin Ghana Uganda Zambia Benin Ghana Uganda Zambia

% of activity cost incurred ASD

19% 29% 7% 30% 19% 16% 13% 21%

Activity ASD cost as % of all ASD costs

36.6% 10.1% 10.6% 10.4 1.4% 2.5% 8.3% 3.1%

Activity ASD cost as % of all program costs

11% 8% 7% 9% 0% 2% 5% 3%

Source: EPIC studies

48

distribution, and storage contributed more to both

ASD and total costs than cold chain maintenance.

Neither activity, however, accounts for a large

proportion of total program costs.

What is known about the impact/efficiency of supply chain ASD activities?

Given the scale of procurement and supply chain

costs (including commodities, 5 billion USD in

anticipated investments by the Global Fund alone

over the next three years) even minor efficiency

gains could lead to huge savings. We found in the

literature that SCM and procurement are perceived

overall as potential areas in which programs can

improve their efficiency and incur cost savings. A

2014 systematic review by Hinrichs et al. examined

72 studies on efficiency savings in procurement and

supply chain management approaches for clinical and

non-clinical goods. The review found that collective

approaches to purchasing, improving relationships

with suppliers, building capabilities and skills for

purchasing decisions, and using technology for

data and materials management may lead to more

efficient procurement and potentially save costs.

However, the study’s results showed that empirical

evidence demonstrating gains from these approaches

was scarce and, when available, tended to be weak in

design and execution.

We found extensive grey literature on strategies to

increase operational efficiency in the supply chain.

For example, level jumping, the process of eliminating

mid-level/district storage facilities and instituting

direct delivery to facilities to increase efficiency, has

shown promising results for cost effectiveness and

efficiency in Mozambique. The intervention used

in Mozambique, called dedicated logistics system

(DLS), combined level jumping, task-shifting, data use,

optimized transport loops, and supportive supervision.

After 4 years, results from an impact evaluation

showed a 27 percent increase in the diphtheria,

pertussis, tetanus (DPT)-hepatitis B3 vaccine coverage

rate and a decrease in the percentage of health

centers reporting a stockout—from 80 percent to 1

percent (Hasselback et al., 2012). Similarly, vendor

managed inventory, the process by which the vendor

takes full responsibility for the agreed inventory,

showed savings of 6.6 million USD per year in

Thailand due to reduction of un-opened vaccine

wastage (Rievpaiboon et al., 2015). Lastly, informed

push systems in Zimbabwe showed significant

improvements in performance, including reducing

stock out rates from 20 percent to 2 percent (Sarley

et al., 2010). Informed push adapts the principles used

in commercial sector distribution by utilizing teams of

trained staff to visit health facilities, review inventory,

and restock shelves from trucks.

Metrics for Procurement and SCM

To identify and implement efficiency gains in

procurement and SMC, decision-makers must

understand how a better supply chain would result

in such gains. To do this, programs must establish

metrics to monitor performance across the spectrum

of activities. A 2010 report by the John Snow Institute

(JSI) outlined key performance indicators (KPI) for

supply chain groups, including quality, response time,

cost/financial, and productivity across the following

five categories (Aronovich et al., 2010):

1. Product selection, forecasting, and

procurement;

2. Supplier/sourcing;

3. Warehousing/storage;

4. Inventory management/LMIS/Customer

response; and,

5. Distribution/Transport.

Similarly, JSI outlined performance indicators for

procurement cost, quality, timeliness, systems

productivity, and integrity. Costs and productivity

indicators included the following:

1. Product Price Variance: Percentage price

variance between contract unit price and

international unit price for focus products;

2. Effective Contract Utilization: Percentage

by value of purchases made under simple

purchase orders, annual contracts, and

multiyear contracts;

3. Emergency Procurement: Percentage, by value

and number, of purchase orders or contracts

issued as emergency orders;

4. Procurement Cost: Ratio of annual

procurement unit cost-to-value of annual

purchases; and,

5. Staff Training: Key training program

components are in place and the percentage of

staff who receive training annually.

Landscape Study of the Cost, Impact, and Efficiency of Above Service Delivery Activities in HIV and Other Global Health Programs 49

We found that effective performance measurement

for SCM is complicated by a lack of focus on

driver metrics (Ebel et al., 2013) and impact

indictors as opposed to output indicators. We

found that program staff do not systematically

measure or manage structural drivers including

responsiveness (e.g., replenishment lead time),

manufacturing frequency, supply reliability (e.g.,

schedule adherence), and stability (share of rush

orders, planning accuracy). Researchers also face

a lack of available central data and “apple to apple”

benchmarks that would enable meaningful targets

based on these performance indicators (Ebel et al.,

2013).

What current, on-going, or planned efforts will improve the knowledge base of ASD costs and their efficiency?

We found that the following efforts are underway

to improve the knowledge base of ASD costs and

develop standardized costing tools for supply chain

and procurement:

1. Supply Chain Costing Tool (SCCT), an

activity-based approach developed by the

USAID/DELIVER PROJECT to standardize

measurement and analyze costs. The SCCT

captures and organizes supply chain costs

into four functions (procurement, storage,

transportation, and management) by

organization, and by each level or tier (central,

regional, facility level, etc.) of the supply chain.

2. HERMES (Highly Extensible Resource for

Modeling Supply-chains), a computational

framework for modeling and optimizing supply

chains.

3. Guide to Public Health Supply Chain Costing:

A Basic Methodology. Arlington, Va.: USAID/

DELIVER PROJECT, Task Order 4.

We conclude that more work needs to be done

to ensure that these tools include standardized

methodologies to disaggregate ASD activities as

discrete categories.

Gaps in knowledge and recommendations to improve procurement and supply chain

In general, we found that the procurement and

supply chain topic has been a more tractable area

of investigation for researchers, in part because

of efforts such as the partnership for supply chain

management, the USAID/Deliver project, and

research efforts funded by BMGF. As such, this is

an area in which researchers have identified strong

approaches for improvement that can be applied in

appropriate country settings, such as level jumping,

optimized transportation loops, informed push

systems, vendor managed inventory, and eLMIS

systems. In procurement, strides have been made

at the Global Fund and elsewhere to bring private

sector procurement principles into public sector

bodies, which has reduced the costs of drugs, by

using principles that are transferable to other settings.

We conclude that by applying these techniques

in the right settings, programs could experience

immediate benefits. Moreover, if policy-makers

accompany efforts with appropriate research design,

the results will bolster the peer-reviewed knowledge

base in this space and supplement the extant grey

literature.

In addition to the actions described above, experts

who we interviewed mentioned the following

remaining gaps in knowledge that could be

addressed by further research:

1. Wastage rates and the associated cost;

2. The total ‘true’ cost of transporting products

in country—very little transparency exists in

these costs, but knowing this could allow more

effective negotiations by governments, donors,

and implementing partners; and,

3. The value received from spending on technical

assistance.

50

Lab operations

What data on ASD costs and expenditures are publicly available?

We found few repositories of spending estimates for

lab operations at the global or country level, despite

the fact that laboratory operations are a prominent and

critical component of above service delivery activities,

particularly for HIV/AIDS, malaria, and tuberculosis. We

found limited unit cost data, and those we found did

not provide a comprehensive cost repository for lab

operations in full.

Laboratory operations are a critical activity within

HIV/AIDS programs. Recently, PEPFAR identified

strengthening laboratory systems as an investment

priority. In PEPFAR’s 2014 expenditure analyses,

lab-related expenditures accounted for 285 million

USD out of the total 3.5 billion USD that it invested in

country programs, making lab-related expenditures

the third largest overall category of spending, behind

facility-based care, treatment, and support (FBCTS),

and activities for the prevention of mother-to-child

transmission (PMTCT) of HIV.

Countries' National AIDS Spending Assessments

(NASAs) report domestic spending on laboratory

operations within HIV programs, although these

data are less consistent and tend to be captured

across different activity lines.

In the NASAs of Kenya, Malawi,

Ethiopia, Zambia, Nigeria, and

Swaziland, HIV-specific laboratory

monitoring typically comprises 5

to 20 percent of Treatment and

Care costs. Countries may include

spending to upgrade laboratory

infrastructure and equipment under

the infrastructure component of

program management when they

report their expenditures.

The Multi-Country Analysis of

Treatment Costs for HIV (MATCH)

study, conducted in 2011 by CHAI

in partnership with the Center

for Global Development and the

governments of Ethiopia, Malawi,

Rwanda, South Africa, and Zambia

reported additional data on

laboratory spending. The MATCH

study gathered data between 2009 and 2011 and then

analyzed expenditures from 161 antiretroviral therapy

facilities. The expenditure data were disaggregated

across four cost categories: personnel, ARVs,

laboratory operations, and other expenses.

The MATCH study found that lab costs generally

accounted for less than 10 percent of ART facility

costs, as illustrated in Figure 29 below. In South

Africa, lab costs appear significantly higher.

However, this is likely explained by the fact that the

data from South Africa are more comprehensive

than those from Ethiopia, Malawi, Rwanda, and

Zambia, in which laboratory spending only includes

consumable commodities (e.g., spending for test

cartridges, reagents, etc.); for South Africa, the data

additionally include personnel and equipment costs.

In all countries, the category of laboratory costs

includes costs from primary, secondary and tertiary

lab facilities, but lab sample transportation costs and

supply chain costs were not included in the study

(Tagar et al., 2014).

CHAI has conducted analytical work to estimate the

size of large laboratory commodity costs, depending

on the size of a country’s ARV program, based on

price data reported in CHAI’s ARV Market Report.

Based on the assumption that patients receive all

tests recommended for clinical care and diagnosis,

CHAI expects lab commodities to account for 13

percent of ARV commodity costs. This estimate does

Figure 29. Laboratory spending as a share of total facility-level expenditure across the five MATCH study countries

South Africa*EthiopiaRwandaZambiaMalawi

4%

5%

6%

9%

15%

Source: MATCH study. Lab costs only include consumables, except in the case of South Africa, where lab commodities, personnel, and equipment were combined.

Landscape Study of the Cost, Impact, and Efficiency of Above Service Delivery Activities in HIV and Other Global Health Programs 51

not include the costs of

maintaining buffer stock

or of service delivery

(e.g., equipment,

maintenance, personnel,

training, infrastructure,

etc.), which could

substantially contribute

to the full cost of a

laboratory system.

The Global Fund

currently tracks budget

data across its program

grant agreements

within its malaria

programs. The Global

Fund disaggregates

these budget data by

expenditure category;

however, there is no

unique category for

laboratory operations

or for diagnosis-related

costs. It is possible

that the Global Fund

captures these costs under “Infrastructure and other

equipment,” although the definition of this cost

category is not clearly established. Similarly, data

from the President’s Malaria Initiative (PMI) are not

granular enough for researchers to identity global or

country laboratory spending for malaria.

With respect to TB programming, the WHO’s annual

country surveys on TB expenditure record data

on budget requirements and expected funding for

“Laboratory infrastructure, equipment and supplies.”

These data are available upon request, but the WHO

does not publish them.

As illustrated in Figure 30 above, the data available

from the WHO expenditure tracking database for

TB laboratory spending indicate a wide variance

in spending levels amongst countries, between

2010 and 2014. Averaged across all data available

from 2010 to 2014, countries reported spending

approximately 12 percent of total TB expenditures

on laboratory services. However, lab expenditure

data are not available for numerous countries (36

of the 47 countries for which TB expenditure data

were obtained did not report lab expenditures in

at least one of the years 2010-2014, inclusive, and

10 countries did not report any lab expenditure

data across all five years), which limit our ability to

perform a comprehensive analysis of this database.

Current efforts by researchers to study TB lab costing

data have focused on the introduction of Xpert MTB/

RIF testing in labs. This new test provides a cartridge-

based automated nucleic acid implication test for TB,

which provides a far more sensitive and rapid results

than the conventional sputum smear microcopy test,

and additionally enables labs to test for resistance to

rifampicin, one of the common first-line antibiotics

used to treat TB. The Xpert assay is substantially

more expensive than smear microscopy testing, but

modeling studies and experts argue that the cost

is offset by the test’s higher sensitivity in detecting

cases of TB and enabling healthcare providers to

use rifampicin and other anti-tubercular drugs more

appropriately and effectively.

What is known about the impact and efficiency of lab operations?

Improving the quality and efficiency of laboratory

operations to enable healthcare workers to accurately

detect cases and perform infectious disease

surveillance is a key priority of donor organizations

and a critical part of providing diagnosis and treatment

in line with international guidelines (Petti et al., 2005).

Figure 30. WHO TB expenditure for laboratory services in thirty-seven sub-Saharan countries (2010-2015), as

a percentage of total country TB expenditure

0

5%

10%

15%

20%

25%

30%

35%

Cha

d

Cab

o Ve

rde

Benin

Mal

awi

Gui

nea-

Bissau

Nam

ibia

Libe

riaSw

azila

ndEt

hiop

iaEr

itrea

Zam

bia

Gha

naTo

go

Sout

h Su

dan

Côt

e d'

Ivoi

reZi

mba

bwe

Mad

agas

car

Burki

na F

aso

Gra

nd T

otal

Mau

ritan

ia

São

Tom

é an

d Pr

ínci

pe Mal

iD

RCBur

undi

Sene

gal

Moz

ambi

que

Botsw

ana

Nig

eria

Gab

onN

iger

Cam

eroo

nLe

soth

oTa

nzan

iaKe

nya

Rwan

daG

uine

a

Cen

tral A

frica

n Rep

ublic

Source: WHO

52

As of 2013, PEPFAR has invested roughly 3 billion

USD to create and strengthen laboratory networks

(including commodity and training spending), with

the goal of creating a lab infrastructure to support

treatment of HIV and other major health challenges.

Healthcare providers who do not have access to

adequate laboratory services often misdiagnose

diseases by relying solely of clinical diagnosis. Clinical

diagnosis, based on signs and symptoms of illness

rather than laboratory tests, requires no additional cost

or specialized equipment, but can be error-prone.

Healthcare providers risk both underdiagnosis—

where the presence of a critical infection is not

diagnosed—and over-diagnosis—where, for instance,

fever-like symptoms are classified as an endemic

illness (e.g., malaria), when, in fact the cause is from a

different source. In Tanzania, a study of 4,670 patients

admitted to hospitals with clinical diagnoses of severe

malaria found that less than half of the patients had

blood smear results that confirmed the presence of

Plasmodium falciparum (Reyburn et al., 2004). These

patients, further, tended to have better outcomes as

a results of treatment than those patients whose test

results did not indicate a malarial infection, suggesting

that many patients were misdiagnosed and did not

receive appropriate treatment.

More sophisticated laboratory operations may enable

health systems to conduct drug resistance tests, which

can enable healthcare providers to prescribe targeted

first and second line drugs, potentially improving both

the impact and efficiency of treatment. In the case

of HIV, genotypic and phenotypic assays can test

viral drug resistance to ensure the appropriate use of

second line ARVs, thus reducing ineffective treatment

and associated waste. Accurately targeting therapies

will be of increasing relevance as HIV care continues

to shift from a paradigm of acute treatment to chronic

disease management. New Xpert assays also are able

to detect whether sample strains of the tuberculosis

bacillus will respond to rifampicin, which ensures that

patients are not initiated on therapies to which the

bacillus is resistant.

Countries and donors must consider efficiency and

cost-effectiveness when they are deciding whether

to adopt new diagnostic or treatment guidelines. As

new diagnostic tests become available, the country,

donor, and program must weigh the effectiveness

of the new approach against the cost of upgrading

laboratory equipment or retraining laboratory

staff. Next-generation diagnostic tests, such as the

Xpert assay, may be more expensive than existing

diagnostics, requiring a program to make large initial

investment in equipment costs; this, however, may

actually result in efficiency gains through more

automated, sensitive, and reliable test results, which

will increase the likelihood that patients will a) receive

accurate diagnoses, b) receive the most appropriate

drug regimens, and c) not be lost to follow-up and be

retained in care.

Importance of standardization

Many of the current global efforts to strengthen and

improve laboratories have focused recently on the

standardization of laboratory operations (USAID,

2010). As parallel public health programs to manage

HIV/AIDS, malaria, and TB have been developed over

the past decades, demand for testing services has

increased. In response to the demand for testing

for disease specific programs, laboratory services

began to decentralize such that testing could occur

at peripheral laboratories, resulting in the growth of

laboratory services at the local and district levels.

In 2008, the WHO convened a consensus meeting

on Clinical Laboratory Testing Harmonization and

Standardization in Maputo, Mozambique. The main

product of the meeting—the Maputo Declaration on

Strengthening of Laboratory Systems—urged countries

to create integrated, tiered laboratory networks. In

addition, the Declaration emphasized that countries

must standardize laboratory equipment and protocols

to strengthen laboratory systems.

From an efficiency perspective, standardizing

operations across lab facilities enables peripheral

laboratories to use the same machines and reagents,

resulting in economies of scale. These benefits may

extend to the supply chain, to improve and make

more efficient systems to procure and distribute

(or redistribute) equipment to prevent expiration

and stockouts. Creating and imposing standards

with respect to testing procures also enables

programs and researchers to collect more rigorous

and comparable incidence and prevalence data,

which critically inform public health planning for

communicable disease management and prevention.

Landscape Study of the Cost, Impact, and Efficiency of Above Service Delivery Activities in HIV and Other Global Health Programs 53

What are the key challenges that impede the efficiency and strengthening of laboratory operations?

The strength and efficiency of laboratory systems

depend on a number of distinct elements, which

may vary significantly across different country

settings. According to Petti et al., the following

critical variables within and across laboratory systems

affect their strength and efficiency:

• Quality of laboratory facilities;

• Access to utilities (e.g., piped water and constant

power supply);

• Availability of laboratory equipment and supplies

(e.g., incubator, refrigerator, freezer, microscope,

and staining reagents);

• Implementation of standard written operating

procedures (including quality-control procedures);

and,

• Knowledge or skill of supervisors and technical

personnel.

As Petti et al. explain, strong laboratory systems

require quality assurance mechanisms by which

health systems can monitor performance and ensure

adherence to clinical and technical guidelines.

Recent PEPFAR-supported work from Nigeria

indicates that “close monitoring and continuous

adherence to policies are vital” to quality assurance

and capacity building (Abimiku et al., 2010).

Underfunding of laboratory facilities

Our review of the literature found that in certain

settings, laboratory facilities are heavily underfunded

and under-resourced within the context of the

broader health program.

In Zambia, for example, the WHO conducted a

Service Availability and Readiness Assessment (SARA)

in 2010, a component of which assessed how many

healthcare facilities had access to 9 core laboratory

tests: hemoglobin, blood glucose, HIV rapid

diagnostic test (RDT), syphilis RDT, malaria RDT or a

smear test, TB microscopy, general microscopy, urine

pregnancy test, and urine dipsticks. The SARA analysis

found that only 8 percent of healthcare facilities

across 17 districts had all 9 core laboratory tests, as

displayed in Figure 31. On average, facilities were able

to conduct less than half of the 9 laboratory tests on

site; in the peri-urban Solwezi district, facilities had

access to only 1 or 2 core tests, on average.

Figure 31. SARA 2010 results regarding the availability of standard laboratory tests, per district, among 17 districts in Zambia

Average numberof lab testsavailable on site

Percentage offacilities withall 9 standardlaboratory tests

Sesh

eke

Kapu

ta

Kalo

mo

Lusa

ka

Cho

ngw

e

Kitw

e

Ove

rall

Livi

ngst

one

Kapi

ri-M

posh

i

Chi

ngol

a

Nic

hele

nge

Mw

inilu

nga

Kate

taC

hibo

mbo

Sam

fya

Solw

ezi

Isok

a

Chi

pata

1

0

2

3

4

5

6

7

0

5

10

15

20

Pe

rce

nta

ge

Ave

rag

e n

um

be

r

Source: SARA results for Zambia (2010). Nine core laboratory tests are hemoglobin, blood glucose, HIV RDT, syphilis RDT, malaria RDT or smear, TB microscopy, general microscopy, urine pregnancy test, and urine dipstick.

54

Importance of strong referral and distribution systems

Proper diagnosis of disease does not depend solely

on well-equipped and well-functioning laboratory

facilities; issues of supply chain and transportation

logistics are also critical in ensuring that specimens

are delivered to testing centers, and that the results of

tests are returned to clinicians and patients. As global

health programs pursue higher rates of coverage,

improving access among rural and other hard-to-

reach populations, laboratory systems increasingly

depend on reliable referral and distribution systems.

Referral and distribution systems will also be

increasingly important as high cost/high impact

assays like Xpert are rolled out within countries.

Because the cost of providing each facility with new

diagnostic machines is prohibitive, countries may

adopt a more efficient system that relies on sample

and test result delivery between the facility and

district or regional level lab facilities. As one of our

informants noted, there is no need for expensive

laboratory equipment at every facility if the health

system has an efficient transportation component

to support the network of national laboratories.

Conversely, our informants noted that in situations

in which distribution systems are poor, specimens

and test results may be lost en route. As a result,

tests need to be re-run because samples and

results are lost in transit. Informants suggests that

these inefficiencies that could be greatly reduced if

countries improve the monitoring and management

of lab information.

Researchers have noted that these and other

considerations raise concerns that patients

regularly fail to receive their test results under

heavily centralized lab systems. Studies from Kenya,

Tanzania, and Mozambique indicate that the results

of 40 to 45 percent of early infant diagnostic tests

for HIV are never received by the patient/guardians

(Dube et al., 2012; Hassan et al., 2011; Nuwagaba-

Biribonwcha et al., 2010), leading to wasted

consumables, unnecessary repeat testing, and infants

lost to follow-up.

Some of our informants noted that point-of-care

(POC) laboratory testing increasingly is becoming

an attractive alternative in certain facilities, in

which volume is sufficiently high to justify the

additional capital expense of investing in POC

testing equipment and training local lab staff. POC

testing, our informants indicated, is more expensive

than centralized laboratory tests, due to the higher

costs for equipment and consumables; however,

our informants argued, when the costs of the

transportation of samples and the rate of waste/

loss incurred by centralized systems are taken into

account, the cost per result returned is similar for

POC and centralized laboratory testing. Meanwhile,

recent research in lab system optimization indicates

that POC investments may generate stronger

operational improvements for diagnostic efficiency

than centralizing investments in transportation and

laboratory capacity (Deo & Sohoni 2015).

Need for further progress in laboratory integration and accreditation

Our informants observed that the integration of

laboratory services remains an unfinished objective

of the 2008 Maputo Declaration. Although countries

have made significant strides in ensuring access

to laboratory facilities across programmatic areas,

inefficiencies remain that could be remedied by

leveraging new technologies and equipment to

generate areas of cross-disease program synergy.

For example, cartridge-based Xpert test machines

have the capability to conduct molecular diagnosis

of both HIV and TB infections. While these machines

may be financed as part of a national HIV or TB

program, both types of programs could benefit from

underutilized capacity. Our informants noted that

political barriers might impede programs’ abilities

to capture and realize these efficiencies due to the

siloed nature of certain vertical disease programs

and the program managers’ potential territoriality.

If countries (and, where relevant, donors) work

to establish ways to fairly distribute the high cost

of Xpert equipment, maintenance, and training

across disease programs, programs will utilize these

diagnostic tools more effectively and efficiently.

Finally, our informants noted that the accreditation

of laboratory facilities in developing countries is still

nascent. In 2008, the WHO Regional Office for Africa

and other stakeholders developed a training program

in laboratory management and quality assurance

systems: the Strengthening Laboratory Management

Towards Accreditation (SLMTA). In 2011, the Regional

Office developed a tiered accreditation scheme, the

Stepwise Laboratory Improvement Process Towards

Accreditation (SLIPTA). Both SLMTA and SLIPTA have

Landscape Study of the Cost, Impact, and Efficiency of Above Service Delivery Activities in HIV and Other Global Health Programs 55

been hailed as key accomplishments in enhancing

laboratory systems in sub-Saharan Africa (Alemnji et

al., 2014). Early reports from Tanzania’s experience

with SLMTA indicate that the programs has enabled

progress and diminished potential barriers to

improvement (Andiric & Massambu, 2015); however,

researchers have not assessed the extent to which

these efforts have had an impact on patient care.

What current, on-going, or planned efforts will improve the knowledge base of lab operation costs and their efficiency?

Our informants identified two specific analyses that

programs are conducting to expand the knowledge

base of lab operation costs and planning. TB-MAC

currently is conducting an analysis of the scale

up costs associated with the roll out of Xpert test

assays to replace/supplement conventional swab

testing. Within HIV, PEPFAR currently is examining

international standards for laboratory support,

specifically in the context of managing HIV as

a chronic disease. The results of these efforts,

however, are not yet available.

Gaps in knowledge and recommendations to improve lab operations

We found that disease programs do not consistently

or comprehensively capture laboratory costs and

expenditures. We perceive that there is a need for

additional costing studies to provide data on the

full systemic costs of laboratory testing, which

should capture the impact of waste and the costs

of equipment maintenance, staff training, and lab

supervision.

Many of our informants noted that laboratory systems

are routinely underfunded, in part because budgets fail

to capture essential activities within the system. Budgets

for laboratory systems—and donors' financial support

for laboratory systems—often focus on the capital

expenditures associated with laboratory equipment

and reagents; however, budgets may not adequately

cover costs associated with maintaining equipment, or

training and supervising laboratory technicians. Without

these activities, laboratory equipment falls into disrepair

or disuse, and reduces the overall efficiency of the

system. In some settings, ensuring adequate funding

and support for laboratories will require some financial

and political reorganization, e.g., separating laboratory

budgets from hospital and other facility budgets, and

placing the laboratory system as a whole under the

supervision of a high-ranking health official. This would

provide centralized accountability and insulate fiscal

space for laboratory operations from hospital budgets.

Based on our review, we recommend that countries’

ministries of health implement the integration of

laboratories across vertical disease programs and

pursue synergies in the molecular assay testing of HIV

and TB, such that laboratory facilities operating below

capacity relieve some of the testing burden of high

volume labs. This integration will require collaboration

by disease programs within ministries of health to

ensure adequate financing, implementation, and

oversight.

We also recommend that as laboratory systems

become more developed, laboratory managers

consider implementing cost effectiveness analyses for

point-of-care and external laboratory testing at various

high volume facilities, in order to appropriately prioritize

sites for point-of-care testing. Within certain facilities,

the high volume of testing may justify the higher capital

expenditure for new POC equipment; however, these

determinations must be done strategically, in order to

selectively distribute POC diagnostics to areas in which

they would be most effective.

Finally, as accreditation and training programs such

as SLMTA and SLIPTA mature, country case studies

and patient impact studies will provide additional

information on the impact of lab standardization and

professionalization on patient care. Country audits of

accreditation and training programs will provide useful

feedback on the uptake and retention of laboratory

quality improvements, indicating where challenges may

arise in promoting high laboratory standards and what

strategies may be best utilized to overcome them.

56

Above Service Delivery Costs Related to Aid Architecture

While Aid Architecture as a whole is a much larger

topic than we can address in this report, in this

chapter we lay out some of the ways in which aid

architecture affects activities and costs above the

point of service delivery.

Overview of aid architecture

Aid architecture, in its broadest definition, refers to

the mechanisms of delivery, implementation, and

management of the international aid system (UN

World Economic and Social Survey 2010). These

mechanisms facilitate the economic management

and disbursement of billions of dollars of official

development assistance (ODA), which provides vital

support for a variety of health programs around the

world and has had an impressive global impact.

However, these mechanisms, and the structures

through which bilateral funding is channeled, carry

significant costs. At each level of the international

aid system, activities designed to ensure successful

program implementation absorb a portion of the

funding, reducing the overall amount of funds that

reach the point of service delivery. By definition, all of

these elements are ASD.

This chapter considers aid architecture costs from

the vantage of the following three levels, depicted in

Figure 32 below:

• Above-national costs, or those costs incurred by

the Development Partners’ (DPs) headquarters

outside of the country

• Costs incurred by in-country implementing

partners (IP) in delivering the services, especially

when IPs are not local organizations within the

country of service delivery

• In-country aid delivered through technical support

and expatriate labor

Figure 32. Conceptual model for understanding ASD activities associated with Aid Architecture

Funding allocated to bilateral activities

Implementing partner funding

In-country program spending

Service delivery spending

ASD activities and costs

Above-national costs andcountry o�ce costs

Technical assistance andexternal consultant costs

Implementing partneroverhead costs

1

2

3

Landscape Study of the Cost, Impact, and Efficiency of Above Service Delivery Activities in HIV and Other Global Health Programs 57

across various agencies and expenses, as shown in

Table 10 below.

Across these different agencies, funding amounted to

142 million to 174 million USD in FY2012 to FY2014, as

shown in Table 11 below. As a general practice, these

Technical Oversight and Management costs have

accounted for approximately three percent of total

funding from USAID Global Health Programs (GHP)–

State Department.

In addition to these technical oversight and

management funds, the following two additional

categories of above-national expenditures are

included in PEPFAR’s “headquarters” budget:

• Technical Support, Strategic Information, and

Evaluation,

• Additional funding for country programs at the HQ

level.

Including these activities, in FY 2013, PEPFAR's

combined “headquarters” funding comprised 8.4

percent of the total bilateral GHP-State PEPFAR

account, excluding contributions to the Global

Fund and UNAIDS. This funding level indicates that a

substantial share of PEPFAR’s funding is retained for

above-national activities. (Office of the United States

Global AIDS Coordinator, 2013.)

Above-national costs and country office costs of international aid organizations

Above-national costs

In interviews, a number of experts noted that a portion

of international funding is allocated to above-national

expenses incurred at the headquarters of coordinating

bodies and other agencies. These central-level costs

are separate from the personnel costs of donor

organization staff who monitor grants and programs

in-country, and reflect the central oversight and

international management of programming within the

headquarters of the donor agency.

We were not able to easily access central costs for all

relevant aid organizations, but we found that some

public data are available from the U.S. government’s

PEPFAR and PMI programs. Central costs of these

programs appear to constitute between three and

nine percent of total program funding per year.

For PEPFAR, funding to cover central and coordination

costs are provided by Technical Oversight and

Management (TOM) funds. This funding is disbursed

Table 10. USG agencies receiving PEPFAR Technical Oversight and Management (TOM) funds

USG agency Expenses supported by PEPFAR TOM funding Associated sub-agencies and centers

Department of State Direct expenses, including salary, benefits, and travel

Office of the U.S. Global AIDS Coordinator and Health Diplomacy (S/GAC)

Bureau of Intelligence and Research (INR)

USAID Direct and indirect expenses including salary, benefits, travel, supplies, professional services, and equipment

Department of Health and Human Services

Direct and indirect expenses including salary, benefits, and travel, overhead, operation and maintenance of facilities, and advisory and assistance services)

Centers for Disease Control and Prevent

Health Resources and Services Administration

National Institutes of Health

Food and Drug Administration

Substance Abuse and Mental Health Services Administration

Peace CorpsDirect and indirect expenses including salary, benefits, travel, supplies, professional services, and equipment

Department of DefenseDirect and indirect expenses including personnel, equipment, supplies, services, professional development, travel and transportation

Source: Office of the United State Global AIDS Coordinator, 2015

58

Figure 33. PMI funding by fiscal year (USD, FY 2006-2014)

HeadquartersCambodiaBurmaMekongZimbabweGuineaNigeriaDRCZambiaMaliMadagascarLiberiaKenyaGhanaEthiopiaBeninSenegalRwandaMozambiqueMalawiUgandaTanzaniaAngola0

100,000,000

200,000,000

300,000,000

400,000,000

500,000,000

600,000,000

700,000,000

FY2006 FY2007 FY2008 FY2009 FY2010 FY2011 FY2012 FY2013 FY2014

Source: PMI

Note: Allocations to Headquarters spending are highlighted in red

Table 12. Share of PMI funding by fiscal year allocated to Headquarters

FY 2006

FY 2007

FY 2008

FY 2009

FY 2010

FY 2011

FY 2012

FY 2013

FY 2014

Average

% Headquarters 5% 6% 7% 9% 7% 6% 6% 6% 6% 7%

For the Presidential Malaria Initiative, a similar pattern

emerges, with between five and nine percent of

total funding shown to have been allocated to PMI’s

activities at the headquarters level between FY 2006

and FY 2014, as shown in Figure 33 and Table 12

below (PMI, 2014).

Country office costs

The managing agencies for large development

partners usually have offices in-country that incur

operational costs, and that allow for the management,

coordination and oversight of all their grant activities.

For example, the PEPFAR funding flows through

organizations such as USAID and CDC, which typically

operate offices in every major recipient country.

The data on these expenses for most donors is

not publicly available. For example, the PEFPAR

EA country data, when collected, is disaggregated

between national and regional levels, but this

disaggregation is not available on the website.

Table 11. Percent of PEPFAR funding by fiscal year approved for Technical Oversight and Management

Funding, in $’000 FY 2012 FY 2013 FY 2014FY 2015

(estimate)FY 2016

(requested)

GHP-State funding (estimates) $5,542,860 $5,439,829 $5,670,000 $5,670,000 $5,426,000

Approved Technical Oversight and Management funds for S/GAC Oversight/Management

$174,096 $154,961 $142,500 $161,628 $162,000

% of GHP-State funding 3.14% 2.85% 2.51% 2.85% 2.99%

Source: Office of the United State Global AIDS Coordinator, 2015

Landscape Study of the Cost, Impact, and Efficiency of Above Service Delivery Activities in HIV and Other Global Health Programs 59

In South Africa, the expenditure tracking exercise

undertaken for the HIV Investment Case found that

PEFPAR spent 20.1 percent of all spending at national

level and 3 percent at the above-national level. Some

of these national level overhead costs, especially

personnel, are ‘attributed’ to the management of

specific programs, as they are considered essential

costs to ensure optimal program performance.

Overhead costs of implementing partners

We found limited publicly available data on how

implementing partner organizations use the funds

that they receive from donor agencies. In many

cases, these are international organizations that may

have substantial expenses at the above-national

level, both in terms of organizational overhead

and in terms of technical support activities being

coordinated from the organization’s headquarters.

In interviews, some experts expressed concern that

large sums of funding are being retained in donor

countries, particularly the United States, and that the

use of these funds is not at all transparent. That is

not to say that these funds are not justified. Some

amount of coordination/administration is necessary,

and program support provided internationally by staff

in the organization headquarters may be of direct

and high value to teams working on the ground. The

lack of transparency, however, makes it impossible

for researchers and donors to verify that these funds

are being allocated effectively.

A recent Center for Global Development paper

analyzed the financial flows of PEPFAR’s funding

for implementing organizations, and found that the

majority of PEPFAR’s most highly funded implementing

organizations were U.S.-based agencies, NGOs, and

universities; by comparison, a small percentage of

contract dollars were awarded to prime recipients in

the developing world (Fan et al., 2013). Researchers

noted concern that a significant portion of PEPFAR

funding likely is retained in the U.S. (e.g., in overhead

payments of U.S.-based organizations), and that this

may not be the most effective use of the scarce

resources available. Global health commentators and

NGOs have taken up this issue over the years. In 2008

Laurie Garrett of the Council of Foreign Relations

noted: “Contractors, universities, NGOs and FBOs take

significant overhead off the top of federal grants, fund

activities executed by Americans on the ground, and

buy American-made goods even when less expensive

products are regionally available” (Garrett, 2008). This

concern still resonates to some extent with many of

our informants.

More recently, Vijaya Ramachandran and colleagues

at the Center for Global Development voiced a

related concern in a study of US spending in Haiti

in the aftermath of the 2010 earthquake; a lack of

transparency regarding subcontractor disbursements

activities. Though disbursements of USAID funding

to Haiti are reported for primary contractors,

subcontractor disbursement data are often not made

public, making it difficult for researchers to determine

how funds are being spent and what the funding has

achieved (Ramachandran and Wlaz, 2013).

PEPFAR’s ongoing efforts on local capacity-building and

drive towards country ownership should lead to some

increase in the percentage of resources reaching the

ground and should be commended; several informants

suggested that greater transparency in how PEPFAR

funds are used by U.S.-based implementing partners

would complement PEPFAR’s larger data dissemination

efforts, and enable researchers to sharpen analyses of

value for money for PEPFAR expenditures.

In-country expenses: Expatriate labor and external consultants

We found that within in-country spending, two

additional phenomena may contribute to high

spending on activities above the point of service

delivery: the employment of expatriate labor rather than

local labor, and the use of external (often international)

consultants to provide technical assistance.

As alluded to in the previous section, U.S. agency-

funded programs often employ Americans in-country

to coordinate and execute program activities. We were

unable to determine the extent to which this occurs—

and the proportion of total funding that goes towards

the salaries of expatriate—based on public data sources.

One recent study (Marseille et al., 2012), found that

expatriate labor accounted for nearly a quarter of the

personnel costs in Zambia’s HIV response; but these

types of data are not available across multiple countries.

In general, salaries of expatriates are significantly

higher than local wages, and the substitution of local

60

for expatriate labor has the potential to realize non-

trivial gains in efficiency. In interviews, experts noted

that the predominant use of expatriate labor may

have been justified in the early days of programs,

when programs were being set up and implemented

as quickly as possible to get vital services in place.

Interviewees said that plans to build local capacity

and transfer ownership of programs to local

organizations have not always been executed.

In addition to the use of expatriate program staff,

interviewees state that costs incurred by external

consultants providing technical assistance are

substantial. Programs engage external consultants

to provide technical assistance on various activities,

including grant proposals, concept notes, and

investment case preparation, to improve the efficiency

of procurement and supply chain management,

and advise on the development of quality assurance

mechanisms. Informants indicated in interviews

that salary and travel expenses of consultants, and

other international organizations providing technical

assistance, substantially contributed to ASD spending,

particular in situations in which local technical

expertise is scarce or in high demand.

We found that limited data are available on the costs

incurred through the use of international technical

assistance. The best data available are the PEPFAR

expenditure analyses, which capture as a line item

the expenses accounted for by “external consultants”

within the country operational plans. These expenses

may include those of both international consultants

and local consultants (and locally incorporated

offshoots of international consultants).

Examining a subset of countries, PEPFAR data indicate

that external consultants constitute a median of 2

percent of PEPFAR program expenditures, as show

in Table 13 below. Although the external consultants

line in Table 13 is not defined to be above the point of

service delivery, the vast majority of external consultant

spending is allocated to the large ASD cost categories

of Health System Strengthening, Strategic Information,

and Program Management.

While the costs of external consultants are not trivial,

several experts raised questions about the impact

and efficiency of spending on external consultants,

particularly international consultants. The value of

international TA has not been well established in

studies (either in its own right, or in contrast to locally

available services), and interviewees expressed some

concern that countries continue to rely on international

TA on a recurring basis rather than for occasional

specialist assistance. On a slightly different angle, some

informants also considered that international TA could

be counter-productive when used intermittently, as

consultants flying in for short-term work may not have

the proper contextual knowledge to provide effective

assistance.

Several experts believed that moving from international

consultants to local consultants could lead to large

cost savings due to lower fees, but this has not always

been easy to achieve in practice. For example, PEPFAR

stakeholders report that fee differences between

international and local TA have been less significant

than expected, and speculate that this may reflect

distortions in the local labor market after years of paying

high prices to international organizations working in

these markets.

In order to make optimal use of

program funding, we recommend

increasing research into the

impact and value-for-money

of consultants (and particularly,

international consultants) to provide

helpful information for countries

and donor programs. In addition,

we recommend that researchers

and donors include consultant

fees and in-country expatriate

labor as standard line items in

comprehensive costing studies.

Table 13. Share of spending identified by PEPFAR expenditure analysis for external consultants

Percent of spending for external consultants

Ethiopia 1.67%

Kenya 1.72%

Malawi 3.40%

Nigeria 2.29%

Swaziland33

9.35%

Zambia 0.92%

Source: PEPFAR

33 The percent share of PEPFAR spending for external consultants in Swaziland is particularly high due, in part, to the fact that the government of Swaziland currently funds 100 percent of the country’s ARV drugs. This spending results in a 0 percent allocation for ARVs under the country’s PEPFAR budget, which increases the percent share of other costs.

Landscape Study of the Cost, Impact, and Efficiency of Above Service Delivery Activities in HIV and Other Global Health Programs 61

Priorities and Opportunities for Future Research and Programming: Recommendations

We have summarized a series of recommendations, based on the analysis presented above and the recommendations

of the experts and stakeholders that we consulted. Up until now, it is understandable that the greatest level of attention

and scrutiny has been paid to service delivery activities and costs rather than those above the point of service delivery:

service delivery activities have the most directly observable impact on patients; facility-level costs can be more easily

defined, measured, and compared than those outside the facility setting; and there has been significant room to

lower the costs of service delivery by focusing on making facility-level activities more efficient and through global

efforts to bring down the price of critical commodities. However, as donor aid budgets tighten and countries take on

increasing levels of ownership for their HIV response within their national health systems—and as the scope for further

efficiency gains at service delivery level diminishes—countries, donors, and the broader global health community must

begin to take on the challenge of improving health program performance above the point of service delivery. Our

recommendations cut across two categories: those that relate to future data gathering, research, and harmonization;

and those that relate to future programming.

Recommendations regarding future research, data gathering, and harmonization

Improved cost and expenditure data are a prerequisite to optimize ASD activities

As discussed throughout the report, the general lack of publicly available, high quality, and readily comparable data

on costs (or expenditures) for activities above the point of service delivery hampers researchers’ efforts to analyze

ASD activities, evaluate their efficiency, or benchmark results across country, provincial, or program settings.

Generating the most useful data will require significant up-front coordination between stakeholders, and an

ongoing commitment of resources from governments and donors, but improved cost and expenditure data could

lay the groundwork for significant gains in efficiency and value for money in major health programs. We believe

that the benefits of better cost and expenditure data justify the additional investments needed to generate them.

We have identified the following priority areas for improvement:

1.1 Comprehensive costing of programmatic activities, to better understand true costs, enable benchmarking of ‘reasonable’ activity costs, and enable countries to consider strategies for cost reduction above facility level, such as task-shifting from more expensive labor to less expensive labor. Comprehensive costing—collecting cost data both at and above the point of service delivery, rather than focusing on one or the other— is the best way of ensuring that no relevant costs are overlooked.

1.2 Creation, validation, and adoption of a standardized taxonomy for ASD cost components to bolster cross-program comparison of ASD spending, a critical tool for health system policymakers.34

1.3 Standardized costing approaches for evaluating investments in shared resources (e.g., labs, blood safety, health information systems), and attributing the costs of these shared resources to specific programs where relevant.

1.4 Collection of high quality sub-national cost and expenditure data.

1.5 Where costing is more robust and data collection has been standardized across a number of countries, expand the exercise to include additional sample countries and longitudinal data collection to facilitate more useful benchmarking and trend analysis to inform country decision-makers (e.g., in immunization: the EPIC studies provide a solid starting point with a standard methodology and baseline data for six countries).

34 Figure 1 of this report provides a starting point for this exercise.

62

1.6 For supply chain efficiency and cost, greater data are needed to better understand wastage rates (and the impact of waste on cost) and the total ‘true’ cost of transporting products in country. Very little transparency exists with respect to these costs, but knowing this information could enable governments, donors, and implementing partners to engage in more effective negotiations.

Impact and performance measurement and analysis would inform ASD activity optimization and strengthen program planning

As discussed at various points in the report, we found that little has been done to connect ASD activities to their

impact on service delivery, making it challenging for researchers to discern whether resources are being allocated

optimally across ASD activities, and to what degree countries should continue to finance these activities if they

transition away from donor support. We have identified the following key priorities:

2.1 Create good, standardized intermediate outcome metrics for ASD activities (in a similar vein to those used in procurement/supply chain) to help better understand performance and enable improvements.

2.2 Additional research and analysis is needed to understand the cost and impact of some specific activities. Experts and stakeholders expressed particular interest in understanding the returns on different levels of demand generation activities (particularly, mass media), monitoring and evaluation, and the use of expatriate labor and external consultants.

2.3 In programs striving to maximize and extend coverage in difficult-to-reach areas and populations, greater research is needed to understand the impact of last-mile efforts on above service delivery costs.

2.4 Initial attempts should be made to link spending data to high quality outcome measures, wherever available. For example, TB treatment outcome indicators, which are well-established, collected in a wide range of countries, and tracked over time by WHO, may provide a promising place to start this type of analysis.

Donors and partner organizations can take immediate steps to enhance transparency regarding ASD expenditures and activities

In addition to the overarching recommendations above, specific donor and development partner organizations can

play a major role in improving the knowledge base on ASD costs and expenditures by sharing their existing data or

creating new knowledge. In so doing, they can provide the foundation for the analyses of ASD activities that are

critical to improve decision-making and increase value-for-money in major global health programs. Organization-

specific recommendations are as follows:

3.1 PEPFAR

3.1.1 PEPFAR’s publicly disseminated expenditure data represent the current high water mark in donor spending transparency and accountability. PEPFAR should continue with their efforts to provide timely and comprehensive expenditure data.

3.1.2 PEPFAR could facilitate more robust analyses of ASD expenditures by adding dimensions to the public data that disaggregate by geography and level incurred (e.g., national, provincial, district, local) where available. This would enable both a clearer accounting of the spending structures, and within-country comparisons/benchmarking of ASD spending.

3.1.3 PEPFAR should extend its transparency agenda by more clearly disclosing the extent of PEPFAR spending on PEPFAR operations in Washington, DC and country program offices. This would be complementary to PEPFAR’s continuing efforts to promote country ownership of the HIV response, and would help identify opportunities for further efficiency gains.

3.1.3 In the same vein, for PEPFAR expenditures supporting implementing partners and technical assistance, PEPFAR should more clearly disclose the proportion of funding that is retained in the United States (or other donor countries), whether through direct funding or indirectly through partner overheads.

Landscape Study of the Cost, Impact, and Efficiency of Above Service Delivery Activities in HIV and Other Global Health Programs 63

3.2 Global Fund

3.2.1 The Global Fund should increase the functional transparency of its grant budget data. While these data currently are made available on the Global Fund website through scanned program grant agreements, the Global Fund should publish past and current grant budget data via a repository or a machine readable format to facilitate analyses across countries and over time.

3.2.2 In the future, for large grants, the Global Fund should require that principal recipients report their expenditures by the organizational level at which the spending occurred. The Global Fund should do the same for smaller grants, except where this would create an undue reporting burden.

3.2.3 The Global Fund should sponsor additional analysis of ASD spending within countries to whom they provide significant support across all three focus disease programs.

3.3 WHO

3.3.1 WHO should continue to refine and strengthen the National Health Accounts and disaggregate, where feasible, the share of expenditures spent at different organizational levels.

3.3.2 WHO should publish validated, self-reported national expenditure data for TB and malaria programming via a repository and/or a machine readable format. (Raw forms of these data are currently available by request from WHO.)

3.3.3 WHO should lead efforts to standardize expenditure questionnaires across disease programs (e.g., malaria, TB), to the extent feasible.

In addition to the major donors and partners above, we recommend that other organizations take part in advancing

understanding of ASD costs and expenditures.

3.4 Global Health Costing Consortium (inception phase underway)

3.4.1 In collaboration with donors and other development partners, we recommend that the Consortium develop standardized methodologies to collect HIV and TB ASD costs.

3.4.2 The Consortium should engage with PEPFAR, the Global Fund, WHO and others to standardize ASD costing and analyze potential efficiency gains to be made through ASD activities.

3.4.3 The Consortium should generate, to the greatest extent possible, standardized cost estimates for ASD activities in HIV and TB, and channel the findings related to ASD costs into advocacy and data visualization tools/websites for broader dissemination.

3.4.4 To the extent feasible, the Consortium should engage with experts in fields outside of its core focus area of HIV/TB, and explore how the ASD costing standards for HIV/TB can be applied usefully in other global health programs.

64

Recommendations regarding future programming

As HIV efforts transition from emergency response to chronic management, structural reforms aimed at efficiency and sustainability should continue

In the first phase of the global response to HIV, donor agencies and development partners prioritized the rapid

creation of treatment and prevention programs to bring much-needed services to patients as quickly as possible,

without delaying to wait for local systems and capacity to be in place, and with less focus on long-term efficiency.

As programs increasingly focus on sustainability, we recommend that programs continue to implement, if not

accelerate, necessary structural reforms that will rationalize or refocus spending above the point of service delivery.

Our recommendations are applicable to multiple health programs, but address issues that were highlighted in

our conversations with experts and stakeholders as particularly relevant for improving the sustainability of the HIV

response.

4.1 Duplication and redundancy of activities and systems should be minimized where possible. In many countries, parallel systems were set up to ensure the delivery of HIV care—significantly increasing many costs above the point of service delivery—and the reintegration of these parallel systems into national health systems remains a work in progress. More broadly, WHO is in the process of piloting diagnostic tools to evaluate the extent of duplication within health systems; countries and donors should apply this type of diagnostic where relevant and continue the reform agenda. As a practical next step, the large donor agencies should consider collaborating with governments in a small number of pilot countries in an effort to reduce duplication across programs and rationalize ASD spending.

4.2 Efforts to reduce reliance on expatriate labor and international consultants must continue, including (where necessary) developing local capacity to assume the roles and responsibilities fulfilled by donor program staff or consultants from donor countries. Country ownership of the HIV response is acknowledged as critical for long-term sustainability, and efforts to transfer the necessary knowledge and skills should be prioritized. While reducing reliance on expatriate labor and international consultants should be a goal at every level of the health system where possible, the majority of these costs —and thus, the largest opportunities for efficiency gains—occur above the point of service delivery.

4.3 As the HIV response shifts towards managing HIV as a chronic disease, countries and donors must explore novel or differentiated approaches to treatment. For example, allowing adherent patients to check in with clinical staff at longer intervals and providing them with alternative means of receiving their drug supply can enable strong treatment outcomes while reducing the strain on front-line staff. Alternative models, remote monitoring, and novel distribution systems require shifts in policy and could imply new activities (and costs) above the point of service delivery in order to achieve improved outcomes or reduced costs at the point of service delivery.

4.4 As part of local capacity building, and in order to optimize local responses in an efficient way, donors should work with larger countries to build sub-national capacity for program evaluation and optimization, so that staff at provincial level or below can play an increased role in ensuring maximum impact from scarce health resources. This would initially represent an increased investment in new activities above the point of service delivery, but with the goal of improving overall performance and efficiency over time. This is particularly relevant in countries such as South Africa, where large parts of the HIV response are already decentralized to the provincial level and local adaptation may allow improved performance.

Landscape Study of the Cost, Impact, and Efficiency of Above Service Delivery Activities in HIV and Other Global Health Programs 65

Procurement, supply chain, and laboratory operations are areas where immediate gains in overall program performance and efficiency may be possible

In addition to the broad cross-cutting efforts that we discuss above, we identify several specific systems as having

potential for immediate gains. In particular, prior research and strengthening efforts in these areas have yielded

performance metrics and concrete strategies for optimization, which experts believe can be applied to increasing

numbers of countries and programs.

5.1 Where relevant, countries should institute procurement consolidation for drugs or consider expanding procurement consolidation to a broader set of commodities (such as diagnostic and clinical equipment). The latter approach has not been universally successful, and gains may vary depending on the capacity of central government and the nature of the relevant commodity market; an incremental approach to expansion in combination with tracking of simple performance metrics (e.g., unit prices paid for commodities) may be advisable.

5.2 Within supply chain management, programs should conduct rapid assessments and apply established techniques for efficiency improvement (such as level jumping, optimized transportation loops, informed push systems, vendor managed inventory, and eLMIS systems), potentially leading to both improved efficiency and performance. In addition to the potential for cost reductions in the supply chain itself, given the significant cost component that drugs/vaccines represent in several disease areas, reducing the number of commodities lost or wasted can in turn reduce overall program costs.

5.3 Relatedly, for laboratory operations, optimized laboratory network design should be implemented, which incorporates appropriate placement of more expensive laboratory equipment, strengthens referral, logistics and distribution systems, and, where relevant, appropriately balances deploying point-of-care diagnostics and referring samples to tertiary laboratories. Laboratory managers should continue to pursue the integration of laboratory services across vertical disease programs, capitalizing (for example) on cross-disease synergies in molecular assay testing of HIV and TB.

66

Appendices

Appendix 1: Experts and stakeholders interviewed

The authors would like to express their gratitude to the following experts and stakeholders who made themselves

available to be interviewed for this project:

Carlos Avila Abt Associates

Sergio Bautista-Arredondo National Institute of Public Health (INSP, Mexico)

Michael Borowitz Global Fund

Logan Brenzel Bill and Melinda Gates Foundation

Win Brown Bill and Melinda Gates Foundation

Rudolph Chandler Avenir Health

Steve Cohen Strategic Development Consultants, South Africa

Jacqueline Darroch Guttmacher Institute

Chad Davenport Partnership for Supply Chain Management

Charlotte Dolenz Clinton Health Access Initiative

Christopher Game Global Fund

Yvette Gerrans Bill and Melinda Gates Foundation

Marelize Gorgens World Bank

Kate Harris Bill and Melinda Gates Foundation

Sue Horton University of Waterloo

José-Antonio Izazola-Licea UNAIDS

David Jamieson Partnership for Supply Chain Management

Zachary Katz Clinton Health Access Initiative

Andrew Kennedy Global Fund

Nthabiseng Khoza National Department of Health, South Africa

Carol Levin University of Washington

Adri Mansvelder Dept. of Health Kwazulu-Natal, South Africa

Elliot Marseille University of California, San Francisco

Bruno Moonen Bill and Melinda Gates Foundation

Nhlanhla Ndlovu Centre for Economic Governance and AIDS in Africa

Regina Ombam National AIDS Control Council (NACC), Kenya

Mead Over Center for Global Development

Richard Owens Partnership for Supply Chain Management

John Palen PEPFAR, Office of Sustainability and Development

Linda Parsons Centers for Disease Control and Prevention (formerly)

Edith Patouillard World Health Organization

Ellen Piwoz Bill and Melinda Gates Foundation

Carel Pretorius Avenir Health

Raja Rao Bill and Melinda Gates Foundation

Stephen Resch Harvard T.H. Chan School of Public Health (HSPH)

Sydney Rosen Boston University School of Public Health

Helen Saxenian Independent

Andrew Siroka World Health Organization

Karin Stenberg World Health Organization

Todd Summers Independent

Elya Tagar Clinton Health Access Initiative

Anna Vassall London School of Hygiene and Tropical Medicine

Damian Walker Bill and Melinda Gates Foundation

Paul Wilson Columbia University Mailman School of Public Health

Nichole Zlatunich The Palladium Group

Landscape Study of the Cost, Impact, and Efficiency of Above Service Delivery Activities in HIV and Other Global Health Programs 67

Appendix 2: Literature search protocol

R4D’s research team conducted the following PubMed search for articles related to ASD costs on August 20, 2015:

This search returned 218 results. A high-level review of the returned results indicated additional terms potentially

used to describe ASD costs (“operational costs,” “off-site costs,” “district costs,” and “national costs”). A second

PubMed search was conducted on August 20, 2015 using these additional search terms; the second search yielded

49 results not previously identified in the initial PubMed search.

The program associate reviewed the title and abstracts of the combined 267 search results for relevance to the

project scope, both in terms of focus and geographic location (i.e. developing countries). In total, 197 articles were

excluded, with the remaining 70 articles retained for additional review by the program associate and the senior

program officer.

In addition, the research team conducted targeted searches for articles related to “laboratory strengthening,”

“laboratory service,” and “aid architecture” to supplement materials for fourth chapter of this report.

Additional articles identified by informants were screened by the research team for relevancy and further review.

Terms Located Boolean Operator

Tuberculosis OR HIV OR Malaria OR Immunization OR “Immunization programs” OR Vaccination OR Vaccine OR “Family Planning”

Title/Abstract AND

Cost* OR Costs OR Costing Title AND

“above service” OR “above delivery” OR “above service delivery” or “above facility” or “above the facility” OR “above the point of service” OR “program level cost*” OR “programme level cost*” OR “program cost” OR “programme cost” OR “above site cost*” OR “above the site” OR “high level cost*” OR “programme management cost*” OR “program management cost*” OR “overhead cost*” OR “central support cost*” OR “administrative cost*” OR “supply chain”

All Fields AND

“English” Filter AND

“published last 5 years” Filter

68

National Health Account data is collected via a standardized process in countries and aims to capture all healthcare

spending in a country in a systematic way, including breakdowns across different activities. These activities vary

slightly depending on the year the data were collected, but include broad categories such as inpatient curative

care, outpatient curative care, preventive care, and governance and health system and financing administration.

Over time, some countries have also reported details on health sub-accounts for specific programs, such as HIV/

AIDS, TB, malaria, and reproductive health.

Although the data do not break out activities by location (e.g., facility-level vs district-level or national-level), in

theory these data could provide some helpful high-level comparisons across countries and across programs within

countries. However, an initial examination of select countries raised questions about the data comparability in the

health sub-accounts.

Table 14 below illustrates some consolidated data from the health sub-accounts of 7 sub-Saharan African countries.

Given the categories provided in the sub-accounts, we initially focused on extracting information on governance

and administration costs, for which the majority of costs would be expected to take place above the point of

service delivery. The relative share of these governance and administration costs across countries and sub-

accounts is shown in the table.

Table 14. Consolidated data from the health sub-accounts of 7 sub-Saharan African countries

% share of spending on governance and administration

Kenya (2012-13)

Tanzania (2009-2010)

Malawi (2011-2012)

Burkina Faso

(2013)

Ghana (2012)

Niger (2013)

DRC (2013)

HIV/AIDS 17% 2% 13%        

Malaria 16% 1% 13% 19% 0% 30% 5%

TB 18%     32% 0% 60% 12%

Reproductive Health 21% 0% 16%        

Nutritional deficiencies 22%            

Vaccine-preventable diseases 48%            

Several cells are highlighted here to illustrate some of our data concerns:

• All spending values for Tanzania and Ghana appear to be implausibly low

• Within Niger’s NHA data, spending for TB appears to be implausibly high

• Kenya’s NHA spending amount for vaccine-preventable diseases appears implausibly out of step with other

disease areas, as well as being contrary to the findings of the carefully collected EPIC study data

Our initial examination of these data suggested that there are issues of data comparability across and within

countries in the historic NHA sub-accounts, and the expenditure categories are, at best, only partially informative

about program expenditures above the point of service delivery. For this reason, we did not pursue analysis of these

data further; future Systems of Health Accounts data are likely to be more comparable, and therefore more useful

for understanding ASD costs.

Appendix 3: Challenges with using historic National Health Account data

Landscape Study of the Cost, Impact, and Efficiency of Above Service Delivery Activities in HIV and Other Global Health Programs 69

References

Alash’le G, A., Croxton, T., Akintunde, E., Okelade, B., Jugu, J., Peters, S., . . . Constantine, N. (2010). Experiences in establishing a PEPFAR-supported laboratory quality system in Nigeria. American Journal of Clinical Pathology, 134(4), 541-549. http://dx.doi.org/10.1309/AJCP5RP4QWEQLUZR.

Alemnji, G. A., Zeh, C., Yao, K., & Fonjungo, P. N. (2014). Strengthening national health laboratories in sub-Saharan Africa: a decade of remarkable progress. Tropical Medicine & International Health, 19(4), 450-458. http://dx.doi.org/10.1111/tmi.12269.

Andiric, L. R., & Massambu, C. G. (2015). Laboratory quality improvement in Tanzania. American Journal of Clinical Pathology, 143(4), 566-572. http://dx.doi.org/10.1309/AJCPAB4A6WWPYIEN.

Aronovich, D., Tien, M., Collins, E., Sommerlatte, A., & Allain, L. (2010). Measuring supply chain performance: Guide to key performance indicators for public health managers. Arlington, Va.: USAID/DELIVER PROJECT, Task Order 1.

Bachmann, M. O. (2009). Cost effectiveness of community-based therapeutic care for children with severe acute malnutrition in Zambia: decision tree model. Cost Effectiveness and Resource Allocation, 7(2). http://dx.doi.org/10.1186/1478-7547-7-2.

Berti, P. R., Krasevec, J., & FitzGerald, S. (2004). A review of the effectiveness of agriculture interventions in improving nutrition outcomes. Public Health Nutrition, 7(5), 599-609.

Brenzel, L. (2015). What have we learned on costs and financing of routine immunization from the comprehensive multi-year plans in GAVI eligible countries? Vaccine, 33(Suppl. 1), A93-A98. http://dx.doi.org/10.1016/j.vaccine.2014.12.076.

Brenzel, L. (2014). Common approach to the costing and financing analyses for routine immunization and New Vaccine Introduction (NUVI). Mimeograph. Bill & Melinda Gates Foundation.

Brenzel, L., & Politi, C. (2012). Historical analysis of the comprehensive multi-year plans in GAVI-eligible countries (2004–2015). World Health Organization: Immunization Vaccines and Biologicals and Bill & Melinda Gates Foundation.

Brenzel, L., Young, D., Walker, D.G. (2015). Costs and financing of routine immunization: Approach and selected findings of a multi-country study (EPIC). Vaccine, 7(33), A13-20. http://dx.doi.org/10.1016/j.vaccine.2014.12.066.

Carvalho, N., Salehi, A.S., & Goldie, S.J. (2013). National and sub-national analysis of the health benefits and cost-effectiveness of strategies to reduce maternal mortality in Afghanistan. Health Policy and Planning, 28,62–74. http://dx.doi.org/10.1093/heapol/czs026.

Castaneda-Orjuela, C., Romero, M., Arce, P., Resch, S., Janusz, C. B., Toscano, C. M., & De la Hoz-Restrepo, F. (2013). Using standardized tools to improve immunization costing data for program planning: The cost of the Colombian Expanded Program on Immunization. Vaccine, 31(Suppl. 3), C72-C79. http://dx.doi.org/10.1016/j.vaccine.2013.05.038.

Chandrashekar, S., Guiness, L., Pickles, M., Shetty, G. Y., Alary, M., Vickerman, P., . . . Vassall, A. (2014). The costs of scaling up HIV prevention for high risk groups: Lessons learned from the Avahan programme in India. PLOS ONE 9(9), e106582. http://dx.doi.org/10.1371/journal.pone.0106582.

cMYP: A Tool and User Guide for cMYP Costing and Financing: Update 2014. wHO/IVB/14.06.

Dangour, A. D., Watson, L., Cumming, O., Boisson, S., Che, Y., Velleman, Y, . . . Uauy, R. (2013). Interventions to improve water quality and supply, sanitation and hygiene practices, and their effects on the nutritional status of children. Cochrane Database of Systematic Reviews, 8. http://dx.doi.org/10.1002/14651858.CD009382.pub2.

Deo, S., & Sohoni, M. (2015). Optimal decentralization of early infant diagnosis of HIV in resource-limited settings. Manufacturing & Service Operations Management, 17(2), 191-207. http://dx.doi.org/10.1287/msom.2014.0512.

Dheda, K., Theron, G., & Welte, A. (2014). Cost-effectiveness of Xpert MTB/RIF and investing in health care in Africa. The Lancet Global Health, 2(10), e554-e556.

70

Dutta, A., Barker, C., & Kallarakal, A. (2015). The HIV treatment gap: Estimates of the financial resources needed versus available for scale-up of antiretroviral therapy in 97 countries from 2015 to 2020. PLoS Med. 12(11):e1001907. http://dx.doi.org/10.1371/journal.pmed.1001907.

Ebel, T., Larsen, E., & Shah, K. (2013, September). Strengthening health care’s supply chain: A five-step plan. McKinsey & Company. Retrieved from http://www.mckinsey.com/insights/health_systems_and_services/strengthening_health_cares_supply_chain_a_five_step_plan.

Family Planning 2020 (FP2020). (n.d.). Retrieved from http://www.familyplanning2020.org/.

Fan, V., Silverman, R., Duran, D., & Glassman, A. (2013). The financial flows of PEPFAR: A profile. CGD Policy Paper 027. Washington DC: Center for Global Development.

Federal Government of Nigeria. (2014). Nigeria Family Planning Blueprint (Scale-up Plan). Retrieved from http://www.healthpolicyproject.com/ns/docs/CIP_Nigeria.pdf.

Garrett, L. (2008). All unquiet on the PEPFAR front. Council on Foreign Relations. Retrieved from http://www.cfr.org/content/thinktank/GlobalHealth/GHU_FoodCrisis_Jul208.pdf.

Government of Ghana. (2015). Ghana Family Planning Costed Implementation Plan. Accra: Ghana Health Service. Retrieved from http://www.healthpolicyproject.com/ns/docs/Ghana_FP_CIP_9_28.pdf. Cost model available at http://www.healthpolicyproject.com/index.cfm?ID=topics-FP2020.

Government of Malawi. (2015). Malawi Costed Implementation Plan for Family Planning, 2016–2020. Lilongwe: Government of Malawi. Retrieved from http://www.healthpolicyproject.com/ns/docs/Malawi_CIP_FINAL.pdf.

Hassan, A. S., Sakwa, E. M., Nabwera, H. M., Taegtmeyer, M. M., Kimutai, R. M., Sanders, E. J., . . . Berkley, J. A. (2012). Dynamics and constraints of early infant diagnosis of HIV infection in rural Kenya. AIDS and Behavior, 16(1), 5-12. http://dx.doi.org/10.1007/s10461-010-9877-7.

Hasselback, L., Spisak, C., & Crawford, J. (2012). Mozambique and Nigeria: Using results from supply chain costing. Arlington, Va.: USAID | Strategic Sub-Objective 2c: PROJECT, Task Order 4.

Health Policy Project (HPP). (2015). Retrieved from http://www.healthpolicyproject.com/index.cfm?ID=topics-FP2020.

Hinrichs, S., Jahagirdar, D., Miani, C., Guerin, B., & Nolte, E. (2014). Learning for the NHS on procurement and supply chain management: A rapid evidence assessment. Health Services and Delivery Research, 2(55). http://dx.doi.org/10.3310/hsdr02550.

Horton, S., Shekar, M., McDonald, C., Mahal, A., & Brooks, J.K. (2010). Scaling Up Nutrition: What Will It Cost? Washington D.C.: The World Bank.

Kempers, J., Ketting, E., & Lesco, G. (2014). Cost analysis and exploratory cost-effectiveness of youth-friendly sexual and reproductive health services in the Republic of Moldova. BMC Health Services Research, 14, 316. http://dx.doi.org/10.1186/1472-6963-14-316.

Langley, I., Lin, H. H., Egwaga, S., Doulla, B., Ku, C. C., Murray, M., . . . Squire, S. B. (2014). Assessment of the patient, health system, and population effects of Xpert MTB/RIF and alternative diagnostics for tuberculosis in Tanzania: an integrated modelling approach. The Lancet Global Health, 2(10), e581-e591.

Lawrence, Y., & Garcia Baena, I. (2015). Presentation of cost data availability. TB-MAC Global post-2015 TB Targets Meeting. Geneva, Switzerland.

Le Gargasson, J.B., Nyonator, F.K., Adibo, M., Gessner, B.D., & Colombini, A. (2015). Costs of routine immunization and the introduction of new and underutilized vaccines in Ghana. Vaccine, 7(33), A40-6. http://dx.doi.org/10.1016/j.vaccine.2014.12.081.

Landscape Study of the Cost, Impact, and Efficiency of Above Service Delivery Activities in HIV and Other Global Health Programs 71

Marseille, E., Giganti, M.J., Mwango, A., Chisembele-Taylor, A., Mulgena, L., Over, M., . . . Stringer, J.S.A. (2012). Taking ART to scale: Determinants of the cost and cost-effectiveness of antiretroviral therapy in 45 clinical sites in Zambia. PLOS ONE 7(12), e51993. http://dx.doi.org/10.1371/journal.pone.0051993.

McCord, J., Tien, M., & Sarley, D. (forthcoming). Guide to public health supply chain costing: A basic methodology. Arlington, Va.: USAID/DELIVER PROJECT, Task Order 4.

Ministry of Community Development, Mother and Child Health of Zambia. (2013). Family Planning Services: Integrated Family Planning Scale-up Plan 2013–2020. Retrieved from http://www.healthpolicyproject.com/ns/docs/CIP_Zambia.pdf.

Ministry of Health and Public Hygiene of Guinea. (2013). Plan d’action national de repositionnement de la planification familiale en Guineé 2014–2018. Retrieved from http://www.healthpolicyproject.com/ns/docs/CIP_Guinea.pdf.

Ministry of Health and Public Hygiene of Malawi. (2014). Plan d’action national de planification familiale du Mali 2014–2018. Retrieved from http://www.healthpolicyproject.com/ns/docs/Plan_d_Action_National_PF_du_Mali_2014_2018FINAL.pdf.

Ministry of Health and Sanitation of Sierra Leone. (2011). Sierra Leone Malaria Control Strategic Plan, 2011-2015. Retrieved from http://www.rollbackmalaria.org/files/files/countries/sierra-leone2011-2015.pdf.

Ministry of Health and Social Services of Namibia. (2010). Malaria Strategic Plan (2010-2016). Retrieved from http://www.rollbackmalaria.org/files/files/countries/namibia2010-2016.pdf.

Ministry of Health of Benin. (2013). Plan d’action national budgetise pour le repositionnement de la planification familiale 2014–2018 au Benin. Retrieved from http://www.healthpolicyproject.com/ns/docs/CIP_Benin.pdf.

Ministry of Health of Malawi. (2011). Malaria Strategic Plan, 2011-2015. Retrieved from http://www.rollbackmalaria.org/files/files/countries/malawi2011-2015.pdf.

Ministry of Health of Mauritania. (2013). Plan national de repositionnement de la planification familiale 2014–2018. Retrieved from http://www.healthpolicyproject.com/ns/docs/CIP_Mauritania.pdf.

Ministry of Health of Myanmar. (2015). Costed Implementation Plan to Meet FP2020 Commitments. Retrieved from http://ec2-54-210-230-186.compute-1.amazonaws.com/wp-content/uploads/2015/08/FPAP-April-2015-formatted_30June2015_wt-signature.pdf.

Ministry of Health of Nigeria. (2014). National Malaria Strategic Plan, 2014-2020. Retrieved from http://www.nationalplanningcycles.org/sites/default/files/planning_cycle_repository/nigeria/nigeria_national_malaria_strategic_plan.pdf.

Ministry of Health of Togo. (2015). Plan d’action pour le repositionnement de la planification familliale au Togo 2013–2017. Retrieved from http://www.healthpolicyproject.com/ns/docs/CIP_Togo.pdf.

Ministry of Health of Uganda. (2014). The Uganda Malaria Reduction Strategic Plan, 2014-2020. Retrieved from health.go.ug/download/file/fid/526.

Ministry of Health of Uganda. (2014). Uganda Family Planning Costed Implementation Plan, 2015–2020. Retrieved from http://www.healthpolicyproject.com/ns/docs/CIP_Uganda.pdf.

Ministry of Public Health of Niger. (2012). Planification familiale au Niger: plan d’action 2012–2020. Retrieved from http://www.healthpolicyproject.com/ns/docs/CIP_Niger.pdf.

72

Mvundura, M., Lorenson, K., Chweya, A., Kigadye, R., Bartholomew, K., Makame, M., . . . Kristensen, D. (2015). Estimating the costs of the vaccine supply chain and service delivery for selected districts in Kenya and Tanzania. Vaccine, 33(23), 2697-703. http://dx.doi.org/10.1016/j.vaccine.2015.03.084.

Nader, A. A., de Quadros, C., Politi, C., & McQuestion, M. (2014). An analysis of government immunization program expenditures in lower and lower middle income countries 2006-12. Health Policy and Planning, 30(3), 281-288. http://dx.doi.org/10.1093/heapol/czu002.

Nuwagaba-Biribonwoha, H., Werq-Semo, B., Abdallah, A., Cunningham, A., Gamaliel, J. G., Mtunga, S., . . . Nash, D. (2010). Introducing a multi-site program for early diagnosis of HIV infection among HIV-exposed infants in Tanzania. BMC Pediatrics, 10(1), 44. http://dx.doi.org/10.1186/1471-2431-10-44.

OECD, Eurostat, & World Health Organization. (2011). A System of Health Accounts: 2011 Edition. Retrieved from http://www.who.int/entity/health-accounts/methodology/sha2011.pdf.

Office of the United State Global AIDS Coordinator. (2013). Congressional Budget Justification Supplement, Fiscal Year 2014. Washington DC: A/GIS/GPS.

Office of the United State Global AIDS Coordinator. (2015). Congressional Budget Justification Supplement, Fiscal Year 2016. Washington DC: A/GIS/GPS.

Pantoja, A., Fitzpatrick, C., Vassall, A., Weyer, K., & Floyd, K. (2013). Xpert MTB/RIF for diagnosis of tuberculosis and drug-resistant tuberculosis: a cost and affordability analysis. European Respiratory Journal, 42(3), 708-720.

Petti, C. A., Polage, C. R., Quinn, T. C., Ronald, A. R., & Sande, M. A. (2006). Laboratory medicine in Africa: a barrier to effective health care. Clinical Infectious Diseases, 42(3), 377-382.

PMI. (2014). Eight Annual Report to Congress. Washington DC: President’s Malaria Initiative.

Portnoy, A., Ozawa, S., Grewal, S., Norman, B. A., Rajgopal, J., Gorham, K. M., . . . Lee, B. Y. (2015). Costs of vaccine programs across 94 low- and middle-income countries. Vaccine. 33(Suppl 1), A99-108. http://dx.doi.org/10.1016/j.vaccine.2014.12.037.

Puett, C., Sadler, K., Alderman, H., Coates, J., Fiedler, J. L., & Myatt, M. (2013). Cost-effectiveness of the community-based management of severe acute malnutrition by community health workers in southern Bangladesh. Health Policy and Planning, 28(4), 386-399. http://dx.doi.org/10.1093/heapol/czs070.

Ramachandran, V., & Walz, J. (2013). US Spending in Haiti: The Need for Greater Transparency and Accountability. Retrieved from http://www.cgdev.org/publication/ft/us-spending-haiti-need-greater-transparency-and-accountability.

Republic of Cameroon. (2014). Plan operational de planification familiale: 2015–2020. Retrieved from http://www.healthpolicyproject.com/ns/docs/Cameroon_CIPfinal.pdf.

Reyburn, H., Mbatia, R., Drakeley, C., Carneiro, I., Mwakasungula, E., Mwerinde, O., . . . Greenwood, B. M. (2004). Overdiagnosis of malaria in patients with severe febrile illness in Tanzania: a prospective study. BMJ, 329(7476), 1212.

Riewpaiboon, A., Sooksriwong, C., Chaiyakunapruk, N., Tharmaphornpilas, P., Techathawat, S., Rookkapan, K., . . . Suraratdecha, C. (2015). Optimizing national immunization program supply chain management in Thailand: An economic analysis. Public Health, 129(7), 899-906. http://dx.doi.org/10.1016/j.puhe.2015.04.016.

Sabot, O., Cohen, J. M., Hsiang, M. S., Kahn, J. G., Basu, S., Tang, L., . . . Aboobakar, S. (2010). Costs and financial feasibility of malaria elimination. The Lancet, 376(9752), 1604-1615.

Sarley, D., Baruwa, E., & Tien, M. (2010). Zimbabwe: Supply Chain Costing of Health Commodities.

Landscape Study of the Cost, Impact, and Efficiency of Above Service Delivery Activities in HIV and Other Global Health Programs 73

Schütte, C., Chansa, C., Marinda, E., Guthrie, T.A., Banda, S., Nombewu, Z., . . . Kinghorn, A. (2015). Cost analysis of routine immunisation in Zambia. Vaccine, 33(Suppl 1), A47-52. http://dx.doi.org/10.1016/j.vaccine.2014.12.040.

Shretta, R., Johnson, B., Smith, L., Doumbia, S., de Savigny, D., Anupindi, R., Yadav, P. (2015). Costing the supply chain for delivery of ACT and RDTs in the public sector in Benin and Kenya. Malaria Journal, 5(14), 57. http://dx.doi.org/10.1186/s12936-014-0530-1.

Shretta, R., & Yadav, P. (2012). Stabilizing supply of artemisinin and artemisinin-based combination therapy in an era of wide-spread scale-up. Malaria Journal, 11(399), 1-10.

Siapka, M., Remme, M., Obure, C. D., Maier, C. B., Dehne, K. L., & Vassall, A. (2014). Is there scope for cost savings and efficiency gains in HIV services? A systematic review of the evidence from low- and middle-income countries. Bulleting of the World Health Organization, 92, 499-511. http://dx.doi.org/10.2471/BLT.13.127639.

Singh, S., Darroch, J. E., & Ashford, L. S. (2014). Adding It Up: The costs and benefits of investing in sexual and reproductive health 2014. New York: Guttmacher Institute.

SPRING. (2015, March). Validated estimates of Nepal nutrition financing; FY 2013/14 (70/71) and 2014/15 (71/72).

SUN Secretariat. (2014). Planning and costing for the acceleration of actions for nutrition: experiences of countries in the Movement for Scaling Up Nutrition. Geneva: SUN Movement.

Tagar, E., Sundaram, M., Condliffe, K., Matatiyo, B., Chimbwandira, F., Chilima, B., . . . Assefa, Y. (2014). Multi-country analysis of treatment costs for HIV/AIDS (MATCH): facility-level ART unit cost analysis in Ethiopia, Malawi, Rwanda, South Africa and Zambia. PLoS ONE 9(11): e108304. http://dx.doi.org/10.1371/journal.pone.0108304.

TB Modelling and Analysis Consortium (TB-MAC) (2015). TB-MAC Global post-2015 TB Targets Meeting Report. Geneva, Switzerland.

Theron, G., Pooran, A., Peter, J., van Zyl-Smit, R., Mishra, H. K., Meldau, R., . . . Dheda, K. (2012). Do adjunct tuberculosis tests, when combined with Xpert MTB/RIF, improve accuracy and the cost of diagnosis in a resource-poor setting? European Respiratory Journal, 40(1), 161-168.

The Global Fund (2015) Fact sheet: Strengthen Procurement, Global and In-Country Supply Chain Systems.

Tien, M., Baruwa, E., & Young, D. (forthcoming). Supply chain costing tool user’s manual. Beta Version. Arlington, Va.: USAID/DELIVER PROJECT, Task Order 4.

United Nations Population Fund (UNFPA). (2009). Revised Cost Estimates for the Implementation of the Programme of Action of the International Conference on Population and Development: A Methodological Report. New York: UNFPA.

USAID. (2010). Laboratory standardization: Lessons learned and practical approaches.

USAID/DELIVER PROJECT, Task Orders 4 and 7. (forthcoming). Improving the financial sustainability of Rwanda’s public health supply chain through cost analysis. Arlington, Va.

USAID/DELIVER PROJECT, Task Order 4. (2012). Procurement performance indicators guide—Using procurement performance indicators to strengthen the procurement process for public health commodities. Arlington, Va.

UN Secretariat, Department of Economic and Social Affairs. (2010). World Economic and Social Survey 2010: Retooling Global Development. New York: United Nations.

Vassall, A., van Kampen, S., Sohn, H., Michael, J. S., John, K. R., den Boon, S., . . . Khaliqov, A. (2011). Rapid diagnosis of tuberculosis with the Xpert MTB/RIF assay in high burden countries: a cost-effectiveness analysis. PLoS Medicine, 8(11), 1518.

74

Warren, C. E., Mayhew, S. H., Vassall, A., Kimani, J. K., Church, K., Obure, C. D., . . . Watts, C. (2012). Study protocol for the Integra Initiative to assess the benefits and costs of integrating sexual and reproductive health and HIV services in Kenya and Swaziland. BMC Public Health, 12, 973. http://dx.doi.org/10.1186/1471-2458-12-973.

Weissman, E. (2013). Scaling up sexual and reproductive health: Program and system costs. [Unpublished working draft; shared by author with disclosure restrictions].

Wilford, R., Golden, K., & Walker, D. G. (2012). Cost-effectiveness of community-based management of acute malnutrition in Malawi. Health Policy and Planning, 27(2), 127-137. http://dx.doi.org/10.1093/heapol/czr017.

WHO. (2005). Estimating the Cost of Scaling-up Maternal and Newborn Health Interventions to Reach Universal Coverage: methodology and assumptions. Technical Working Paper.

WHO (2011). Global supply of artemether-lumefantrine before, during, and after the Memorandum of Understanding between WHO and Novartis. Geneva: World Health Organization.

WHO IVB (2014). Analysis of Immunization Financing Indicators of the WHO-UNICEF Joint Reporting Form (JRF), 2006-2012.

WHO (2015). World Malaria Report, 2015.

Zlatunich, N. (2016, February 2). Senior Program Adviser, The Palladium Group. E-mail correspondence.

Zlatunich, N., & Reeves, M. (2015). Getting It Right: Lessons Learned for Family Planning Costed Implementation Plans. Washington, D.C.: Health Policy Project, Futures Group. Retrieved from http://www.healthpolicyproject.com/pubs/685_CIPLessonsLearnedbrief.pdf.

1111 19th Street, N.W., Suite #700

Washington, DC 20036

Tel: (202) 470.5711 | Fax: (202) 470.5712

info@r4d.org | www.r4d.org