creating a priority list of non-communicable diseases to
TRANSCRIPT
ISSN 1178-2293 (Online)
University of Otago
Economics Discussion Papers
No. 2003
APRIL 2020
Creating a priority list of non-communicable
diseases to support health research funding
decision-making
Saeideh Babashahi,1* Paul Hansen1,2 & Trudy Sullivan3
1 Department of Economics, University of Otago 2 1000minds Ltd, www.1000minds.com
3 Department of Preventive & Social Medicine, University of Otago
Address for correspondence:
Saeideh Babashahi
Department of Economics
University of Otago
PO Box 56
Dunedin
NEW ZEALAND
email: [email protected]
Telephone: 64 3 479 8645
1
Creating a priority list of non-communicable diseases to
support health research funding decision-making
Saeideh Babashahi,1* Paul Hansen1,2 & Trudy Sullivan3
1 Department of Economics, University of Otago 2 1000minds Ltd, www.1000minds.com
3 Department of Preventive & Social Medicine, University of Otago
* Corresponding author:
Saeideh Babashahi
Department of Economics
University of Otago
Dunedin
New Zealand
email: [email protected]
ph: +64 3 479 8645
2
Abstract
To develop and pilot a framework based on multi-criteria decision analysis (MCDA) for
creating a priority list of non-communicable diseases (NCDs) to support health research
funding decisions, where the NCDs are prioritised with respect to their overall burden to
society. The framework involves identifying NCDs to be prioritised, specifying prioritisation
criteria and determining their weights from a survey of stakeholders. The mean weights from
the survey are applied to the NCDs’ ratings on the criteria to generate a ‘total score’ for each
NCD, by which the NCDs are ranked (prioritised). Nineteen NCDs and five criteria were
included. The criteria, in decreasing order of importance (mean weights in parentheses), are:
deaths across the population (27.7%); loss of quality-of-life across the population (23.0%);
cost to patients, families and the community (18.6%); cost to the health system (17.2%); and
whether vulnerable groups are disproportionately affected (13.4%). The priority list of NCDs,
stratified into four tiers in decreasing order of importance, is: ‘Very critical’ priority: coronary
heart disease, back and neck pain, diabetes mellitus; ‘Critical’ priority: dementia and
Alzheimer’s disease, stroke; ‘High’ priority: colon and rectum cancer, depressive disorders,
chronic obstructive pulmonary disease, chronic kidney disease, breast cancer, prostate cancer,
arthritis, lung cancer; and ‘Medium’ priority: asthma, hearing loss, melanoma skin cancer,
addictive disorders, non-melanoma skin cancer, headaches. The results from applying the
MCDA-based framework for prioritising NCDs indicate that it is feasible and effective. The
framework could also be used to support health research funding decision-making for other
conditions.
Keywords: health research funding, non-communicable diseases (NCDs), priority-setting
framework, multi-criteria decision analysis (MCDA), PAPRIKA method
Acknowledgements: The article arose from the first author’s PhD thesis supervised by the
other authors and Ronald Peeters. Thank you to the experts consulted during the course of the
study, including Tony Blakely for his helpful comments on health care expenditure in New
Zealand. Thank you to the survey participants. The second author (PH) co-invented and co-
owns the 1000minds software mentioned in the article.
3
1. Introduction
Research into health problems affecting society, including health inequities borne by
vulnerable population sub-groups, is very important (Khan et al., 2019; Smith et al., 2009).
Although billions of dollars are invested in health research annually (Khan et al., 2019), only
a small proportion of this spending targets health problems imposing the greatest burden on
society (Allen, 2017; Viergever, 2013), and often with minimal attention to achieving a more
equitable distribution of health outcomes. Given limited resources available for health
research, prioritising areas for health research funding is necessary (Allen, 2017; Allen et al.,
2017; Viergever, 2013). Health research priority-setting aims to direct funding to research
areas of greatest need that will result in the biggest health gain, including reducing health
inequities (Allen, 2017; Allen et al., 2017). Despite the importance of such priority-setting
being generally well accepted, the development of priority lists of the most important research
‘investment opportunities’ remains challenging in practice (Rottingen et al., 2013; Smith et
al., 2009). Notwithstanding a large number of studies into prioritising health interventions per
se, very few studies have focused on prioritising health conditions, including non-
communicable diseases (NCDs) (Adeyi et al., 2008; Allen, 2016; Marsh et al., 2017), to
support health research funding – the subject of this paper.
NCDs are illnesses that are non-transmissible in the sense that they are not spread from
person to person and are often characterised by slow deterioration and long duration (Khan et
al., 2019; Strong et al., 2006; Vos et al., 2017). The four main groups of NCDs, according to
the World Health Organization (WHO), are cardiovascular diseases, cancers, chronic
respiratory diseases and diabetes (WHO, 2019). NCDs also include mental, neurological and
musculoskeletal disorders – major causes of health burden in many countries (Carroll, 2019;
Dieppe, 2013). Population aging and unhealthy lifestyles have resulted in an exponential
growth of NCDs (Bigna & Noubiap, 2019; Tripathy, 2018). Globally, NCDs are the leading
causes of mortality, morbidity and high health care expenditures (IHME, 2017; Muka et al.,
2015; Vos et al., 2017), contributing to 73% of years of life lost (YLL) due to premature
deaths and 80% of years lived with disability (YLD) (IHME, 2017; Prynn & Kuper, 2019;
Roth et al., 2018).
Given the growing burden of NCDs, it is important that valid and reliable methods are used to
prioritise NCDs for research funding. The WHO Global Forum for Health Research (GFHR)
emphasises the importance of using structured priority-setting frameworks to direct
investment into research areas, with particular attention to improving health equity and
supporting vulnerable population sub-groups (Viergever et al., 2010). Multi-criteria decision
analysis (MCDA) is a well-recognised tool for systematically setting priorities in the health
sector (Marsh et al., 2017; Tacconelli et al., 2018). MCDA has been widely endorsed, with
two reports written by the MCDA Emerging Good Practices Task Force of the International
Society for Pharmacoeconomics and Outcomes Research (ISPOR) published in Value in
Health (Hansen & Devlin, 2019; Marsh et al., 2017; Marsh et al., 2016; Thokala et al., 2016).
In this paper, an MCDA-based framework for prioritising NCDs with respect to their overall
burden to society – developed and piloted in New Zealand (NZ) – is presented. The resulting
4
priority list of NCDs is intended to be used to support research funding decision-making.
“Support” is emphasised because additional considerations such as the cost of the research
and its likelihood of success are also important factors (and not included in this framework)
when research projects are being assessed and, ultimately, funds are allocated in pursuit of
‘value for money’. The framework is similar (though, on a logistically smaller scale) to the
one used recently by the WHO to create a priority list of antibiotic-resistant bacteria to
support research and development into new antibiotics (Tacconelli et al., 2018). The present
study is the first to prioritise NCDs for health research funding.
2. Methods
Consistent with the recommendations of ISPOR’s MCDA Task Force (Marsh et al., 2016;
Thokala et al., 2016) and the process discussed in the article by Hansen and Devlin (Hansen &
Devlin, 2019), the MCDA-based framework for prioritising NCDs with respect to their
overall burden to society (and hence their importance for health research funding) involves
three key components: (1) identifying NCDs to be prioritised for health research funding; (2)
specifying prioritisation criteria and levels of performance within each criterion; and (3)
determining weights for the criteria (and their levels), representing their relative importance to
stakeholders.
Applying the information from these three components, each NCD can be rated according to
its ‘performance’ on each of the criteria. Each NCD’s ratings are aggregated using a linear
(i.e. additive) equation – also referred to as a ‘weighted-sum model’ or ‘points system’ – to
produce a ‘total score’ for each NCD (Hansen & Ombler, 2008; Tacconelli et al., 2018).
Finally, based on their total scores (in the range 0-100%), the NCDs are ranked (prioritised).
Each of the three key components mentioned above is now explained in turn, followed by a
brief discussion of the check of test-retest reliability of the survey used for determining the
criteria weights. The NCDs’ ratings on the criteria and their ranking are then explained.
Finally, a brief explanation of further analyses to explore possible associations between
participants’ preferences – i.e. their criteria weights – and their socio-demographic and
background characteristics is provided.
Identifying NCDs to be prioritised
The English-language literature was searched to identify NCDs responsible for large health
burdens and spending in NZ and those that disproportionately affect vulnerable groups such
as children, poor people, Māori (NZ’s indigenous people) and other ethnic minorities (Blakely
et al., 2019; MoH, 2009, 2016; OECD, 2017; Roth et al., 2018). This literature – academic
and grey – included published articles, papers, reports, health system reviews and the websites
of major international and national organisations such as the WHO, World Bank, Institute for
Health Metrics and Evaluation, Organisation for Economic Co-operation and Development,
NZ Ministry of Health and Health Research Council. Experts in NCDs from a wide range of
clinical, research and policy-making backgrounds in NZ were also consulted to corroborate
interpretations and improve understanding of the information gleaned from the literature
search.
5
Specifying prioritisation criteria
The literature was searched again, but more systematically this time, to discover criteria used
for priority-setting in health systems around the world, potentially in a variety of applications
(e.g. health technology prioritisation, etc). Articles, reports and grey literature for the period
1990-2019 were searched using PubMed’s and Google Scholar’s search engines and
combinations of these keywords: priority-setting, prioritisation criteria, social and ethical
issues, decision-making, health research funding and/or disease, non-communicable
disease(s) and NCD(s).
Because the weights on the criteria and the levels within each criterion are to be determined
using a survey (explained in the next sub-section), the criteria need to be specified concisely
using language that most survey participants would be expected to understand. Likewise, as
well as being concise and simple, the levels should be generic in nature – e.g. ranging from
‘low’ to ‘high’ – and sufficiently granular to be able to distinguish between the NCDs
included in the study. The final specification of the criteria and levels was validated and
refined following in-depth interviews with the experts referred to in the previous sub-section,
and pilot-testing of the framework with several participants from diverse backgrounds.
Determining criteria weights
The weights were determined by surveying people from three health sector groups: (1)
patients or members of the general public; (2) health providers (e.g. nurses or doctors); and
(3) health policy-makers or researchers. Consistent with the literature, patients and the general
public were included because they are the ultimate beneficiaries of health research (Kapiriri,
2018). The other two groups were included because of their expert knowledge and interest in
NCDs. Convenience and purposive sampling with ‘snowballing’ (Etikan et al., 2016;
Goodman, 1961; O’Haire et al., 2011; Street et al., 2014) – whereby participants who were
initially contacted were asked to forward the survey link to other eligible and potentially
interested people – was used to identify the participants, who were initially given two weeks
to complete the survey, with a reminder email sent out after 10 days.
The survey applied the PAPRIKA method for determining weights – an acronym for
Potentially All Pairwise RanKings of all possible Alternatives (Hansen & Ombler, 2008) – as
implemented by 1000minds software (www.1000minds.com). As mentioned earlier, this
method and software was used by the WHO to create its priority list of antibiotic-resistant
bacteria for research funding (Tacconelli et al., 2018). PAPRIKA and 1000minds have also
been widely used for patient (Hansen et al., 2012) and health technology prioritisation (Golan
& Hansen, 2012; Sullivan & Hansen, 2017), disease classification (Aringer et al., 2019) and
health preferences research (Sullivan et al., 2020).
The PAPRIKA method, in the present context, involves participants being asked to pairwise
rank two hypothetical NCDs defined on two criteria at a time and involving a trade-off – in
terms of which is more of a problem for society and therefore more eligible for research
funding. An example of a pairwise-ranking question from 1000minds software appears in
Figure 1.
6
Figure 1: Example of a pairwise-ranking question (screenshot from 1000minds software)
Such pairwise-ranking questions are repeated with different pairs of hypothetical NCDs –
always defined on two criteria at a time and involving a trade-off. Each time a participant
answers a question, the PAPRIKA method applies the logical property of transitivity to
identify and eliminate all other pairs of hypothetical NCDs defined on two criteria at a time
that are pairwise ranked, thereby minimising the number of questions asked. For example, if a
participant ranks NCD A ahead of NCD B and B ahead of NCD C, then A is ranked ahead of
C (and so would not be asked about). Each time a person answers a question, PAPRIKA
adapts with respect to choosing the next question (always one whose answer is not implied by
earlier answers) based on all preceding answers (Hansen & Ombler, 2008). This adaptivity
and the transitivity-based elimination procedure ensures the number of questions a participant
is asked is minimised while ensuring they end up having pairwise ranked all hypothetical
NCDs defined on two criteria at a time, either explicitly or implicitly (by transitivity).
The number of questions to be answered by participants was also reduced, thereby reducing
the elicitation burden, by utilising the software’s interpolation feature. For example, if a
criterion has five levels – e.g. ‘low’, ‘low to moderate’, ‘moderate’, ‘moderate to high’ and
‘high’ – then only the first (‘low’), third (‘moderate’) and fifth (‘high’) levels are included in
the pairwise-ranking questions. The weights for the second (‘low to moderate’) and fourth
(‘moderate to high’) levels are interpolated using Bézier interpolation (Farin et al., 2002) – i.e.
in essence, by fitting a smoothed curve through the weights for the first, third and fifth levels.
Thus, the granularity arising from having the full set of levels available for rating the NCDs
on the criteria is maintained while the number of questions that survey participants are asked
to answer is limited.
Two checks related to the quality of each participant’s data were performed by the software.
First, three pairwise-ranking questions were repeated at the end of each participant’s survey to
check the consistency of their answers. Second, any participants who answered all their
7
questions by clicking “they are equal” (see Figure 1 again) – i.e. universal indifference – were
identified, to gauge participants’ engagement with the survey.
From each participant’s answers to the pairwise-ranking questions, PAPRIKA uses linear
programming methods to determine weights for the criteria (and for the levels within each
criterion), representing their relative importance to the participant. Participants were also
asked questions about their socio-demographic and background characteristics, and how easy
or difficult they found answering the pairwise-ranking questions.
Checking test-retest reliability
As a check of the survey’s test-retest reliability, a sub-sample of 40 participants completed the
survey twice, almost two weeks apart. The mean criteria weights obtained from the two
implementations of the survey were tested for statistically significant differences.
Rating NCDs on the criteria
Each NCD was rated on the criteria using information from the literature, including YLL,
YLD and economic burden. This rating exercise was performed by the first author (SB) in
consultation with the other authors and the experts involved at the other stages of the study, as
mentioned earlier. The latest YLL and YLD data for NZ were obtained from the website of
the Institute for Health Metrics and Evaluation (IHME, 2019). NCDs’ economic burdens were
derived from reports and cost of illness (CoI) studies for NZ. International and national
reports and papers related to the NZ context were reviewed to identify NCDs
disproportionately affecting vulnerable groups (Blakely et al., 2019; MoH, 2009, 2016).
As there is very little information relating to indirect health care costs available, only direct
health care expenditure for each NCD was considered in the analysis. Cost estimates from
Blakely et al (2019), the best data source available at the time, were used to estimate the cost
burdens on the NZ Government – i.e. publicly-funded health care – and patients (and
families). According to that study, approximately 82% of direct health care costs are publicly-
funded, with the remaining 18% paid by individuals – e.g. co-payments, out-of-pocket
payments and private health insurance (Blakely et al., 2019). (Although this is the best data
source available, the estimated costs should be treated with caution.) Costs were adjusted for
inflation using the NZ general consumer price index (CPI) for 2017 via the Reserve Bank of
New Zealand inflation calculator (The Reserve Bank of New Zealand, 2017).
Ranking NCDs
Applying the mean criteria weights from the survey to the ratings of the NCDs generates a
‘total score’ for each NCD in the range 0-100%. Based on their total scores, the NCDs are
ranked (prioritised).
Predicting participants’ preferences
Two-step cluster analysis was performed to identify clusters of participants with similar
preferences (i.e. criteria weights). A chi-squared test (along with Cramér’s V) was used to
explore possible associations (and their effect size) between participants’ preferences and
8
their socio-demographic and background characteristics. As participants were sampled mainly
from the higher educated population, a t-test was used to determine whether there are
statistically significant differences between the preferences of participants with higher and
lower levels of education.
3. Results
NCDs to be prioritised
Twenty-one NCDs were initially identified. However, two of them – anxiety and dental
disorders – were excluded due to a paucity of information. These remaining 19 NCDs, in
alphabetical order, are: addictive (drug and alcohol use) disorders, arthritis, asthma, back and
neck pain, breast cancer, chronic kidney disease, chronic obstructive pulmonary disease,
colon and rectum cancer, coronary heart disease, dementia and Alzheimer’s disease,
depressive disorders, diabetes mellitus (mainly type 2), headaches, hearing loss, lung cancer,
melanoma skin cancer, non-melanoma skin cancer, prostate cancer and stroke.
Prioritisation criteria
The literature search revealed three main groups of criteria used for prioritising NCDs for
research funding: (1) burden of disease (BoD) – i.e. morbidity and mortality, including health-
related quality of life; (2) cost burden and efficiency; and (3) additional considerations such as
ethical and social issues (Drummond et al., 2015; Golan et al., 2011; Shmueli et al., 2017;
Thokala et al., 2016; Tromp & Baltussen, 2012).
The first of these three groups of prioritisation criteria, BoD – often described in terms of
quality- or disability-adjusted life years (QALYs or DALYs) or the latter’s two components,
YLL and YLD (IHME, 2017; Prynn & Kuper, 2019; Roth et al., 2018; Tromp & Baltussen,
2012; Vos et al., 2017) – is the most widely-used criterion for evaluating health losses and
prioritising health conditions, including NCDs. The second group recognises that NCDs
impose substantial health care costs (Allen, 2017; Muka et al., 2015; Smith et al., 2009) in
terms of publicly-funded health care costs, and expenses incurred by patients and families’
including out-of-pocket expenses, as well as the value of unpaid and informal family support
(Drummond et al., 2015). The third main group is associated with reducing health inequities
affecting vulnerable population sub-groups such as children, poor people and ethnic
minorities, including indigenous people, as these groups experience a disproportionately
larger share of the disease burden (Best, 2012). As reported in Table 1, five criteria were
specified for prioritising NCDs, with 2-7 levels within each criterion. Table 1 also
summarises the health system goals addressed by each criterion and the data source used to
assess the NCDs on each criterion (Tromp & Baltussen, 2012).
9
Table 1: Criteria for prioritising NCDs
Criterion Level Health system
goal addressed
Data source
Deaths across the population –
i.e. reduced life expectancy
None (or low)
Low to moderate
Moderate
Moderate to high
High
High to very high
Very high
Mortality Latest NZ YLL data
(2017) extracted from the
IHME website.
Loss of quality-of-life across
the population – e.g. pain,
disability
Low
Low to moderate
Moderate
Moderate to high
High
Morbidity Latest NZ YLD data
(2017) extracted from the
IHME website.
Cost of the disease to the
health system – i.e. publicly-
funded health care
Low
Low to moderate
Moderate
Moderate to high
High
Health system
costs
CoI studies, reports and
papers – e.g. from MoH,
BODE3 and PHARMAC.
Cost of the disease to patients,
families and community – e.g.
unpaid family support
Low
Low to moderate
Moderate
Moderate to high
High
Societal costs
Disproportionately affects
vulnerable groups – e.g.
Māori, children, poor people
Yes
No
Equity Reports and papers –e.g.
from WHO, IHME,
OECD, MoH, HRC,
DHBs.
BODE3: Burden of Disease Epidemiology, Equity and Cost-Effectiveness Programme. BODE3 is a
funded program by the Health Research Council (HRC) of New Zealand (NZ) to provide health
economics data. CoI: Cost of Illness. DHBs: 20 District Health Boards in NZ are responsible for
providing health care services to the population within their districts. IHME: Institute for Health
Metrics and Evaluation. MoH: Ministry of Health. OECD: Organisation for Economic Co-operation
and Development. PHARMAC: the Pharmaceutical Management Agency is an NZ crown entity that
decides which pharmaceutical products and devices are subsidised for use in the public sector. YLD:
Years Lived with Disability. YLL: Years of Life Lost due to premature deaths. WHO: World Health
Organization.
Criteria weights
The survey for determining the criteria weights required participants to answer 20 pairwise-
ranking questions on average, taking 15-20 minutes in total. The survey was completed by
517 participants; however, 27 (5%) answered all three repeated questions contradictorily
(inconsistently), and another 14 (3%) answered ‘they are equal’ for all questions (universal
indifference). These participants were excluded from the data set because both behaviours are
suggestive of the participants not having engaged seriously with the pairwise-ranking exercise
10
or not having understood the questions. The remaining 476 participants answered at least one
of the three repeated questions identically (consistently), 356 (75%) answered two questions
consistently and 168 (35%) answered all three consistently.
The socio-demographic and background characteristics of these 476 participants are
summarised in Table 2, where, inter alia, it can be seen that almost 31% were patients or
members of the general public, 35% were health providers (e.g. nurses or doctors) and 34%
were health policy-makers or researchers. Almost 53% of participants said they found
completing the survey and answering the pairwise-ranking questions relatively easy.
The mean weights of the criteria and their levels, representing their relative importance to
participants, are reported in Table 3 with their standard deviations (SD). The most important
criterion for prioritising NCDs with respect to their overall burden to society (and hence their
importance for health research funding, all else being equal) is ‘deaths across the population’
(mean weight = 27.7%), followed by ‘loss of quality-of-life across the population’ (23.0%),
‘cost to patients, families and community’ (18.6%), ‘cost to the health system’ (17.2%) and –
the least-important criterion – ‘disproportionately affects vulnerable groups’ (13.4%). Criteria
weights indicate the relative strength of participants’ preferences with respect to reducing the
multi-dimensional burden of NCDs on society.
Test-retest reliability
For the 40 people who completed the MCDA survey twice, the results of a paired sample t-
test revealed no statistically significant differences between the mean criteria weights from
the first and second surveys.
Ratings of NCDs on the criteria
The ratings of the 19 NCDs on the five criteria are reported in Table 4.
11
Table 2: Summary of characteristics of survey participants (n=476)
Characteristic n (%)
Gender
Male 201 (42.2)
Female 274 (57.6)
Gender diverse 1 (0.2)
Age (years)
18-24 15 (3.2)
25-34 62 (13.0)
35-44 108 (22.7)
45-54 129 (27.1)
55-64 114 (23.9)
65 and over 48 (10.1)
Ethnicity
NZ European 302 (63.4)
Māori 45 (9.5)
Chinese 42 (8.8)
Pacific 18 (3.8)
Indian 29 (6.1)
Others 40 (8.4)
Qualification
No qualification 30 (6.3)
Secondary school 47 (9.9)
Post-secondary school qualification 72 (15.1)
University degree equivalent 327 (68.7)
Region
North Island 325 (68.3)
South Island 151 (31.7)
Work status
Working 312 (65.5)
Not working 73 (15.3)
Retired 91 (19.1)
Participant (or immediate family member) with NCD(s)
Yes 373 (78.4)
No 103 (21.6)
Use of health care services
Never 4 (0.8)
Occasionally 373 (78.4)
Frequently 99 (20.8)
Background
Patient or member of the general public 149 (31.3)
Health provider – e.g. nurse or doctor 166 (34.9)
Health policy-maker or researcher 161 (33.8)
Ease of completing the survey and answering pairwise-ranking questions
Relatively easy 250 (52.5)
Relatively difficult 226 (47.5)
Percentages for ethnicity do not sum to 100 as some people identify with multiple groups.
12
Table 3: Mean criteria weights (in decreasing order of importance)
Criterion and levels
Mean
weight
(SD)
Deaths across the population – i.e. reduced life expectancy
None (or low) 0.0 (0.0)
Low to Moderate 6.9
Moderate 12.7 (5.0)
Moderate to high 17.0
High 20.4 (6.5)
High to Very high 24.1
Very high 27.7 (8.1)
Loss of QoL across the population – e.g. pain, disability
Low 0.0 (0.0)
Low to Moderate 1.0
Moderate 11.9 (4.8)
Moderate to high 17.5
High 23.0 (6.3)
Cost of the disease to the health system – i.e. publicly-funded health care
Low 0.0 (0.0)
Low to Moderate 4.3
Moderate 8.6 (4.7)
Moderate to high 12.9
High 17.2 (6.7)
Cost of the disease to patients, families and community – e.g. unpaid family support
Low 0.0 (0.0)
Low to Moderate 4.7
Moderate 9.3 (4.2)
Moderate to high 14.0
High 18.6 (6.3)
Disproportionately affects vulnerable groups – e.g. Māori, children, poor people
No 0.0 (0.0)
Yes 13.4 (6.3)
SD: Standard Deviation is only available for the levels presented in the survey. Note that other levels
are interpolated using Bézier interpolation, as explained in the context. Values reported are
percentages. Bolded values sum to one (100%) and represent the relative weights of the criteria
overall.
13
Table 4: Ratings of the NCDs on the criteria
NCD
Deaths across
the population
– i.e. reduced
life expectancy
Loss of QoL
across the
population –
e.g. pain,
disability
Cost of the
disease to the
health system
– i.e. publicly-
funded health
care
Cost of the
disease to
patients,
families and
community –
e.g. unpaid
family support
Disproportionately
affects vulnerable
groups – e.g.
Māori, children,
poor people
Addictive
disorders None (or low)
Low to
moderate Low Low Yes
Arthritis None (or low) Low to
moderate Moderate Moderate Yes
Asthma None (or low) Low to
moderate
Low to
moderate
Low to
moderate Yes
Back and neck
pain None (or low) High High High Yes
Breast cancer None (or low)
to Moderate Low Moderate Moderate Yes
CHD Very high Low High High Yes
CKD None (or low)
to Moderate Low Moderate Moderate Yes
Colon and
rectum cancer Moderate Low Moderate Moderate Yes
COPD Moderate Low to
moderate
Low to
moderate
Low to
moderate Yes
Dementia and
Alzheimer’s
disease
Moderate to
High
Low to
moderate High High No
Depressive
disorders None (or low) Moderate Moderate Moderate Yes
Diabetes
mellitus
None (or low)
to Moderate Moderate High High Yes
Headaches None (or low) Moderate Low Low No
Hearing loss None (or low) Moderate Low Low Yes
Lung cancer Moderate Low Low to
moderate
Low to
moderate Yes
Melanoma skin
cancer
None (or low)
to Moderate Low Low Low Yes
Non-melanoma
skin cancer None (or low) Low Low Low Yes
Prostate cancer None (or low)
to Moderate Low Moderate Moderate Yes
Stroke Moderate to
High
Low to
moderate Moderate Moderate Yes
QoL: Quality-of-life; CHD: Coronary heart disease; CKD: Chronic kidney disease; COPD: Chronic
obstructive pulmonary disease
14
Ranking of NCDs
Applying the mean criteria weights (Table 3) to the NCDs’ ratings (Table 4) resulted in a total
score for each of the 19 NCDs – in the range 0-100% – as shown in Figure 2. With a total
score of 77%, coronary heart disease (CHD) is the top-ranked NCD, followed by back and
neck pain (72%) and diabetes mellitus (68%), and so on for the other 16 NCDs.
Consistent with the presentational style and terminology used by the WHO for its priority list
of antibiotic-resistant bacteria (Tacconelli et al., 2018), the ranking of the 19 NCDs were
stratified into four tiers of priority: ‘very critical’, ‘critical’, ‘high’ and ‘medium’. The four
tiers are (total score ranges in parentheses): ‘Very critical’ priority (68-77%): coronary heart
disease, back and neck pain, diabetes mellitus; ‘Critical’ priority (54-59%): dementia and
Alzheimer’s disease, stroke; ‘High’ priority (35-44%): colon and rectum cancer, depressive
disorders, chronic obstructive pulmonary disease, chronic kidney disease, breast cancer,
prostate cancer, arthritis, lung cancer; and ‘Medium’ priority (12-29%): asthma, hearing loss,
melanoma skin cancer, addictive disorders, non-melanoma skin cancer, headaches.
Predicting participants’ preferences
The results of the cluster analysis (and chi-squared test and Cramér’s V) indicate that the
variation in participants’ preferences is generally unrelated to their socio-demographic and
background characteristics, suggesting that people’s preferences are largely idiosyncratic. The
results of the t-test indicate that there are no statistically significant differences between the
preferences of participants with higher and lower levels of education.
15
Figure 2: NCDs total scores
Values reported are percentages.
16
4. Discussion
In this study, 19 NCDs were prioritised based on five criteria, where their weights were
determined from a survey of NZ health sector stakeholders, and information about the NCDs’
performance on the criteria. Like the WHO’s priority list of antibiotic-resistant bacteria
(Tacconelli et al., 2018), for ease of communication (e.g. with researchers and policy-
makers), the priority list of NCDs was stratified into four tiers of priority. NCDs in the ‘very
critical’ tier – coronary heart disease, back and neck pain and diabetes mellitus – have high
rates of YLL or YLD and high health system costs. In contrast, NCDs in the (lowest)
‘medium’ tier – asthma, hearing loss, melanoma skin cancer, addictive disorders, non-
melanoma skin cancer and headaches – have the lowest burden. The intended use of such a
(tiered) priority list is to support research funding decision-making. Additional considerations
such as the cost of the research and its likelihood of success would also need to be included
when research projects are being assessed and, ultimately, funds are allocated in pursuit of
‘value for money’ (Tuffaha et al., 2019; Tuffaha et al., 2018).
The inclusion of back and neck pain in the ‘very critical’ tier is consistent with the findings
from global burden of disease (GBD) studies (Roth et al., 2018; Vos et al., 2017). For
example, Blakely et al (2019) point out that “in GBD 2016, New Zealand had 1.31 times
higher morbidity burden for back pain than expected based on its level of sociodemographic
development” (p. 17). As well as having high rates of YLD, back and neck pain is associated
with high health care costs – reflecting the correlation between YLD and health care costs, a
common finding in other studies (Blakely et al., 2019; Fun et al., 2019; Kinge et al., 2017;
Wieser et al., 2018).
In NZ, the Ministry of Health, the Ministry of Business, Innovation and Employment and the
Health Research Council (HRC) are closely involved with setting health research priorities at
the national level (HRC, 2019). The HRC, in conjunction with the Healthier Lives National
Science Challenge (a national research collaboration), recently embarked on setting health
research priorities for 2017-27 for cancer, cardiovascular diseases, diabetes and obesity. The
priority list created in the present study suggests that, in addition to these NCDs, other NCDs
– e.g. back and neck pain and dementia and Alzheimer’s (i.e. in the ‘very critical’ and
‘critical’ priority tiers) – should probably also be considered as priorities for research funding
in NZ.
Study limitations
This study has several limitations, mainly related to data deficiencies. First, anxiety disorders
and dental disorders – the former associated with relatively major health burden (IHME,
2019; Lee et al., 2017) and the latter with high health care expenditure (mostly not publicly-
funded) (MoH, 2017) – were excluded from the priority-setting framework due to a paucity of
available data. Second, the 19 NCDs included in the framework are largely heterogeneous –
e.g. stroke can range from being mild and transient to a major disabling event requiring
nursing home care for the rest of a person’s life – however, recognising such heterogeneity
and representing the NCDs with more granularity was impossible because the required
17
information was unavailable. Third, due to a lack of detailed data on NCDs’ health care costs,
the proportion proposed by Blakely et al. (2019) – indicating that 82% of NZ health care
expenditure is publicly-funded and the remaining 18% is privately-funded – was used in this
study to estimate the cost burden by the government and patients (and their families) (Blakely
et al., 2019). Fourth, there was very little evidence on NCDs-related health care expenditures
by the private sector and non-governmental organisations (e.g. Cancer Society); however,
given 82% of NZ’s total health care spending is publicly-funded, this may not be a serious
limitation (Blakely et al., 2014, 2015; Blakely et al., 2019). Overall, there is a need to
improve the quality (and availability) of data on NCDs. Further efforts are needed to conduct
CoI studies from the perspective of both health care providers and patients to generate data
for economic evaluation studies and to assist policy-makers in allocating resources.
Another limitation of the study is that the sample of participants used for the survey to
determine the criteria weights is not representative of the NZ population. However, the
variation in participants’ preferences (weights) was found to be related more to their personal
preferences than to their background characteristics; also, no significant differences between
the preferences of participants with high levels and low levels of education respectively were
found. Because participants were recruited using convenience and purposive sampling with
‘snowballing’, calculating a response rate is impossible; nonetheless, the number of
participants (n=517, with 476 usable responses) was larger than for other studies that
conducted online surveys to set research priorities across health interventions (O’Haire et al.,
2011; Street et al., 2014). The survey included in the present framework could be repeated
using a more representative sample.
5. Conclusion
To the best of the authors’ knowledge, the MCDA-based framework developed and piloted in
this study is the first attempt to create a priority list of NCDs to support health research
funding decision-making. The framework enables multiple criteria to be incorporated for
evaluating a wide range of NCDs, and for multiple stakeholders to be involved. The priority
list of NCDs created confirms that it is important to recognise their multi-dimensional nature
– e.g. mortality, morbidity and health care costs – when evaluating their relative priority. The
successful application of the framework in this study confirms that it is feasible and effective.
The framework could also be used to support priority-setting for health research funding for
other health conditions.
18
References
Adeyi, O., Smith, O., & Robles, S. (2008). Public policy and the challenge of chronic
noncommunicable diseases. International Journal of Epidemiology, 37(3), 686-687.
Allen, L. (2016). Non-communicable disease research. International Journal of Non-Communicable
Diseases, 1(3), 131-133.
Allen, L. (2017). Non-communicable disease funding. The Lancet, 5, 92.
Allen, L., Cobiac, L., & Townsend, N. (2017). Quantifying the global distribution of premature
mortality from non-communicable diseases. Journal of Public Health, 1, 1-6.
Aringer, M., Costenbader, C., Daikh, D., & et al. (2019). 2019 European League against
Rheumatism/American College of Rheumatology classification criteria for systemic lupus
erythematosus. Arthritis & Rheumatology, 71, 1400-1412.
Best, R. (2012). Disease politics and medical research funding: Three ways advocacy shapes policy.
American Sociological Review, 77(5), 780-803.
Bigna, J., & Noubiap, J. (2019). The rising burden of non-communicable diseases in Sub-Saharan
Africa. The Lancet Global Health, 7(10), 1295-1296.
Blakely, T., Atkinson, J., Kvizhinadze, G., & et al. (2014). Health system costs by sex, age and
proximity to death, and implications for estimation of future expenditure. The New Zealand
Medical Journal, 127(1393), 12-25.
Blakely, T., Atkinson, J., Kvizhinadze, G., & et al. (2015). Updated New Zealand health system cost
estimates from health events by sex, age and proximity to death: Further improvements in the age
of ‘big data’. The New Zealand Medical Journal, 128(1422), 13-23.
Blakely, T., Kvizhinadze, G., Atkinson, J., Dieleman, J., & Clarke, P. (2019). Health system costs for
individual and comorbid noncommunicable diseases: An analysis of publicly-funded health events
from New Zealand. PLOS Medicine, 16(1), e1002716.
Carroll, W. (2019). The global burden of neurological disorders. The Lancet, 18(5), 418-419.
Dieppe, P. (2013). Chronic musculoskeletal pain. British Medical Journal, 16(346), f3146.
Drummond, M., Sculpher, M., Claxton, G., & Torrance, G. (2015). Methods for the Economic
Evaluation of Health Care Programmes (4th ed): New York: Oxford University Press.
Etikan, I., Musa, S., & Alkassim, R. (2016). Comparison of convenience sampling and purposive
sampling. American Journal of Theoretical and Applied Statistic, 1-4.
Farin, G., Hoschek, J., & Kim, M. (2002). Handbook of Computer Aided Geometric Design: Elsevier
Science Ltd, North Holland.
Fun, W., Sararaks, S., Tan, E., & et al. (2019). Research funding impact and priority setting –
advancing universal access and quality healthcare research in Malaysia. BMC Health Services
Research, 19(248), 1-8.
Golan, O., & Hansen, P. (2012). Which health technologies should be funded? A prioritization
framework based explicitly on value for money. Israel Journal of Health Policy Research, 1, 44.
Golan, O., Hansen, P., Kaplan, G., & et al. (2011). Health technology prioritization: Which criteria for
prioritizing new technologies and what are their relative weights. Health Policy, 102(2-3), 126-
135.
Goodman, L. (1961). Snowball sampling. Annals of Mathematical Statistics, 32(1), 148-170.
Hansen, P., & Devlin, N. (2019). Multi-criteria decision analysis (MCDA) in healthcare decision-
making. The Oxford Encyclopedia of Health Economics. Oxford University Press.
19
Hansen, P., Hendry, A., Naden, R., Ombler, F., & Stewart, R. (2012). A new process for creating
points systems for prioritizing patients for elective health services. Clinical Governance: An
International Journal, 17, 200-209.
Hansen, P., & Ombler, F. (2008). A new method for scoring additive multi-attribute value models
using pairwise rankings of alternatives. Journal of Multi-Criteria Decision Analysis, 15(3-4), 87-
107.
HRC. (2019). The New Zealand health research prioritisation framework.
www.hrc.govt.nz/sites/default/files/2020-01/NZ%20Prioritisation-Framework-FA-web_0.pdf.
IHME. (2017). Findings from the Global Burden of Disease Study 2017.
www.healthierlives.co.nz/about/about-healthier-lives.
IHME. (2019). New Zeland burden of disease. https://vizhub.healthdata.org/gbd-compare/.
Kapiriri, L. (2018). Stakeholder involvement in health research priority setting in low income
countries: The case of Zambia. Research Involvement and Engagement, 4, 41.
Khan, M., Rahman-Shepherd, A., Painter, H., & Fletcher, H. (2019). How can we improve priority-
setting for investments in health research? A case study of tuberculosis? Health Research Policy
and Systems, 17(1), 68.
Kinge, J., Saelensminde, K., Dieleman, J., Vollset, S., & Norheim, O. (2017). Economic losses and
burden of disease by medical conditions in Norway. Health Policy, 121(6), 691-698.
Lee, C., Duck, I., & Sibley, C. (2017). Ethnic inequality in diagnosis with depression and anxiety
disorders. The New Zealand Medical Journal, 130(1454), 10-20.
Marsh, K., Goetghebeur, M., Thokala, P., & Baltussen, R. (2017). Multi-Criteria Decision Analysis to
Support Healthcare Decisions. Springer, New York.
Marsh, K., M, I., Thokala, P., R, B., Boysen, M., Kaló, Z., & et al. (2016). Multiple criteria decision
analysis for health care decision making - Emerging Good Practices: Report 2 of the ISPOR
MCDA Emerging Good Practices Task Force. Value in Health, 19(1), 125-137.
MoH. (2009). Report on New Zealand cost-of-illness studies on long-term conditions.
www.health.govt.nz/system/files/documents/publications/nz-cost-of-illness-jul09.pdf.
MoH. (2016). Health Loss in New Zealand 1990–2013: A report from the New Zealand burden of
diseases, injuries and risk factors study.
www.health.govt.nz/system/files/documents/publications/health-loss-in-new-zealand-1990-2013-
aug16.pdf.
MoH. (2017). Publicly funded dental care. www.health.govt.nz/your-health/services-and-
support/health-care-services/visiting-dentist/publicly-funded-dental-care.
Muka, T., Imo, D., Jaspers, L., & et al. (2015). The global impact of non-communicable diseases on
healthcare spending and national income: a systematic review. European Journal of
Epidemiology, 30(4), 251-277.
O’Haire, C., McPheeters, M., Nakamoto, E., & et al. (2011). Engaging stakeholders to identify and
prioritize future research needs. Methods for engaging stakeholders to identify and prioritize
future research needs. Methods Future Research Needs Report No. 4. (Prepared by the Oregon
Evidence-based Practice Center and the Vanderbilt Evidence-based Practice Center under
Contract No. 290-2007-10057-I.) AHRQ Publication No. 11-EHC044-EF. Rockville, MD:
Agency for Healthcare Research and Quality, 2011.
OECD. (2017). Health policy in New Zealand. www.oecd.org/els/health-systems/Health-Policy-in-
New-Zealand-March-2017.pdf.
20
Prynn, J., & Kuper, H. (2019). Perspectives on disability and non-communicable diseases in low- and
middle-income countries, with a focus on stroke and dementia. International Journal of
Environmental Research and Public Health, 16(18), 3488.
Roth, G., Abate, D., Abate, K., & et al. . (2018). Global, regional, and national age-sex-specific
mortality for 282 causes of death in 195 countries and territories, 1980-2017: A systematic
analysis for the global burden of disease study 2017. The Lancet, 392, 1736-1788.
Rottingen, J., Regmi, S., Eide, M., & et al. (2013). Mapping of available health research and
development data: What’s there, what’s missing, and what role is there for a global observatory?
The Lancet, 382, 1286-1307.
Shmueli, A., Golan, O., Paolucci, F., & Mentzakis, E. (2017). Efficiency and equity considerations in
the preferences of health policy-makers in Israel. Israel Journal of Health Policy Research, 6(18),
1-11.
Smith, N., Mitton, C., Peacock, S., Cornelissen, E., & MacLeod, S. (2009). Identifying research
priorities for health care priority-setting: A collaborative effort between managers and researchers.
BMC Health Services Research, 9, 165.
Street, J., Duszynski, K., Krawczyk, S., & et al. (2014). The use of citizens’ juries in health policy
decision-making: A systematic review. Social Science & Medicine, 109, 1-9.
Strong, K., Mathers, C., Epping-Jordan, J., & Beaglehole, R. (2006). Preventing chronic disease: A
priority for global health. International Journal of Epidemiology, 35(2), 492-494.
Sullivan, T., & Hansen, P. (2017). Determining criteria and weights for prioritizing health
technologies based on the preferences of the general population: A New Zealand pilot study.
Value in Health, 20(4), 679-686.
Sullivan, T., Hansen, P., Ombler, F., Derrett, S., & Devlin, N. (2020). A new tool for creating personal
and social EQ-5D-5L value sets, including valuing ‘dead’. Social Science & Medicine, 246,
112707.
Tacconelli, E., Carrara, E., Savoldi, A., & et al. . (2018). Discovery, research, and development of new
antibiotics: The WHO priority list of antibiotic-resistant bacteria and tuberculosis. The Lancet
Infectious Diseases, 18(3), 318-327.
The Reserve Bank of New Zealand. (2017). The Reserve Bank of New Zealand inflation calculator.
www.rbnz.govt.nz/monetary-policy/inflation-calculator.
Thokala, P., Devlin, N., Marsh, K., & et al. (2016). Multiple criteria decision analysis for health care
decision making – An introduction: Report 1 of the ISPOR MCDA Emerging Good Practices Task
Force. Value in Health, 19(1), 1-13.
Tripathy, J. (2018). Research priorities in non-communicable diseases in developing countries: Time
to go beyond prevalence studies. Public Health Action, 8(2), 98-99.
Tromp, N., & Baltussen, R. (2012). Mapping of multiple criteria for priority setting of health
interventions: An aid for decision makers. BMC Health Services Research, 12, 454.
Tuffaha, H., Aitken, J., Chambers, S., & Scuffham, P. (2019). A framework to prioritize health
research proposals for funding: Integrating value for money. Applied Health Economics and
Health Policy, 17(6), 761-770.
Tuffaha, H., El Saifi, N., Chambers, S., & Scuffham, P. (2018). Directing research funding to the right
research projects: A review of criteria used by research organizations in Australia in prioritising
health research projects for funding. BMJ Open, 8(12), e026207.
Viergever, R. (2013). The mismatch between the health research and development (R&D) that is
needed and the R&D that is undertaken: An overview of the problem, the causes, and solutions.
Global Health Action, 6, 22450.
21
Viergever, R., Terry, R., & MP, M. (2010). Health research prioritization at WHO: An overview of
methodology and high level analysis of WHO led health research priority setting exercises.
Geneva: World Health Organization, www.who.int/rpc/publications/en/.
Vos, T., Abajobir, A., Abate, K., & et al. (2017). Global, regional, and national incidence, prevalence,
and years lived with disability for 328 diseases and injuries for 195 countries, 1990–2016: A
systematic analysis for the global burden of disease study 2016. The Lancet, 390(10100), 1211-
1259.
WHO. (2019). Noncommunicable diseases. www.who.int/gho/ncd/en/.
Wieser, S., Riguzzi, M., Pletscher, M., Huber, C., Telser, H., & Schwenkglenks, M. (2018). How
much does the treatment of each major disease cost? A decomposition of Swiss National Health
Accounts. European Journal of Health Economics, 19(8), 1149-1161.