Measuring Health OutcomesPatient-reported Outcomes (PRO)

Jeffrey Johnson, PhD

Professor

School of Public Health

University of Alberta

IHE Luncheon Series

November 21, 2016

• Board of Directors, EuroQol Research Foundation (2002-present)

• Research funding from Canadian Institutes for Health Research, Canadian Diabetes Association, Alberta Innovates Health Solutions, Alberta Health Services, Health Quality Council of Alberta and Alberta Health.

Disclosures

What is a PROM?

• A Patient Reported Outcome Measure (PROM) is a questionnaire that asks patients to self report about aspects of their own health.

• PROMs focus on health status and health-related quality of life.

Health:

“A state of complete physical, mental, and social well-being,

and not just merely the absence of disease or infirmity”

- WHO, 1948

Health & Health Status

“… the state of optimum capacity for the effective performance of

valued tasks ...” - Talcott Parsons (sociologist)

“… physical and mental variables ‘end at the skin’… social

functioning extends the concept ...”

- John Ware (social psychologist)

Health Status / HRQL

Ferrans et al., 2005

Wilson & Cleary 1995

Rationale:

Improved health/functioning is goal of health care system;

Patient-centered care;

Key element of Quality;

Self-care is part of health care.

Why PROMs?

Pursuing the ‘Triple Aim’

Health Outcomes

Patient Experience

Cost ofCare

How can PROMs be used?

• Assessing/monitoring population health status;

• Enhance patient-provider communication and inform clinical practice;

• Evaluation of the quality/performance of health systems.

• In economic evaluations to inform decision making and resource allocation.

Health-Related Quality of Life (PROMs)

Patient Experience/Satisfaction (PREMs)

Self-reported behaviours Behavioural Risk Factors (smoking, PA, diet, alcohol)

Self-care (self-efficacy, health literacy)

Patient-Reported Outcome Measures

HRQL Measures

Generic versus Specific Generic: Applicable to any or all populations

Specific: condition, population, symptom

Profile vs Index Profile: a range (profile) of scores (e.g., SF36/SF12)

Index: a single overall score (e.g., EQ-5D)

HRQL Index Scores

Ratings versus Preference-based

• Ratings are a non-comparative assessment (judgment) of a health state

- e.g., visual analog scale

• Preference-based index scores reflect the relative desirability of a health state (i.e., choice-based)

- direct: e.g., standard gamble, time-trade off

- indirect: e.g., HUI3, SF-6D, EQ-5D

HRQL Index Scores

Indirect measures (e.g., EQ5D) include:

1. Descriptive system for health states

- i.e., a multi-dimensional questionnaire

- profile-like health status measure

1. Preference-based value set- i.e., a scoring system- generates a single overall index score

HRQL Index Scores

Indirect measures (e.g., EQ5D):

1. Multidimensional health status

2. Index score integrates Morbidity & Mortality

3. Incorporate into QALY (quantity & quality) of life

The EQ-5D

The EQ-5D is a generic preference-based patient-reported outcome measure of health related quality of life.

It can be used to generate utility values for use in economic evaluation.

It is the most commonly used preference based measure around the world.

EQ-5D™ is a trademark of the EuroQol Research Foundation.

10

0

20

30

40

50

60

80

70

90

100

5

15

25

35

45

55

75

65

85

95

The best health you can imagine

The worst health you can imagine

Visual Analogue ScaleDescriptive System

EQ-5D-3L

Mobility

Self-Care

UsualActivities

Pain/Discomfort

AnxietyDepression

I have no problems washing or dressing myself I have some problems washing or dressing myselfI am unable to wash or dress myself

I have no problems doing my usual activities I have some problems doing my usual activitiesI am unable to do my usual activities

I have no pain or discomfortI have moderate pain or discomfortI have extreme pain or discomfort

I am not anxious or depressedI am moderately anxious or depressedI am extremely anxious or depressed

I have no problems in walking aboutI have some problems in walking aboutI am confined to bed

☐☐☐

☐☐

☐

☐☐☐

☐☐☐

☐☐☐

EQ-5D-Y

I have no problems walking about I have some problems walking about I have a lot of problems walking about

I have no problems washing or dressing myselfI have some problems washing or dressing myselfI have a lot of problems washing or dressing myself

I have no problems doing my usual activitiesI have some problems doing my usual activitiesI have a lot of problems doing my usual activities

I have no pain or discomfortI have some pain or discomfortI have a lot pain or discomfort

I am not worried, sad or unhappyI am a bit worried, sad or unhappyI am very worried, sad or unhappy

Mobility (walking about)

Looking after myself

Doing Usual Activities (for example, going to school, hobbies, sports, playing, doing things with family or friends)

Having pain or discomfort

Feeling worried, sad or unhappy

☐☐☐

☐☐☐

☐

☐

☐

☐☐☐

☐

☐

☐

243 health states

EQ-5D-5L

I have no problems in walking aboutI have slight problems in walking about I have moderate problems in walking about I have severe problems in walking about I am unable to walk about

I have no problems washing or dressing myselfI have slight problems washing or dressing myselfI have moderate problems washing or dressing myselfI have severe problems washing or dressing myselfI am unable to wash or dress myself

I have no problems doing my usual activities I have slight problems doing my usual activities I have moderate problems doing my usual activities I have severe problems doing my usual activities I am unable to do my usual activities

I have no pain or discomfortI have slight pain or discomfortI have moderate pain or discomfortI have severe pain or discomfortI have extreme pain or discomfort

I am not anxious or depressedI am slightly anxious or depressedI am moderately anxious or depressedI am severely anxious or depressedI am extremely anxious or depressed

☐☐☐☐☐

☐☐☐☐☐

☐☐☐☐☐

☐☐☐☐☐

☐☐☐☐☐

3125 health states

EQ-5D Descriptive System

10

0

20

30

40

50

60

80

70

90

100

5

15

25

35

45

55

75

65

85

95

The best health you can imagine

The worst health you can imagine

EQ-Visual Analogue Scale (VAS)

Your Health today =

We would like to know how good or bad your health is TODAY.

This scale is numbered from 0 to 100.

100 means the best health you can imagine.0 means the worst health you can imagine.

Mark an X on the scale to indicate how your health is TODAY.

Now, please write the number you marked on the scale in the box below.

EQ-5D: Data Elements

1. EQ-5D Health Profile

1. EQ-VAS score

2. EQ-5D Index score

1. EQ-5D Health Profile

• Presenting the frequency or the proportion of reported problems for each level for each dimension.

• Overall, or by subgroup, such as age, sex, presence of disease, before vs. after treatment, treatment vs. comparator, etc.

• Dichotomize into: – Level 1: No problems

– Levels 2, 3, 4, 5: Some problems

2014 Alberta Population Norms for

EQ-5D-5L

Counts of respondents reporting no problems versus problems by EQ-5D-5L dimension and age group

10

0

20

30

40

50

60

80

70

90

100

5

15

25

35

45

55

75

65

85

95

The best health you can imagine

The worst health you can imagine

2. EQ-VAS score

Your Health today =

We would like to know how good or bad your health is TODAY.

This scale is numbered from 0 to 100.100 means the best health you can imagine.0 means the worst health you can imagine.

Mark an X on the scale to indicate how your health is TODAY.Now, please write the number you marked on the scale in the box below.

✗

95

Mobility

Self-Care

UsualActivities

Pain/Discomfort

AnxietyDepression

I have no problems in walkingI have slight problems in walkingI have moderate problems in walkingI have severe problems in walkingI am unable to walk

I have no problems washing or dressing myselfI have slight problems washing or dressing myselfI have moderate problems washing or dressing myselfI have severe problems washing or dressing myselfI am unable to wash or dress myself

I have no problems doing my usual activities I have slight problems doing my usual activities I have moderate problems doing my usual activities I have severe problems doing my usual activities I am unable to do my usual activities

I have no pain or discomfortI have slight pain or discomfortI have moderate pain or discomfortI have severe pain or discomfortI have extreme pain or discomfort

I am not anxious or depressedI am slightly anxious or depressedI am moderately anxious or depressedI am severely anxious or depressedI am extremely anxious or depressed

☐☐☐☐☐

☐☐☐☐☐

☐☐☐☐☐

☐☐☐☐☐

☐☐☐☐☐

✓

✓

✓

✓

✓

Levels of perceived problems are coded as follows1 ☐2 ☐3 ☐4 ☐5 ☐

1 ☐2 ☐3 ☐4 ☐5 ☐

1 ☐2 ☐3 ☐4 ☐5 ☐

1 ☐2 ☐3 ☐4 ☐5 ☐

1 ☐2 ☐3 ☐4 ☐5 ☐

✓

✓

✓

✓

✓

Level = 2

Level = 1

Level = 1

Level = 3

Level = 4

Health state 21134

Scoring (Canadian Algorithm)Health state 21134

EQ-5D-5L index score = 1.1351 -0.0389*MO-0.0458*SC-0.0195*UA-0.0444*PD-0.0376*AD-0.0510*(MO45)-0.0584*(SC45)-0.1103*(UA45)-0.1409*(PD45)-0.1277*(AD45) +0.0085*Num45sq

EQ-5D-5L index score = 1.1351 -0.0389*2-0.0458*1-0.0195*1 -0.0444*3-0.0376*4-0.0510*0-0.0584*0-0.1103*0 -0.1409*0-0.1277*1 +0.0085*02

= 0.5807

3. EQ-5D Index Score (5L version)

Xie F1, et al. A Time Trade-off-derived Value Set of the EQ-5D-5L for Canada. Med Care. 2016 Jan;54(1):98-105.

EQ-5D 5L index score range: -0.148 (55555) to 0.949 (11111)

Applications of EQ-5D

• In evaluating effectiveness of services– Effectiveness of services and interventions/

programs/ treatments– Performance of health service providers

• In assessing population health – Describing and monitoring the health status

of the population

• In informing clinical practice – Screening – Management and monitoring of patient

outcomes

• In healthcare resource allocation decisions– Program evaluation– Cost-effectiveness analysis

Which EQ-5D to use?

How and when to administer them?

Utilize EQ-5D data

EQ-5D in evaluating effectiveness of services

Analysis of EQ-5D scores from two phase 3

clinical trials of romiplostim in the treatment

of immune thrombocytopenia (ITP) (Sanz et

al, Value in Health, 2011)

Data from two international, randomized, placebo-controlled, double-blind, 25-week trials that evaluated the efficacy and safety of romiplostim in adult patients with chronic idiopathic thrombocytopenia (ITP)

N = 125

EQ-5D-3L

Conclusion: Using romiplostim in cases of chronic ITP was associated with improvement in health-related quality of life

http://hqca.ca/studies-and-reviews/health-outcomes-measurement/

2013 Alberta Population Norms for

EQ-5D-3L

Mean EQ-5D index score for Alberta population sample by age/sex

EQ-5D in population health

http://www.ahw.gov.ab.ca/IHDA_Retrieval/

Percentage of respondents reporting problems with Anxiety/Depression, by Sex (ACHS, 2014) [N=7559]

IHDA

0

10

20

30

40

50

60

70

Female Male Both

No Problems Some Problems

EQ-5D in population health

EQ-5D/PROMs in clinical practice

Use of PROMs at the Alberta Bone and Joint Health Institute (ABJHI)

Source: Dr. Deborah Marshall

To describe

To inform

WOMACScore@3Months Coeffic

i

entSE P [95%Conf.Interval]

Obese(YvsN) -0.16 0.89 0.860 -1.90 1.59Pre-SurgWOMAC 0.21 0.02 <0.0010.17 0.25Age 0.01 0.04 0.824 -0.08 0.10Malevs.Female 1.47 0.87 0.092 -0.24 3.17Kneevs.Hip -6.97 0.89 <0.001-8.70 -5.231Pre-SurgRiskFactorvs.0-0.70 0.99 0.483 -2.65 1.252+Pre-SurgRiskFactorsvs.0 -1.47 1.10 0.182 -3.62 0.69

To PredictPatient Decision

Aids

Using large scale routine collection of the EQ-5D to enhance patient decision making:A proof of concept study (Bansback, et al. )

PROMs - what are the rationales?

For patients: - information and choice;

For clinicians:- improved decision making;

For hospitals: - managing clinical quality;

For commissioners: - quality, value and equity;

Appleby, Devlin & Parkin, 2016

PROMS in UK NHS elective surgeries

Available at: http://www.hscic.gov.uk/proms

PROMS in Clinical Care

– a theoretical framework

Source: Santana & Feeny, QOLR 2014

Source: Velikova et al., J Clin Oncol 2004;22:714 Source: Velikova et al., Eur J Can 2010;46:2381

PROMs in Oncology

- 28 controlled trials- In most trials, effect of providing PRO feedback was limited

- “some grounds for optimism in the possible impact of measurement of PRO in clinical practice (specifically in improving diagnosis and recognition of problems and patient–physician communication), considerable work is still required before clinicians can invest resources in the process and rely on consistent evidence for the benefits for their patients.”

• Other factors include enabling technology, appropriate infrastructure, integration into clinical workflow, enhanced interpretability of PROMs data & engaging professionals in the implementation planning.

• Focused on controlled trials with PRO as the outcome;• 16 studies; only 6 showed benefit, only 1 of which was stat. sig.;• Quality of trials was poor; • Impact depended on purpose – management of actionable

symptoms or problems in specific patient populations;

• 16 studies; • Barriers and Facilitators in 4 themes:

1. collecting and incorporating the data (practical);2. valuing the data (attitudinal);3. making sense of the data (methodological);4. using info to make changes to patient care (impact);

• Limitations and barriers of PROMs may represent a failing to engage professionals in the planning stages;

• Not acknowledging the conflict between managerial and professional objectives

QALYs & Economic Evaluations

Length of Life (years)

Quality of life

0Dead

1Perfect Health

6

6 x 1= 6 QALYs

10 x 0.5= 5 QALYs

10

0.5

• A quality-adjusted life-year (QALY) is a measure that takes into account both quantity and quality of life.

• QALY = (length of life “years”)*(quality weight)

QALYs are used in cost-utility analysis to calculate an “incremental cost utility ratio” (ICUR): Ratio of incremental costs over incremental QALYs:

cost (A) – cost (B)

QALY (A) – QALY (B)

QALYs provide a common currency to assess the extent of the benefits gained from a variety of interventions in terms of health related quality of life and survival for the patient.

ICUR=

EQ-5D in economic evaluationCost-Effectiveness of Physician Notification and Follow-up or Collaborative Care for Patients with Diabetes Who Screen Positive for Depression in Primary Care: Results from a Controlled Trial (Johnson et al. American Journal of Preventative Medicine 2015)

To evaluate the cost-effectiveness of three strategies to improve depressive symptoms in patients with T2D (N=227)

Interventions: Usual care vs. screening for depression & physician notification vs. comprehensive care model.

HRQL measure: EQ-5D-5L. Measurement: 0, 6, 12 mon

Conclusion: Physician notification and follow-up is a clinically effective strategy compared with usual care, but investing more up-front resources in collaborative care yielded the most cost-effective strategy.

Cost-Effectiveness Analysis Based on Quality-Adjusted Life-Years (Cost-Utility analysis)

Arm Average Cost($)

Incremental Cost($)

Average Effect

Incremental

Effect (QALY)

ICUR

($)

Usual Care 5889 [13283] - 0.685 [0.179] - -

Enhanced Usual Care 6339 [11841] 450 [-3814, 4727] 0.691 [0.188] 0.006 [-0.067, 0.069] 76,271

Collaborative Care (Compared with Usual Care)

6910 [10720] 1021 [-2750, 4775] 0.727 [0.165] 0.042 [-0.011, 0.096] 24,368

Collaborative Care (Compared with Enhanced

Care)

6910 [10720] 571 [-3129, 4241] 0.727 [0.165] 0.036 [-0.023, 0.095] 15,861

Contact us

Located at: 2-040 Li Ka Shing Centre for Health Research Innovation, University of Alberta

Email: jeff.johnson@ualberta.ca

apersu@ualberta.ca

Phone: 780-248-1010

Website: www.apersu.ca

If you have been,

thanks for listening.

Questions?

Who’s using the EQ-5D in Alberta?

– Health Quality Council of Alberta: Biannual population survey (N=5000)

– Alberta Health – 2014-15 Alberta Community Health Survey (N=7559)

– Alberta Retired Teachers Association: Annual Health & Wellness Survey (N=14,000)

– Alberta Health Services• Alberta Tomorrow Project: cancer etiologic cohort study (N=50,000)• Workforce Model Transformation: Program evaluation (N=6,500) • APPROACH (cardiac revascularization) – EQ-5D in 12-month post-

procedure follow-up• Northern Alberta Renal Program (NARP) – EQ-5D and ESAS• Cancer Control Alberta: under development – EQ-5D and ESAS

– Alberta Bone and Joint Health Institute: Routine outcome measurement QA/QI for hip & knee arthroplasty (N=20,000) • EQ-5D (3L5L), WOMAC

– Academia: • University of Alberta & Calgary: Various Research Studies

Primary Care Networks and EQ-5D

– PCNs in Alberta are funded under a joint agreement between AH and AMA

– Alberta Health set out a Primary Health Care Strategyincluding system level indicators for performance of primary care delivery through PCN

– Draft of Schedule B indicators • Percent of patients with a chronic conditiona who report maintaining or

improving quality of life as measured by the EQ-5D Health Questionnaire during the Fiscal YearaChronic diseases for this indicator include: Hypertension, Diabetes, COPD, CAD, CHF

– In current FY, expected reporting is just on use of the EQ-5D as indicator;

– SF-12 will also be considered as an alternative indicator;