pcori methodology standards: academic curriculum · “americans are very measurement oriented …...

PCORI Methodology Standards:

Academic Curriculum

© 2016 Patient-Centered Outcomes Research Institute. All Rights Reserved.

Prepared by Albert Wu, MD, MPH

Clifton Bingham, MD

Presented by Albert Wu, MD, MPH

Module 6c: Patient-Reported Outcomes:

Measurement and Measure Development

Category 2: Patient-Centeredness

Measurement

3

“Americans are very measurement oriented … we tend to emphasize what is

measured.”

—John Kenneth Galbraith

Measurement Matters

4

Measurement

The process of applying a standard

scale to something that varies in

the dimension of interest

5 Image: Goele. (2007). Via Wikimedia Commons: https://commons.wikimedia.org/wiki/File:Folding_Rules_2.jpg. Public domain. Accessed October 13, 2015.

How Do You Measure Health-Related Quality of Life?

Health-related quality of life is an

abstract concept (not directly

observable)

6 Image: Kr-val. (2008). Via Wikimedia Commons: https://commons.wikimedia.org/wiki/File:Altocumulus_se_2.jpg. Public domain. Accessed October 13, 2015.

There is no standard scale, so we need to specify what we want to measure

Assemble several indicators that approximate the concept

Create scale scores by combining responses to questions

Measuring Health-Related Quality of Life

7

Assumes concept being measured is unidimensional (e.g., depression)

Comprises questions/items to estimate the concept

Method to elicit responses = response scale

Summated Rating Scale

8

Goal is to measure a person’s self-reported mental health

Five questions rate extent of anxiety, depression, calm, and happiness

Responses are on a six-point “Likert-type” ordinal, adjective rating response scale

Example: Mental Health Inventory Five-Item Scale (MHI-5)

9

9b. Have you been a very nervous person?

9c. Have you felt so down in the dumps that nothing could cheer you up?

9d. Have you felt calm and peaceful?

9f. Have you felt downhearted and blue?

9h. Have you been a happy person?

Scoring Example: MHI-5 Questions

10

Check one answer:

1. All of the time

2. Most of the time

3. A good bit of the time

4. Some of the time

5. A little of the time

6. None of the time

Scoring Example: MHI-5 Responses

11

Write in Your Responses to These Items

12

Points (1 – 6) Recoded score

Nervous

Down in the dumps

Calm and peaceful

Downhearted and blue

Happy

Sum after reversing Items (5 – 30)

Sum of item scores

Recode and reverse

Linear conversion to 0-100 scale

Scaling and Scoring

13

Item Scoring

Items 9b, 9f, and 9c

Use precoded values

Items 9d and 9h

Require recoding before computation of the scale

score

14

All of the time 1 = 6

Most of the time 2 = 5

A good bit of the time 3 = 4

Some of the time 4 = 3

A little of the time 5 = 2

None of the time 6 = 1

Recode and reverse items

Compute sum of recoded items scores

Five items

Six response categories:

• Lowest possible score 5

• Highest 30

• Range 25

Transformation of raw summated score:

[(Raw scale score – lowest possible score) / possible score range] 100

For example, mental health score of 21

(21 – 5)/25] 100 = 64

Scale Scoring

15

64 on a 0-100 scale

Mean for the U.S. population is approximately 75, with standard deviation of

approximately 20

Your Score for MHI-5 Mental Health Scale

16

PROs are measured using indicators (questions), with responses summed to

approximate the construct being measured

Unidimensional constructs are commonly estimated using summated rating scales

Scores on PRO scales are expressed as raw scores and transformed linearly into 0-100

scales

Summary

17

Measure Development

18

1. Define what it is you are trying to measure

2. Identify relevant domains

3. Identify relevant items

4. Select response scale

5. Pilot-test

Steps in Developing a PRO Measure

19

A model of the main concepts of a domain and their relationships

Conceptual Model

20

Clinical observation

Theory

Research findings

Preliminary studies

Expert opinion

Identifying Relevant Domains

21

Clinical observation

Theory

Research findings

Expert opinion

Existing questions

Patient engagement

Devising Indicators/Items

22

How does your health condition affect your ability to function and how you feel?

Engaging Patients for Item Development

23

Exclude items that are …

Redundant

Awkward

Too idiomatic

Difficult to translate

Item Reduction

24

1. Specify what you are trying to measure

2. Identify relevant domains, using various sources including engaging patients

3. Identify items that tap those domains (item reduction)

4. Select a response scale

5. Pilot-test

Summary: Steps in Developing a PROM

25