PROMIS®

Patient Reported Outcomes

Measurement Information System:

The Case for an International PROMIS initiative

Jordi Alonso, MD, PhD, on behalf of PROMIS International

Director, Program Epidemiology & Public HealthIMIM-Hospital del Mar Medical Research Institute

Pompeu Fabra UniversityBarcelona, Spain

King’s Fund. PROMS Research Conference

November18, 2014. London.

Content of presentation

• PROMIS

• Measures, Tools & Metric

• Assessment Centre

• PROMIS International

• Translation & Requirements for Release

• Any questions…

Click to edit Master title style

Vision-Mission:

• Vision– The Patient-Reported Outcomes Measurement

Information System (PROMIS®), funded by the National Institutes of Health, aims to provide clinicians and researchers access to efficient, precise, valid, and responsive adult- and child-reported measures of health.

• Mission

– PROMIS uses measurement science to create an efficient state-of-the-art assessment system for self-reported health.

Click to edit Master title style

DOMAINS

A domain is the specific feeling, function, or perception you want to measure.

Cuts across different diseases

PROMIS Measures

• Adult Health Measures• More than 1,000 individual items (questions)

• 51 distinct item banks or scales

• 12-124 items per bank

• Pediatric Health Measures• More than 150 items (questions)

• 18 distinct banks or scales

• 22 more on the way

PROMIS Basic Tools

Derived from Item Banks

• Computerized Adaptive Testing (CAT) Dynamic testing averaging 4-5 items per domain

• Fixed Length (“Short”) Forms By individual domain (4-10 items)

7-domain health profiles (-29, -43, -57)

Global Health Index

• Global-10

Click to edit Master title style

ITEM BANKS

An item bank is a large collection of items measuring a single domain.

Therefore…..Any and all items can be used to provide a score for that domain.

Fatigue Item Bank

Chemotherapy trial

Same metric, same meaning

Osteoarthritis trial

Diabetes trial

Heart Failure trial

Epilepsy trial

Items 1-10 CAT Items 6-12

Items 1-5Items 2, 4, 9, 13

The PROMIS Metric

• T Score Mean = 50

SD = 10

• Referenced to the US General Population

Click to edit Master title style

CAT demo results

5035 40 45 55 60 65

PROMIS Fatigue Across Five Clinical Conditions

Average for General Population

COPD Stable (B) COPD Exacerbation (B)

HF Pre-transplantHF Post-transplant

Exacerbation to Stable

Depression

(B)

Depression

(1 mo)

Depression

(3 mos)

Cancer

Chemo

(B)

Cancer

w/ benefit

(2 mos)

Back Pain

(B)

Back Pain

(1 mo)

Back Pain

(3 mos)

N = 64

N = 310

N = 114

N = 229

N = 125

Click to edit Master title style

What is Assessment Center?

• Web-based research management tool

– Primary mechanism for distribution of PROMIS instruments

– Display of item- and instrument-level statistics

• Allows creation of study-specific URLs for data collection

– Enabled to administer CATs

• Administration of PROMIS and other instruments

• FREE!!

Click to edit Master title style

Select CATs

PROMIS: Potential for International Community

• Solid, comprehensive, new generation PRO measurement initiative

• Oriented to clinical research and practice

• International research (RCTs, Cumulative Data, Meta-analyses, CER)

• International comparisons

15

PROMIS International (non-US)

16

1st Name Surname Country

Susan Bartlett Canada

Alain Leplège France

Matthias Rose Germany

Sandra Nolte Germany

Ulrike Ravens-Sieberer Germany

Aiping Lu Hong Kong

Fabio Efficace Italy

Agnieszka Żukowska Poland

Wojciech Glinkowski Poland

Gemma Vilagut Spain

Jordi Alonso Spain

John Chaplin Sweden

Caroline Terwee The Netherlands

Hein Raat The Netherlands

Neil Aaronson The Netherlands

Chema Valderas UK

John Brazier UK

PROMIS International: Steps

17

• Translation

• DIF assessment

• Calibration

• Country-specific reference scores

• Further validation

• Other issues

PROMIS Translation & Cultural Adaptation

• Harmonization & Universal Approach

Harmonization across languages –

o Not an exact replica, unavoidable variation, harmonization determines acceptable variation.

Universal approach to translation –

o Goal: one language version for multiple countries

(vs country-specific versions)

18

PRISM - PROMIS International Standards and Measures

Objective is to increase the availability of:

o full item banks (9) to a total of 12 languages, and of

o selected short forms (14) to 26 languages

19

Minimum Requirements for Release

20

Thank you!

jalonso@imim.es

PROMIS: http://www.nihpromis.org/

Asmnt. Center: https://www.assessmentcenter.net/

21

Recommendations for further Evaluation

22

Instrument Recommendations for further psychometric evaluation AFTER release

Sample size

Item bank Calibration of item banks in relevant patient groups and the general population;

Minimum 500 Optimal 1000-2000

Short form / item bank The collection of language- or country-specific reference scores in the general population;

At least 300-400

Short form / item bank An evaluation of the relevance and comprehensiveness of the items (content validity) of the item bank and cultural adaptation;

4-6 focus groups or 12 interviews

Short form / CAT Further psychometric evaluation (construct validity, internal consistency, test-retest reliability, measurement error, DIF among different patient groups) of PROMIS instruments in specific patient populations;

200 subjects per group for DIF 50-100 for other measurement properties

Short form / CAT Responsiveness and Minimal Important Change of PROMIS instruments in relevant patient populations.

50-100

PROMIS Translation & Cultural Adaptation

Universal approach to translation –

o Goal: one language version for multiple countries (vs country-specific versions)

Strategies:

1. Translators from various countries or dialectscontribute to the translation process;

2. Avoiding colloquial and idiomatic expressions;

3. Pretesting and debriefing items with samples from relevant countries.

23

PROMIS Instrument Maturity Model

Stages

1.Developmental• Conceptualization & Item Pool Development

2.Developmental• Calibration Phase – DIF impact

3.Public Release • Calibrated & Preliminary Validation Completed

4.Maturing• Responsiveness & Expansion

5.Fully Mature • Interpretation & User Support

PROMIS Psychometrics & Calibration

1. Traditional Descriptive Statistics• Items analysis, Scale analysis

2. Evaluation of IRT Assumptions• Unidimensionality, Local independence &

Monotonicity3. Evaluation of IRT Model Fit

• IRT parameters, Model fit, Item & Scale properties4. Evaluation of DIF

• Hypotheses, IRT-based & non-IRT-based models5. Item Calibration for Item Banks

• Full/Incomplete blocks, Standardization, CAT algorithms

PROMIS Item Development

1. Identification of Extant Items

2. Binning & Winnowing

3. Item Revision Process

4. Response Options

5. Recall Time Frame

6. Focus Groups

7. Cognitive interviews

8. Final Item Reviews

PROMIS Int’l: The Netherlands

Website: www.dutchflemihspromis.nl

27

PROMIS Int’l: Germany

28

Website: http://promis-germany.de/

PROMIS Int’l: Spain

o Translation: not needed. Cognitive debriefings of

Pain SFs. PI: Dr. Rafael Gálvez

o 3 Mental Health Item Banks:

o Cognitive Debriefings

o Clinical Validation

o Calibration: Gen population (internet panel= 2,000)

PI: Gemma Vilagut. Funded: ISIII, M. Economy & Competitivity

29

PROMIS Int’l: Canada• National stakeholder meeting: needs, priorities, infrastructure

to support and sustain Canada PRO Initiative and create a Canada PRO website

30

www.mcgill.ca/can-pro-network

• to include PROMIS-Canada CAT platform and French versions

• Publish Canada PRO Initiative workshop reports in a special edition of the Journal of Clinical Epidemiology