FDA Guidance on PROsBreakfast Seminar 19th March 2009

Diane Wild, MScOxford Outcomes Ltd., Oxford, UK

Overview

• What are PROs?

• Why and when to use them

• The FDA draft guidance

• After the guidance was released

• ePRO migration – levels of evidence

PROs – FDA Definition

• A PRO is a measurement of any aspect of a patient’s health status that comes directly from the patient (i.e., without the interpretation of the patient’s responses by a physician or anyone else) (FDA Draft Guidance, 2006).

The PRO umbrella

• Conceptual confusion between QoL, HRQoL, functional status etc.

• The term patient reported outcomes was agreed by a group of interested parties (ISPOR, FDA etc) in September 2000.

• PRO describes the source rather than the content.

Why use PROs

• Some treatment effects only known to the patient (e.g. pain, fatigue)

• Patients provide a unique perspective on effectiveness (e.g. spirometry)

• Formal assessment more reliable than informal interview

For what purpose?

• Definition of entry criteria

• Evaluation of efficacy

• Evaluation of adverse events

Timing and frequency

• As events occur

• At regular intervals

• Baseline and end of treatment

Taxonomy of PROs

• Concepts:– Functional status (physical, psychological, social).– Symptoms/signs (frequency, severity,

bothersomeness)– Health perceptions/perceived health status.– Satisfaction/preference for treatment– Adherence – QoL– HRQoL

QoL/HRQoL?

• FDA draft guidance – extremely complex.

• QoL is distinct from HRQoL as it includes perceptions of immediate environment, housing conditions etc – Have you enough money to meet your needs?– How safe do you feel in your daily life?

Hickey, A. M et al. BMJ 1996;313:29-33

Fig 1--The segments represent five areas of life nominated by the individual; the size of the segment can be adjusted to show the relative importance of each area

for the individual's quality of life

Generic & Condition Specific

Generic Measures• Designed for general use/broad base • Comparable across disease states

Disease Specific Measures• Designed for use within a particular disease • More sensitive to smaller clinically significant changes• Maybe more sensitive to specific impact of clinical

variables• Relatively narrow focus may prevent sensitivity to

treatment side-effects.

0

10

20

30

40

50

60

70

80

90

100

PF RP BP GH VT SF RE MH

Age & Gender Equivalent Norms(Ware et al, 1993, 94)

Chronic Obstructive PulmonaryDisease (Jones & Bosh, 1997)

HRQL of Patients with COPD vs. General Population Norms (SF-36)

High scores = good HRQL

FDA Draft Guidance

Draft guidance – rationale for development

• FDAs thinking on how sponsors can develop and use study results measured by PROs to support labelling claims.

• To increase efficiency of discussions, streamline FDAs review and provide optimal information about the patients view of treatment benefit.

Draft guidance contents (1)

• Selection of PROs

• Development of PROs

• Modification of PROs

• Validation of PROs

Assess Measurement PropertiesAssess score reliability, validity and ability to detect change.

Evaluate administrative & respondent burden. Add, delete or revise items. Identify meaningful differences in scores. Finalise instrument formats,

scoring, procedures & training materials.From FDA Draft PRO Guidance document 2006

Identify Concepts & Develop Conceptual Framework

Identify concepts & domains that are important to patients. Determine intended population and research application.

Hypothesize expected relationships among concepts.

Create InstrumentGenerate items.

Choose administration method, recall period &

response scales.Draft instructions.

Format instrument. Draft procedures for

scoring & administration. Pilot test draft instrument.

Refine instrument & procedures.

Modify InstrumentChange concepts measured, populations studied, research application, response options, recall periodor method of administration.

PRO

Modifications – The FDA’s View

1. Revising the PRO instrument’s measurement concept,

2. Applying an instrument to a new population or condition,

3. Changing the PRO’s item content or formatting,

4. Changing the mode of administration, OR

5. Changing the target culture or language for application of the instrument

Draft Guidance, Lines 570-671

Draft guidance contents (2)

• Specific populations (children, cognitively impaired)

• Study design (blinding and randomisation, frequency of assessments etc)

• ePROs (record keeping, maintenance and access issues)

• Data analysis (missing data, interpretation etc)

After the draft guidance

• Consultation period – comments sent to FDA• Series of papers in Value in Health expanding

on the issues in the guidance • ISPOR Task forces

– Content validity– Translation and linguistic validation– ePRO

• Final guidance will be released……

PRO Review on Critical PathPreclinical Phase 2A Phase 2B Phase 3Phase I

Pre-IND EOP2BEOP2AEOP1

Initial PRO Submission

(Updated) PRO submission

Submit Final PRO

Evidence Dossier

PRO Selection/ Development

PRO Validation/ Modification

Approved Product LabelsPRO Claims in FDA and EMEA Product Labels Approved between 1 January 2000 and 2 June 2008

FDA Approvals by Year EMEA Approvals by Year

Year HRQLSigns & Sympto

ms

AnyPRO

All Approva

lsHRQL

Signs & Sympto

ms

Any PRO

All Approva

ls

2000 0 25 25 96 3 1 4 31

2001 1 18 18 71 6 11 13 38

2002 1 14 15 79 0 10 10 38

2003 1 17 18 79 2 6 9 18

2004 0 20 23 114 6 8 11 36

2005 1 13 15 82 3 7 7 23

2006 4 15 18 98 0 5 5 42

2007 1 11 14 75 7 13 13 69

2008* 0 4 4 34 2 4 4 26

Total 9 137 150 728 29 65 76 331

8.5 yrMean

1.1 16.1 17.6 85.6 3.4 7.6 8.9 37.8

Label Claims Based on Patient-Reported Outcomes in EMEA and FDA Approvals Since 2000Jane Scott etal. Poster presented at ISPOR Athens 2008

ePRO migration:level of evidence required to establish equivalence

Migration and Re-validation

• FDA guidance states “the extent of additional validation recommended depends on the type of modification made.” (pg 20, line 582-583)

• Ambiguity

ISPOR ePRO consensus group: Modification categories

• Minor – no change in content or meaning (non substantive changes in instructions, minor changes in format)

• Moderate – May change content or meaning (changes in item wording or changes in presentation that might alter interpretability, visual to aural)

• Substantial – Clear change in content or meaning (substantial changes in response options or item wording)

ISPOR ePRO consensus group: Levels of evidence

• Minor – Small scale cognitive debriefing and usability testing

• Medium – Formal assessment of equivalence and usability testing

• Substantial – large scale usability testing and full psychometric testing

Levels of Evidence

• Cognitive Debriefing– Explores the way individuals understand, mentally

process and respond to items on a questionnaire (Willis 2005)

– Similar to target sample– 5-10 patients– Verbal probing technique– Needs to be documented

Levels of Evidence

• Usability Testing– Related to the software and device– Navigation of electronic platform– Scale should be based on the complexity of

the physical and cognitive tasks required by the ePRO application

Levels of Evidence

• Equivalence Testing– Administer paper and ePRO versions– Parallel vs cross-over– Similar to target sample– Basic classical test theory techniques (internal

consistency, means, standard deviations etc)

Levels of Evidence

• Psychometric Evaluation – Factor structure– Internal consistency– Test-retest– Construct validity– Responsiveness– MID

Comparisons of ePRO and paper PRO

• Gwaltney et al 2008 – meta-analysis of 46 studies and 275 PROs

• Very high correlations (.90)• Little evidence to suggest that size of

computer screen, respondent age, or amount of computer experience had any impact

• Respondents prefer ePRO over paper

Translation issues

• Translation can be significantly more wordy than source text – mixed modes of administration not recommended

• Possibilities of combining linguistic validation steps with migration cognitive debriefing

Summary

• PROs useful umbrella term for a variety of measures

• FDA draft guidance has helped to clarify and streamline review process

• Level of evidence for equivalence of ePROs is dependent on changes required for migration

• Growing evidence to suggest that equivalence is generally high and ePROs preferred by patients

Contacts

• Diane Wild,

Oxford Outcomes,

Diane.wild@oxfordoutcomes.com