FDA Approach to Review of FDA Approach to Review of Outcome Measures for Drug Outcome Measures for Drug

Approval and Labeling:Approval and Labeling:Content ValidityContent Validity

Initiative on Methods, Measurement, and Pain Assessment in Clinical Trials (IMMPACT)

April 17-18, 2014

Elektra Papadopoulos, MD, MPHStudy Endpoints and Labeling Development (SEALD)

Office of New Drugs (OND)Center for Drug Evaluation and Research (CDER)

The views expressed in this presentation are those of the speaker, and do not necessarily represent an official FDA position.

Disclaimer

2

Outline

• Elements of the review of content validity

• Common instrument design shortcomings

3

Elements of the review of content validity for clinical outcome assessments

4

5

Content validity• Content validity is the extent to which the content of

an instrument represents important aspects of a given concept for an intended use and for a defined target population

• Establishing content for a new instrument may involve both qualitative and quantitative research methods. Qualitative data are essential for establishing content validity of a COA.

• Input from the target population of respondents is essential 6

Content validity review• Content validity review necessitates review of each

element of an instrument• Instrument: A means to capture data plus all the

information and documentation that supports its use e.g., – User manual – Instructions– Mode of administration (e.g., self administration or interviewer

administered)– Data collection method (e.g., paper-based, computer assisted)– Scoring

7

Sources of evidence to support content validity

• Literature review• Expert input

– Clinical – Instrument development

• Patient input– Concept elicitation interviews– Cognitive interviews

8

Sample criteria for evaluating items

9

Patrick DL et al; Value in Health; 2011

Common Design Shortcomings in item development

10

Common instrument design shortcomings

• Global ratings of a complex concept• Ratings of change• Assessment of an unobservable concept by an observer• Measuring indirect impacts without also measuring

direct• Inappropriate recall period• Combining more than one concept within a single item

(e.g., double-barreled items)

11

Global ratings of a complex concept• Example:

– Overall, based on all your years of medical practice, how severe is this patient’s overall health condition?

• It is a problem because:– Respondents consider unknown/unspecified aspects of the

patient’s condition and assign a value/rating in an unstandardized manner

• To avoid the problem:– Select/develop instruments that include separate items or

scores for each important sub-concept (e.g., sign or symptom) relevant to the concept of interest

12

Ratings of change• Example:

– Compared to your pain before you had cancer, please rate your pain today

• It is a problem because:– Respondents are required to recall their previous state and

make a comparison to some period in the past– Ratings of change do not describe the patient’s current state

• To avoid the problem– Use items that ask about the patient’s current state in a non-

comparative way

13

Assessment of an unobservable concept by an observer

• Example: – How severe is your child’s pain

• It is a problem because:– Assessments of patients by other individuals should be

limited to observable concepts; ratings of a patient’s feelings/symptoms/mental states cannot validly be done by a person other than the patient

• To avoid the problem:– Use items that ask about observable behaviors (e.g. How

frequently did your child cry?)14

Measuring indirect impacts without also measuring direct

• Example: – I am afraid I won’t have enough time to reach the

bathroom. I worry my incontinence will get worse.• It is a problem because:

– Emotions such as “bother”, “distress”, “afraid” do not measure the effect of treatment on core disease symptoms

• To avoid the problem:– Use a more direct measure (e.g., log of the number of

incontinence episodes)15

Inappropriate recall period• Example:

– For a condition where itch fluctuates daily: Over the past four weeks, on average, how severe was your itching?

• It is a problem because:– The recall period requires not only accurate recall, but

also complex mental tasks (e.g., averaging of experiences that may fluctuate over a lengthy period of time)

• To avoid the problem:– Use items with an appropriate recall period for the

population, concept and clinical trial context16

Combining more than one concept within a single item

• Example: – Today, how many times did you cough up blood OR

sputum?• It is a problem because:

– Does not allow discrimination of which concept the respondent is rating

• To avoid the problem: – Use one concept per item

• Today, how many times did you cough up blood?• Today, how many times did you cough up sputum?

17

18

To Illustrate (Lack of) Content Validity

• Study population = primarily bed-ridden, minimal physical activity (e.g., walking to the bathroom) very difficult

• Concept of interest to evaluate treatment benefit = physical functioning

• Clinical Outcome Assessment = patient-reported questionnaire to assess physical functioning

• Question in the assessment = “Do you have trouble running to the bus?”

19

To Illustrate (Lack of) Content Validity

• Study population reports that mobility is a significant problem

• While interviewing patients, it is identified that their biggest concern/challenge is the ability to stand up from a chair and start walking

• Performance measure used in clinical trials: 6 minute walk test

• No assessment of the relevant component of mobility that is meaningful to this population of patients in their daily lives

SF-36 Physical Function Domain

20

21

Carpal Tunnel Release

Hip Joint Replacement

Patient report of “ability to perform daily activities”

22FDA Guidance: Patient Reported Outcome Measures: Use in Medical Product Development to Support Labeling Claims

Challenges with the use of actigraphy• What aspect of physical activity is measured by

actigraphy?

• What types of data (qualitative and quantitative) would be needed to link actigraphy results to a meaningful aspect of daily functioning in the patient population of interest?

• For example, patients with lower back pain are limited in some aspects of functioning that are not captured by acceleration data such as the ability to stand in one place or lift/carry objects 23

Summary

• Content validity is a critical measurement property that pertains to all COAs

• Inadequate content validity may lead to:– inability to detect a treatment effect and– inability to interpret/describe a treatment effect

• Careful consideration of instrument design is critical to ensure the COA can be interpreted and accurately described in product labeling

24