implementation and analysis of pros in clinical trials
DESCRIPTION
Implementation and Analysis of PROs in Clinical Trials. Jeff A. Sloan, Ph.D. DIA Meeting, D.C., June 26, 2005. Why is it difficult to deal with PROs?. Relatively recent acceptance 25 years ago physicians were the sole raters of patient pain - PowerPoint PPT PresentationTRANSCRIPT
CM923712-1
Implementation and Analysis of Implementation and Analysis of PROs in Clinical TrialsPROs in Clinical Trials
Implementation and Analysis of Implementation and Analysis of PROs in Clinical TrialsPROs in Clinical Trials
DIA Meeting, D.C., June 26, 2005DIA Meeting, D.C., June 26, 2005
Jeff A. Sloan, Ph.D.Jeff A. Sloan, Ph.D.
CM923712-2
Why is it difficult to deal with PROs?Why is it difficult to deal with PROs?Why is it difficult to deal with PROs?Why is it difficult to deal with PROs?
• Relatively recent acceptance
• 25 years ago physicians were the sole raters of patient pain
• JCAHO 2000 guideline: every patient’s pain to be assessed upon intake on a 0-10 scale
• Time and experience alleviates novelty and skepticism
• Relatively recent acceptance
• 25 years ago physicians were the sole raters of patient pain
• JCAHO 2000 guideline: every patient’s pain to be assessed upon intake on a 0-10 scale
• Time and experience alleviates novelty and skepticism
CM923712-3
Checklist for designing, conducting and reporting Checklist for designing, conducting and reporting HRQL - PRO in clinical trialsHRQL - PRO in clinical trials
Patient Reported Outcomes (PRO) and Regulatory Issues : A European Guidance Document for the Patient Reported Outcomes (PRO) and Regulatory Issues : A European Guidance Document for the improved integration of health-related quality of life assessment in the drug regulatory process. improved integration of health-related quality of life assessment in the drug regulatory process. Chassany Chassany
O et O et ERIQAERIQA Working Group. Drug Information Journal 2002. Working Group. Drug Information Journal 2002.
Checklist for designing, conducting and reporting Checklist for designing, conducting and reporting HRQL - PRO in clinical trialsHRQL - PRO in clinical trials
Patient Reported Outcomes (PRO) and Regulatory Issues : A European Guidance Document for the Patient Reported Outcomes (PRO) and Regulatory Issues : A European Guidance Document for the improved integration of health-related quality of life assessment in the drug regulatory process. improved integration of health-related quality of life assessment in the drug regulatory process. Chassany Chassany
O et O et ERIQAERIQA Working Group. Drug Information Journal 2002. Working Group. Drug Information Journal 2002.
HRQL / PRO objectives• Added value of HRQL / PRO• Choice of the questionnaires• Hypotheses of HRQL / PRO changes
Study design• Basic principles of RCT fulfilled ?• Timing and frequency of assessment• Mode and site of administration...
HRQL / PRO measure• Description of the measure (items, domains…)• Evidence of validity• Evidence of cultural adaptation
Statistical analysis plan• Primary or secondary endpoint • Superiority or equivalence trial• Sample size• ITT, type I error, missing data
Reporting of results• Participation rate, data completeness• Distribution of HRQL / PRO scores
Interpreting the results• Effect size, • Minimal Clinically Important Difference• Comparison with other criteria / scores• Number needed to treat…
CM923712-4
Take home messages:Take home messages:there is good newsthere is good news
Take home messages:Take home messages:there is good newsthere is good news
• There are problems with using PROs as There are problems with using PROs as indicators of efficacy in clinical trials.indicators of efficacy in clinical trials.
• There are scientifically sound solutions There are scientifically sound solutions to these problems. The problems have to these problems. The problems have been disseminated widely and been disseminated widely and consistently. The solutions have not.consistently. The solutions have not.
• There are problems with using PROs as There are problems with using PROs as indicators of efficacy in clinical trials.indicators of efficacy in clinical trials.
• There are scientifically sound solutions There are scientifically sound solutions to these problems. The problems have to these problems. The problems have been disseminated widely and been disseminated widely and consistently. The solutions have not.consistently. The solutions have not.
CM923712-5
It takes a certain amount of bravery to work with PRO’sIt takes a certain amount of bravery to work with PRO’s
CM923712-6
Primary goal: advance the state of the science Primary goal: advance the state of the science to help patients soarto help patients soar
CM923712-7
How do you analyze PRO data?How do you analyze PRO data?
CM923712-8
Science is a candle in the darkScience is a candle in the dark - Carl Sagan- Carl Sagan
We will use the candle of science to We will use the candle of science to improve the QOLimprove the QOL of cancer patients of cancer patients
CM923712-9
… … by answering by answering scientific questionsscientific questions
… … by answering by answering scientific questionsscientific questions
• What is the value added of PROs to What is the value added of PROs to treatment trials?treatment trials?
• How do you deal with multiple endpoints?How do you deal with multiple endpoints?
• How do you handle missing data?How do you handle missing data?
• What is the clinical significance of PRO What is the clinical significance of PRO assessments?assessments?
• What is the value added of PROs to What is the value added of PROs to treatment trials?treatment trials?
• How do you deal with multiple endpoints?How do you deal with multiple endpoints?
• How do you handle missing data?How do you handle missing data?
• What is the clinical significance of PRO What is the clinical significance of PRO assessments?assessments?
CM923712-10
What is the value added What is the value added of additional questions?of additional questions?
CM923712-11
CM923712-12
Situations where a single item maysuffice
Situations where a multi-item indexmay be needed
Phase II study attempting to assesswhether a treatment has any impact onQOL
A Phase III study where it is known thatQOL is impacted and more delineationof which QOL components are affectedis needed
A stratification factor for the presenceor absence of depressive issues
A screen to identify the presence orabsence of clinical depression
Need to assess fatigue/pain as acorrelate of toxicity (brief fatigue/paininventory)
Need to assess the impact offatigue/pain on the activities of dailyliving (ADL items for pain/fatigue)
Identifying patients who have need offurther QOL assessment (e.g., score of6 or less on a single item)
Detailing the QOL-related issues oncea cut off score on a single item hasbeen obtained
A clinical setting wherein a basic ideaof which domains of QOL (mental,physical, social) may be affected by aparticular treatment or situation
A clinical setting wherein preciseindications of the way in which thedifferent domains of QOL may beaffected by a particular treatment orsituation
Single-Item or Multiple-Item PRO?Single-Item or Multiple-Item PRO?
CM923712-13
Guidelines for endpoint determinationGuidelines for endpoint determinationGuidelines for endpoint determinationGuidelines for endpoint determination
• Several good references Several good references (Beitz, 1996; (Beitz, 1996; Chassany, 2002; Fayers, 1999; Sloan, 2002)Chassany, 2002; Fayers, 1999; Sloan, 2002)
• Reliability and validity data availableReliability and validity data available
• Pilot/focus groups to establish R/VPilot/focus groups to establish R/V
• What aspects of PROs are likely to What aspects of PROs are likely to change? change?
• Can one expect an overall change in Can one expect an overall change in well-being, health status or QOL?well-being, health status or QOL?
• Several good references Several good references (Beitz, 1996; (Beitz, 1996; Chassany, 2002; Fayers, 1999; Sloan, 2002)Chassany, 2002; Fayers, 1999; Sloan, 2002)
• Reliability and validity data availableReliability and validity data available
• Pilot/focus groups to establish R/VPilot/focus groups to establish R/V
• What aspects of PROs are likely to What aspects of PROs are likely to change? change?
• Can one expect an overall change in Can one expect an overall change in well-being, health status or QOL?well-being, health status or QOL?
CM923712-14
Recipe for endpoint determinationRecipe for endpoint determinationRecipe for endpoint determinationRecipe for endpoint determination
• List PRO aspects likely to change.List PRO aspects likely to change.
• Operationalize each item from a tool.Operationalize each item from a tool.
• Survey clinicians/patients if unsure.Survey clinicians/patients if unsure.
• Keep the total number of items under 25.Keep the total number of items under 25.
• Mock up tables with “perfect world” data, Mock up tables with “perfect world” data, labels with “perfect” results.labels with “perfect” results.
• Link sample size to a priori clinical Link sample size to a priori clinical significance.significance.
• List PRO aspects likely to change.List PRO aspects likely to change.
• Operationalize each item from a tool.Operationalize each item from a tool.
• Survey clinicians/patients if unsure.Survey clinicians/patients if unsure.
• Keep the total number of items under 25.Keep the total number of items under 25.
• Mock up tables with “perfect world” data, Mock up tables with “perfect world” data, labels with “perfect” results.labels with “perfect” results.
• Link sample size to a priori clinical Link sample size to a priori clinical significance.significance.
CM923712-15
How do you deal How do you deal with multiple endpoints?with multiple endpoints?
How do you deal How do you deal with multiple endpoints?with multiple endpoints?
CM923712-16
An example of combined symptoms:An example of combined symptoms:Gemzar (gemcitabine)Gemzar (gemcitabine)
An example of combined symptoms:An example of combined symptoms:Gemzar (gemcitabine)Gemzar (gemcitabine)
• Indication: Indication: Advanced pancreatic cancerAdvanced pancreatic cancer
• Instrument or Method: Instrument or Method:
• Negotiated PRO outcome, “clinical benefit response”Negotiated PRO outcome, “clinical benefit response”
• PRO Domains Assessed: PRO Domains Assessed:
• Pain, analgesic consumption, performance status, Pain, analgesic consumption, performance status, weightweight
• Results:Results:
• Clinical benefit response was experienced by 24% of Clinical benefit response was experienced by 24% of patients receiving Gemzar versus 5% of patients patients receiving Gemzar versus 5% of patients receiving 5FU, p=0.002receiving 5FU, p=0.002
• Indication: Indication: Advanced pancreatic cancerAdvanced pancreatic cancer
• Instrument or Method: Instrument or Method:
• Negotiated PRO outcome, “clinical benefit response”Negotiated PRO outcome, “clinical benefit response”
• PRO Domains Assessed: PRO Domains Assessed:
• Pain, analgesic consumption, performance status, Pain, analgesic consumption, performance status, weightweight
• Results:Results:
• Clinical benefit response was experienced by 24% of Clinical benefit response was experienced by 24% of patients receiving Gemzar versus 5% of patients patients receiving Gemzar versus 5% of patients receiving 5FU, p=0.002receiving 5FU, p=0.002
CM923712-17
Gemzar-specific clinical benefit response
CM923712-18
A patient was considered a clinical A patient was considered a clinical benefit responder to Gemzar if ….benefit responder to Gemzar if ….
A patient was considered a clinical A patient was considered a clinical benefit responder to Gemzar if ….benefit responder to Gemzar if ….
• The patient showed >=50% reduction in pain The patient showed >=50% reduction in pain intensity or analgesic consumption, or a 20+ point intensity or analgesic consumption, or a 20+ point improvement in performance status (for at least 4 improvement in performance status (for at least 4 weeks with no worsening of other parameters)weeks with no worsening of other parameters)
• Memorial Pain Assessment Card and Karnofsky Memorial Pain Assessment Card and Karnofsky Performance ScalePerformance Scale
• The patient was stable on all of the parameters The patient was stable on all of the parameters mentioned and showed a marked, sustained mentioned and showed a marked, sustained weight gain not due to fluid accumulation ( >7% weight gain not due to fluid accumulation ( >7% increase maintained for 4 weeks)increase maintained for 4 weeks)
• The patient showed >=50% reduction in pain The patient showed >=50% reduction in pain intensity or analgesic consumption, or a 20+ point intensity or analgesic consumption, or a 20+ point improvement in performance status (for at least 4 improvement in performance status (for at least 4 weeks with no worsening of other parameters)weeks with no worsening of other parameters)
• Memorial Pain Assessment Card and Karnofsky Memorial Pain Assessment Card and Karnofsky Performance ScalePerformance Scale
• The patient was stable on all of the parameters The patient was stable on all of the parameters mentioned and showed a marked, sustained mentioned and showed a marked, sustained weight gain not due to fluid accumulation ( >7% weight gain not due to fluid accumulation ( >7% increase maintained for 4 weeks)increase maintained for 4 weeks)
CM923712-19
O’Brien Global Test O’Brien Global Test for Multiple Outcomesfor Multiple OutcomesO’Brien Global Test O’Brien Global Test
for Multiple Outcomesfor Multiple Outcomes
• Example: Venlafaxine for Hot FlashesExample: Venlafaxine for Hot Flashes
• Hot flash frequency per dayHot flash frequency per day
• Hot flash average severity per day Hot flash average severity per day
• none, mild, moderate, severe, very severenone, mild, moderate, severe, very severe
• scored 0, 1, 2, 3, 4scored 0, 1, 2, 3, 4
• Hot flash score (severity times frequency)Hot flash score (severity times frequency)
• Uniscale QOLUniscale QOL
• Hot flash affect on QOLHot flash affect on QOL
• Toxicity incidence on 11 variablesToxicity incidence on 11 variables
• Example: Venlafaxine for Hot FlashesExample: Venlafaxine for Hot Flashes
• Hot flash frequency per dayHot flash frequency per day
• Hot flash average severity per day Hot flash average severity per day
• none, mild, moderate, severe, very severenone, mild, moderate, severe, very severe
• scored 0, 1, 2, 3, 4scored 0, 1, 2, 3, 4
• Hot flash score (severity times frequency)Hot flash score (severity times frequency)
• Uniscale QOLUniscale QOL
• Hot flash affect on QOLHot flash affect on QOL
• Toxicity incidence on 11 variablesToxicity incidence on 11 variables
CM923712-20
O’Brien p-valuesO’Brien p-valuesO’Brien p-valuesO’Brien p-values
Endpoints IncludedEndpoints Included p-valuep-value
Hot Flash FrequencyHot Flash Frequency
Hot Flash Average Severity 0.0071 Hot Flash Average Severity 0.0071
Hot Flash Score 0.0050Hot Flash Score 0.0050
Uniscale QOL 0.7528 Uniscale QOL 0.7528
Hot Flash Affects QOL Hot Flash Affects QOL
ToxicityToxicity
Endpoints IncludedEndpoints Included p-valuep-value
Hot Flash FrequencyHot Flash Frequency
Hot Flash Average Severity 0.0071 Hot Flash Average Severity 0.0071
Hot Flash Score 0.0050Hot Flash Score 0.0050
Uniscale QOL 0.7528 Uniscale QOL 0.7528
Hot Flash Affects QOL Hot Flash Affects QOL
ToxicityToxicity
CM923712-21
How do you handle the How do you handle the problem of missing data?problem of missing data?
How do you handle the How do you handle the problem of missing data?problem of missing data?
CM923712-22
Impact of hydrazine sulfate on colorectal cancer patient QOL
Impact of different imputation methods for missing data
CM923712-23
Effect of imputation method on treatment comparison
CM923712-24
The data are usually trying to tell you something….
…you just have to pay attention
CM923712-25
What is the clinical significanceWhat is the clinical significance
of PRO assessments?of PRO assessments?
CM923712-26
Two general methods Two general methods for clinical significancefor clinical significanceTwo general methods Two general methods for clinical significancefor clinical significance
• Anchor-based methods requirementsAnchor-based methods requirements
• independent interpretable measure independent interpretable measure (the anchor) which has appreciable (the anchor) which has appreciable correlation between anchor and correlation between anchor and targettarget
• Distribution-based methodsDistribution-based methods
• rely on expression of magnitude of rely on expression of magnitude of effect in terms of measure of effect in terms of measure of variability of results (effect size)variability of results (effect size)
• Anchor-based methods requirementsAnchor-based methods requirements
• independent interpretable measure independent interpretable measure (the anchor) which has appreciable (the anchor) which has appreciable correlation between anchor and correlation between anchor and targettarget
• Distribution-based methodsDistribution-based methods
• rely on expression of magnitude of rely on expression of magnitude of effect in terms of measure of effect in terms of measure of variability of results (effect size)variability of results (effect size)
CM923712-27
The MID method in one slide
CM923712-28
The ERES ApproachThe ERES ApproachThe ERES ApproachThe ERES Approach• QOL tool range QOL tool range = 6 standard Deviations= 6 standard Deviations
• SD Estimate SD Estimate == 100 percent / 6100 percent / 6
= 16.7% of theoretical range= 16.7% of theoretical range
• Two-sample t-test effect sizes (Cohen): Two-sample t-test effect sizes (Cohen):
small, moderate, large effectsmall, moderate, large effect (0.2, 0.5, 0.8 SD shift)(0.2, 0.5, 0.8 SD shift)
• S,M,L effects S,M,L effects = 3%, 8%, 12% of range= 3%, 8%, 12% of range
• QOL tool range QOL tool range = 6 standard Deviations= 6 standard Deviations
• SD Estimate SD Estimate == 100 percent / 6100 percent / 6
= 16.7% of theoretical range= 16.7% of theoretical range
• Two-sample t-test effect sizes (Cohen): Two-sample t-test effect sizes (Cohen):
small, moderate, large effectsmall, moderate, large effect (0.2, 0.5, 0.8 SD shift)(0.2, 0.5, 0.8 SD shift)
• S,M,L effects S,M,L effects = 3%, 8%, 12% of range= 3%, 8%, 12% of range
CM923712-29
Assessing Clinical SignificanceAssessing Clinical SignificanceAssessing Clinical SignificanceAssessing Clinical Significance
• 1) Methods used to date
• 2) Group versus individual differences
• 3) Single item versus multi-item
• 4) Patient, clinician, population perspectives
• 5) Changes over time
• 6) Practical considerations for specific audiences
• MCP, April, May, June 2002MCP, April, May, June 2002
• 1) Methods used to date
• 2) Group versus individual differences
• 3) Single item versus multi-item
• 4) Patient, clinician, population perspectives
• 5) Changes over time
• 6) Practical considerations for specific audiences
• MCP, April, May, June 2002MCP, April, May, June 2002
CM923712-30
The solutions found for tumor response The solutions found for tumor response cutoffs may provide guidancecutoffs may provide guidance
The solutions found for tumor response The solutions found for tumor response cutoffs may provide guidancecutoffs may provide guidance
• We call a reduction of 50% a response. We call a reduction of 50% a response.
• Have reductions of 49% all the time, but Have reductions of 49% all the time, but do not worry about misclassification.do not worry about misclassification.
• Moertel (1976) basis for 50% cutoffMoertel (1976) basis for 50% cutoff
• Find a cutoff and stick to it?Find a cutoff and stick to it?
• We call a reduction of 50% a response. We call a reduction of 50% a response.
• Have reductions of 49% all the time, but Have reductions of 49% all the time, but do not worry about misclassification.do not worry about misclassification.
• Moertel (1976) basis for 50% cutoffMoertel (1976) basis for 50% cutoff
• Find a cutoff and stick to it?Find a cutoff and stick to it?
CM923712-31
The Good NewsThe Good NewsThe Good NewsThe Good News
• Statistical, Philosophical, Empirical, Clinical, Historical, Practical Statistical, Philosophical, Empirical, Clinical, Historical, Practical approaches to defining a clinically significant effect for approaches to defining a clinically significant effect for symptom assessments are all in the same ballparksymptom assessments are all in the same ballpark
• A 10 point difference on a 100-point scale (1/2 SD) is almost A 10 point difference on a 100-point scale (1/2 SD) is almost always going to be clinically significantalways going to be clinically significant
• Smaller differences may also be meaningful (data)Smaller differences may also be meaningful (data)
• Applies to groups or individuals (just different SD)Applies to groups or individuals (just different SD)
Norman GR, Sloan JA, Wyrwich KW. Expert Review of Pharmacoeconomics and Outcomes Research Sept 2004; 4(5): Norman GR, Sloan JA, Wyrwich KW. Expert Review of Pharmacoeconomics and Outcomes Research Sept 2004; 4(5): 515 – 519515 – 519
Sloan JA, Cella D, Hays R. J Clin Epidemiol (in press).Sloan JA, Cella D, Hays R. J Clin Epidemiol (in press).
• Statistical, Philosophical, Empirical, Clinical, Historical, Practical Statistical, Philosophical, Empirical, Clinical, Historical, Practical approaches to defining a clinically significant effect for approaches to defining a clinically significant effect for symptom assessments are all in the same ballparksymptom assessments are all in the same ballpark
• A 10 point difference on a 100-point scale (1/2 SD) is almost A 10 point difference on a 100-point scale (1/2 SD) is almost always going to be clinically significantalways going to be clinically significant
• Smaller differences may also be meaningful (data)Smaller differences may also be meaningful (data)
• Applies to groups or individuals (just different SD)Applies to groups or individuals (just different SD)
Norman GR, Sloan JA, Wyrwich KW. Expert Review of Pharmacoeconomics and Outcomes Research Sept 2004; 4(5): Norman GR, Sloan JA, Wyrwich KW. Expert Review of Pharmacoeconomics and Outcomes Research Sept 2004; 4(5): 515 – 519515 – 519
Sloan JA, Cella D, Hays R. J Clin Epidemiol (in press).Sloan JA, Cella D, Hays R. J Clin Epidemiol (in press).
CM923712-32
What’s next?What’s next?
CM923712-33
A Mayo/NCCTG meeting onA Mayo/NCCTG meeting onFDA guidances on patient-reported outcomes (PRO)FDA guidances on patient-reported outcomes (PRO)
Discussion, Education, and OperationalizationDiscussion, Education, and Operationalization
A Mayo/NCCTG meeting onA Mayo/NCCTG meeting onFDA guidances on patient-reported outcomes (PRO)FDA guidances on patient-reported outcomes (PRO)
Discussion, Education, and OperationalizationDiscussion, Education, and Operationalization
• FDA to release guidances for assessing PRO’s in all clinical trials (3rd quarter 2005?)
• Meeting co-sponsored with FDA to:• provide a focused process to facilitate discussion among all
stakeholders
• educate stakeholders on background, content, and concerns
• provide an opportunity for input
• delineate ways to best operationalize the guidance into clinical trials
• February 23-25, 2006, DC (Westfields Marriott, Chantilly, VA, 7 miles from Dulles)
• Seeking stakeholders involvement
• FDA to release guidances for assessing PRO’s in all clinical trials (3rd quarter 2005?)
• Meeting co-sponsored with FDA to:• provide a focused process to facilitate discussion among all
stakeholders
• educate stakeholders on background, content, and concerns
• provide an opportunity for input
• delineate ways to best operationalize the guidance into clinical trials
• February 23-25, 2006, DC (Westfields Marriott, Chantilly, VA, 7 miles from Dulles)
• Seeking stakeholders involvement
CM923712-34
New ideas have enabled us to make advances in PRO science New ideas have enabled us to make advances in PRO science
With your help, there will be more to comeWith your help, there will be more to come
