the golden standard or fools gold selective data, impact on safety clinical trials on trial hai...
TRANSCRIPT
The golden standard or fool’s goldSelective data, impact on safety
Clinical trials on trialHAI Europe Open Seminar 2008
Neues Stadthaus, Berlin, 21 November 2008
Joan-Ramon Laporte
RCTs – The golden standard or fool’s gold?
Internal validity
External validity - Efficacy vs Effectiveness
Publication bias
Fraud
Internal validity
• Did the control group receive optimal treatment?
Internal validity
• Did the control group receive optimal treatment?
• Was the dose of the control group adequate?
Favoursatypical
Favourshaloperidol
Drop out rates by dose of comparator drug in trials of patientswith schizophrenia or related disorders (risk difference and95 % confidence intervals)
Geddes et al., 2000
≤ 12 mg haloperidol
> 12 mg haloperidol
-0.5 -0.4 -0.3 -0.2 -0.1 0 0.1
Internal validity
• Did the control group receive optimal treatment?
• Was the sample size adequate to identify any relevant difference?
• Was the dose of the control group adequate?
Internal validity
• Did the control group receive optimal treatment?
• Did the published results refer to the primary variable?
• Was the sample size adequate to identify any relevant difference?
• Was the dose of the control group adequate?
Internal validity
• Did the control group receive optimal treatment?
• Have all the trial results been published?
• Did the published results refer to the primary variable?
• Was the sample size adequate to identify any relevant difference?
• Was the dose of the control group adequate?
Internal validity
• Did the control group receive optimal treatment?
• Were the results presented as a relative risk reduction, or as an absolute risk reduction?
• Have all the trial results been published?
• Did the published results refer to the primary variable?
• Was the sample size adequate to identify any relevant difference?
• Was the dose of the control group adequate?
RCTs – The golden standard or fool’s gold?
Internal validity
External validity - efficacy vs effectiveness
Publication bias
Fraud
External validity of clinical trials
• Context
• Reference population
• Selection criteria
• Diagnostic criteria
• Follow up
• Treatments (doses, compliance)
• Duration
• Primary and other variables
• Adverse effects
Lancet 2005; 365: 82-93
(Un)transferability to clinical practice
Patients in RCTs differ from those in real practice:– Age– Comorbidity– Other treatments– Doses taken, compliance– Diagnostic criteria
Efficacy vs effectiveness
RCT UCP
Nº of patients 102-103 104-107
Duration Short Longer
Populations High risk groups Potentially the excluded whole
population Comorbidity Generally excluded Often present
Conditions Well defined Ill-defined
Nº of drugs One or limited Undetermined
Dose/dosage Generally constant Often variable
Pattern of use Continuous Intermittent
Follow up Careful Less careful
RCTs – The golden standard or fool’s gold?
Internal validity
External validity - efficacy vs effectiveness
Publication bias
Fraud
RCTs – The golden standard or fool’s gold?
Internal validity
External validity - efficacy vs effectiveness
Publication bias
Fraud
Third Report of the Expert Panel on Detection, Evaluation, and Treatment of High Blood Cholesterol
in Adults (Adult Treatment Panel III) (ATP III)
Financial Disclosure: Dr Grundy has received honoraria from Merck, Pfizer, Sankyo, Bayer, and Bristol-Myers Squibb. Dr Hunninghake has current grants from Merck, Pfizer, Kos Pharmaceuticals, Schering Plough, Wyeth Ayerst, Sankyo, Bayer, AstraZeneca, Bristol-Myers Squibb, and G. D. Searle; he has also received consulting honoraria from Merck, Pfizer, Kos Pharmaceuticals, Sankyo, AstraZeneca, and Bayer. Dr McBride has received grants and/or research support from Pfizer, Merck, Parke-Davis, and AstraZeneca; has served as a consultant for Kos Pharmaceuticals, Abbott, and Merck; and has received honoraria from Abbott, Bristol-Myers Squibb, Novartis, Merck, Kos Pharmaceuticals, Parke-Davis, Pfizer, and DuPont. Dr Pasternak has served as a consultant for and received honoraria from Merck, Pfizer, and Kos Pharmaceuticals, and has received grants from Merck and Pfizer. Dr Stone has served as a consultant and/or received honoraria for lectures from Abbott, Bayer, Bristol-Myers Squibb, Kos Pharmaceuticals, Merck, Novartis, Parke-Davis/Pfizer, and Sankyo. Dr Schwartz has served as a consultant for and/or conducted research funded by Bristol-Myers Squibb, AstraZeneca, Merck, Johnson & Johnson-Merck, and Pfizer.
Conclusions
• The RCT is the best epidemiological method for causal inference
• However, it is often performed in a way which favours the sponsor’s treatment:– In its design
– In data analysis and interpretation
– In the publication of results
• At best, RCTs are one of many pieces of evidence about therapeutic interventions
Conclusions
• The appraisal of innovation should not only take into account the so-called EBM, but also other evidence:– Pharmacodynamics– Pharmacokinetics– Availability of therapeutic alternatives
• The medical literature is no longer reliable for valid information
• Research should be performed, analyzed and published in a way which should be independent from commercially interested parties