aifa european conference on clinical research for decision making
Post on 20-Jan-2016
36 Views
Preview:
DESCRIPTION
TRANSCRIPT
1
AIFA EUROPEAN CONFERENCE ON CLINICAL RESEARCH FOR DECISION MAKING
External validity of clinical trials and transferability of
their results to medical practice
March 30, 2007
Luigi Pagliaro, Professor of Medicine, University of Palermo, Palermo, Italy
2
External validity of clinical trials and transferability of their results to medical
practice
1.External validity or applicability
2.Applicability: information needed
3.Problems of applicability
4.Help for evaluating applicability
Final remarks
References
3
In an RCT:
Internal validity refers to the comparability between the intervention (I) and the control group (C), and to the care for minimizing the risk of bias External validity refers to the availability of information to guide the applicability of the trial results to the target population in a specific setting (1)
I. group C. group
Trial results
Target population and
setting
External validity,
applicability
External validity or applicability
4
There is consensus that RCTs with adequate internal validity most reliably assess the efficacy of clinical treatments (2) The statistical efficiency of RCTs is maximized in the megatrials that enroll very large numbers of pts belonging to different categories to detect “moderate but still worthwhile treatment effects” on major outcomes (3)
A fundamental idea underlying the logic of the megatrials is that the direction (although not the size) of a treatment effect is similar throughout the different categories of pts (4)
1. External validity or applicability
5
From 5
“The modest benefit ascribed to many treatments in clinical trials can be misleading because modest average effects may reflect a mixture of substantial benefit for some, little benefit for many, and harm for a few”
1. External validity or applicability
6
RCT
Megatrials are the best method to answer the question: “can this treatment work”? (6)
Individual pts
……But the applicability of the treatment to the practice must take into account the characteristics of the individual patients (7)
1. External validity or applicability
7
External validity of clinical trials and transferability of their results to medical
practice
1.External validity or applicability
2.Applicability: information needed
3.Problems of applicability
4.Help for evaluating applicability
Final remarks
References
8
1. Characteristics of randomised
pts
Eligibility; inclusion & exclusion criteria; ratio of randomised pts to eligible non-randomised pts; baseline risk; stage in the natural history and severity of disease; comorbidity
2. Setting
Recruitment from primary, secondary or tertiary care; characteristics & expertise of participating centres and clinicians
2 Applicability: information needed (from 1, modified)
9
3. Differences of trial
protocol vs practice
Doses and administration of the trial treatment; adequacy of non- trial treatments; therapeutic or diagnostic advances since the trial
4. Outcome measures
and follow-up
Clinical relevance of the end points and effect size; effect of the intervention on the most relevant components of a combined end point
5. Adverse effects of treatment
Rates of adverse effects and of treatment discontinuations; were pts at risk of complications excluded from the trial?
2 Applicability: information needed (from 1, modified)
10
Lack of this information is one of the main reasons for the underuse in practice of treatments that were beneficial in trials (1), and/or of the use of a treatment in the “wrong” patients,sometimes with harmful consequences
Many RCTs don’t report the information needed to evaluate the applicability of their results to the practice, and/or they exclude specific subgroups of pts that could benefit from the treatment (8-11)
2 Applicability: information needed (from 1, modified)
11
External validity of clinical trials and transferability of their results to medical
practice
1. External validity or applicability
2. Applicability: information needed
3.Problems of applicability
4. Help for evaluating applicability
Final remarks
References
12
RCTs lacking information needed to apply the
results in groups of patients; some examples:
Treatment for patients with hypertension and comorbidities
Peg-interferon in HCV-related cirrhosis without knowledge of esophagogastric varices
Tolvaptan for hyponatremia in cirrhosis
Warfarin for patients requiring caution
3. Problems of applicability
13
Inappropriate use of trial results to “wrong” patients and risk of harmful consequences; some examples
Endarterectomy for asymptomatic carotid stenosis in centres and surgeons with lesser expertise than those participating in the trial
Spironolactone prescribed to pts with heart failure more vulnerable to hyperkalemia than those in the trial
3. Problems of applicability
14
RCT Main comorbidities excluded
Appel & al Renal damage &/or diabetes
Hansson & al None reported
Wing & al Plasma creatinine >2.5 mg/dl
ALLHAT, pts≥ 55-year-old
History of HF or EF < 35%
Sacks & al Heart disease, diabetes+insulin, renal failure
Comorbidities were present in 89-100% of the Fortin’s pts potentially eligible to
each RCT
3. Problems of applicability
Comorbidity in hypertensive pts in primary care (12)
15
Heathcote EJ & al. Peginterferon alfa-2° in patients with chronic hepatitis and cirrhosis. N Engl J Med 2000; 343: 1673-80 (13) The paper does not report whether patients
with esophago-gastric varices were included in the study This information would have been of help for decision making: esophagogastric varices (prevalence about 40% in Child A cirrhosis, target of the study) suggest an advanced stage of disease (14), with a lower probability of interferon response (15)
3. Problems of applicability
Lack of information
16
Schrier RW & al (SALT trials). Tolvaptan, a selective oral vasopressin V2-receptor antagonist, for hyponatremia. N Engl J Med 2006; 355: 2099-112 (16); letters: 2007; 356: 961-3 The trial excluded cirrhotics with Child-Pugh score >10
The information about these pts would have been of paramount relevance: at least 50% of cirrhotics who develop significant hyponatremia have Child-Pugh score >10 (17, 18)
Only 67 cirrhotics were treated with Tolvaptan
The trial cannot provide any information on the risk of adverse events of the drug in cirrhotics
3. Problems of applicability
Lack of information
17
Schrier RW & al (SALT trials). Tolvaptan, a selective oral vasopressin V2-receptor antagonist, for hyponatremia. N Engl J Med 2006; 355: 2099-112 (16); letters: 2007; 356: 961-3 The trial excluded cirrhotics with Child-Pugh score >10; only 67 cirrhotics received Tolvaptan
3. Problems of applicability
Lack of information
The information about these pts would have been of paramount relevance: at least 50% of cirrhotics who develop significant hyponatremia have Child-Pugh score >10 (17, 18) The trial cannot provide any information on the risk of adverse events of the drug in cirrhotics
18
Hart RG & al. Antithrombotic therapy to prevent stroke in pts with atrial fibrillation: a meta-analysis. Ann Intern Med 1999; 131: 492-501 (19)
Exclusion criteria of the RCTs are often ill-defined (eg: “medical disorders” [20]; “contraindications for Warfarin or aspirin therapy” [21]), and information on co-treatments for pts with comorbidities is generally omitted
3. Problems of applicability
Lack of information
Clinicians may be uncertain about the risk/benefit of Warfarin in a substantial number of pts: a reason for its underuse (30-60% [22])?
19
Heterogeneity X2: p < 0.0001
Baseline riskEndoscopic sclerotherapy for prevention of first variceal bleeding in cirrhosis: only RCTs with bleeding rate > 40% in untreated controls show a significant benefit (data adjusted for the duration of follow up); data subsequently confirmed (2)
1. From: Pagliaro L & al. Ann Intern Med 1992; 117: 59-70
2. D'Amico G, Hepatology 1995; 22: 332-54
3.1 External validity and applicability to subgroups
and individuals
20
The ACAS (Asymptomatic Carotid Artery Surgery) trial of carotid endarterectomy vs medical treatment rejected 40% of applicants,subsequently barring those with adverse operative outcomes (1)
However: in USA, most carotid endarterectomies are performed by surgical teams whose complication rates would have rendered them ineligible for ACAS (1,23): results of the ACAS trial and of 8 published series
ACAS 8 series
Stroke & death, % (CI)
1.50 (0.6-2.4)
4.30 (3.5-5.2)
Inappropriate use of RCT results
3. Problems of applicability
21
The RALES trial (24): Spironolactone significantly reduced all-cause & cardiac mortality in pts with severe congestive heart failure (NYHA class III-IV)
Pts with serum creatinine concentration > 2.5 mg/dl were excluded Mean age 65 years; females 27%; proportion of diabetics not reported Mean doses of spironolactone 26 mg/day; pts co- treated with β-blockers 10%
No hyperkalemia-related deaths in the trial; however, in the years following the publication of the RALES:
Inappropriate use of RCT results
3. Problems of applicability
22
Hyperkalemia-related deaths per 1000 pts
Prescriptions of Spironolactone per 1000 pts
RALES
RALES
3. Problems of applicability
After the RALES publication, spironolactone prescriptions greatly increased, even for pts more vulnerable to hyperkalemia than those in the trial (eg for elderly, diabetics, and for pts co-treated with β-blockers), or for pts excluded from the trial (ie HF in NYHA class I and II; mild renal failure); higher doses were often given (25, 26)
Inappropriate use
of RCT results
23
External validity of clinical trials and transferability of their results to medical
practice
1. External validity or applicability
2. Applicability: information needed
3. Problems of applicability
4. Help for evaluating applicability
Final remarks
References
24
The drug X is effective in the disease X1
Evidence from an unbiased RCT
The drug X can work in my pts with the disease X1, provided that:
They have the characteristics a, b, c….m
However, there is no evidence to support the drug X if they have the characteristics n, o, p…..z
An approach requiring time
and competence
Interpretive medicine (27, 28)
4.Help for evaluating applicability
25
Guidelines built on the global evidence of many RCTs may fill in the gaps
of information of the individual RCTs
0
20
40
60
80
100
Individual RCTs
Pre-appraised summaries Guidelines
5
72 84
% preferences of 302 GP(From 29)
Guidelines
4.Help for evaluating applicability
26
Potential heterogeneity in risks of treatment or in risk without treatment, eg baseline risk
Potential heterogeneity of treatment effect related to pathophysiology, eg multiple pathologies underlying a clinical syndrome; or genetic variation
Underuse of treatment in clinical practice due to uncertainty about benefit, eg underuse in specific groups of pts such as in elderly
4.Help for evaluating applicability
A subgroup analysis can be worthwhile when established a priori from pathophyisiologic principles in trial populations probably heterogeneous (30,31), eg:
Subgroup analysis
From 31
27
Pharmacogenomic polymorphism
Conclusions. “VKORC1haplotypes can be used to stratify patients into low-, intermediate-,and
high- dose warfarin groups and may explain
differences in dose requirements among
pts of different ancestries”(32) {VKORC1: Vit K
epoxide reductase complex 1}
Subgroup analysis
4.Help for evaluating applicability
28
Pharmacogenomic
polymorphism
Conclusions: in pts with IBD,
very high TPMT activity predicts
treatment failure (33)
{TPMT: Thiopurine
methyltransferase}
Subgroup analysis
4.Help for evaluating applicability
29
Some final remarks
External validity refers to the information needed to promote the applicability of trial results to clinical practice
Main issues of external validity are the characteristics of randomised pts, comorbidities and co-treatments, outcome measures, and setting
Information gaps due to inadequate external validity may lead to clinical decisions unsupported by adequate knowledge on the most appropriate applicability of the treatments
1
30
Overviews and guidelines gathering the global evidence of individual RCTs and analysis of predefined subgroups may aid the transferability of clinical research to practice
A more effective remedy would be the inclusion of the most relevant issues of external validity in the design and reports of the RCTs submitted for approval or publication
Some final remarks
2
31
References
1. Rothwell PM. Lancet 2005; 365: 82-932. Guyatt GH & al for the EBM Working Group. JAMA 1993; 270: 2958-603. Collins R & al. In: Maynard A & Chalmers I Eds. Non-random
reflections on Health Services Research. BMJ Publ Group, 1997, p.197-230
4. Peto R. J Clin Epidemiol 1995; 48: 23-405. Kravitz RL & al. Milbank Quarterly 2004; 82: 661-876. Haynes B. BMJ 1999; 319: 652-37. Rothwell PM & al. Lancet 2005; 365: 256-658. Hejat A& al. Arch Intern Med 2002; 162: 1682-88 (HF)9. Herland K & al. Respir Med 2005; 99: 11-19 (COPD)10.Uijen AA & al. J Clin Epidemiol 2007; 60: 330-5 (Hypertension)11.Van Spall HGC & al. JAMA 2007; 297: 1233-40 (Eligibility)12.Fortin M, & al. Ann Fam Med 2006; 4: 104-10813.Heathcote EJ & al. N Engl J Med 2000; 343: 1673-8014.Pagliaro L & al. Portal hypertension in cirrhosis: natural history. In: Portal hypertension. Pathophysiology and treatment, J Bosch & RJ Groszmann Eds. Blackwell Scientific Publ, 1994: p.72-9215. Poynard T & al Gastroenterology 2002; 122: 1303-1316. Schrier RW & al. N Engl J Med 2006; 355: 2099-2112
32
References
17. Angeli P & al. Hepatology 2006; 44: 1535-4218. Biggins SW & al. Hepatology 2005; 41: 32-919. Hart RG & al. Ann Intern Med 1999; 131: 492: 50120. Ezekowitz MD & al. N Engl J Med 1992; 327: 1406-1221. GullovAL & al. Arch Intern Med 1998; 158: 1913-2122. Choudhry NK & al. BMJ 2006; 332: 141-323 Hartling L & al. Ann Intern Med 2005; 142: 1100-1124. Pitt B & al (RALES). N Engl J Med 1999; 341: 709-1725 Juurlink DN & al. (Ontario Study) N Engl J Med 2004; 351: 543-5126. Bozkurt B & al. JACC 2003; 41: 211-427. Horton R. Stat Med 2000; 19: 3149-6428. Horton R. Can Med Ass J (CMAJ) 1998; 158: 245-9
29. Guyatt GH & al. BMJ 2000; 320: 954-530. Feinstein AR. J Clin Epidemiol 1998; 51: 297-9931. Rothwell PM, Lancet 2005; 365: 176-8632. Rieder MJ & al. N Engl J Med 2005; 352: 2285-9333. Ansari A& al. Aliment Pharmacol Ther 2002; 16: 1743-50
33
stop
top related