what the clinician needs to know about statistics : when is biostatistics not to be trusted ?
DESCRIPTION
What the clinician needs to know about statistics : when is biostatistics not to be trusted ?. Giuseppe Biondi Zoccai Division of Cardiology , University of Turin , Turin , Italy Meta-analysis and Evidence-based medicine Training in Cardiology (METCARDIO), Ospedaletti , Italy. - PowerPoint PPT PresentationTRANSCRIPT
www.metcardio.org
What the clinician needs to know about statistics: when is when is biostatistics not to be trusted?biostatistics not to be trusted?
Giuseppe Biondi ZoccaiGiuseppe Biondi Zoccai
Division of Cardiology, University of Turin, Turin, ItalyDivision of Cardiology, University of Turin, Turin, Italy
Meta-analysis and Evidence-based medicine Training in Meta-analysis and Evidence-based medicine Training in Cardiology (METCARDIO), Ospedaletti, ItalyCardiology (METCARDIO), Ospedaletti, Italy
2nd Fellows’ Meeting, 2-3 October 2009, Bubbio – 3 October 2009, 10:30-10:45
www.metcardio.org
LEARNING GOALS
• What are the goals of biostatistics in clinical research?• Is there a hierarchy in
biostatistics?• When is biostatistics not to be
trusted?
www.metcardio.org
LEARNING GOALS
• What are the goals of biostatistics in clinical research?• Is there a hierarchy in
biostatistics?• When is biostatistics not to be
trusted?
www.metcardio.org
GOALS OF BIOSTATISTICS• Biostatistics is mainly used for causality
appraisal in clinical research• Biostatistics alone cannot however enable
causality inference (i.e. necessary but not sufficient)• As it works within Popper’s approach, it can
only ultimately falsify and never actually prove
www.metcardio.org
KARL POPPER’S EPISTEMOLOGY
You can never prove that something is correct in science, you can only disprove something, i.e. show it is wrong
Thus, only falsifiable hypotheses are scientific*(and that’s why religion is not scientific)
www.metcardio.org
• Strength:* precisely (p<0.05) defined and strong relative risk (≤0.83 or ≥1.20) in the absence of multiplicity issues (strongest)
• Consistency:* finding of association needs to be replicated in other studies
• Temporality: to infer causality, exposure must appropriately precede outcome
• Coherence: makes cause-effect relationship should not conflict with natural history/biologic facts
Mente et al, Arch Intern Med 2009
BRADFORD HILL CRITERIA FOR CAUSATION
*STATISTICS PLAYS A ROLE HERE
www.metcardio.org
• Biological gradient:* dose of exposure and risk of disease are positively (or negatively) related
• Experiment: experimental evidence from laboratory studies (weak) or randomized clinical trials (strongest)
• Specificity: specific exposure is related to 1 disease only (NA for multifactorial diseases)
• Plausibility: makes biological/clinical sense (weak)• Analogy: arguing on analogical reasoning (weakest)
BRADFORD HILL CRITERIA FOR CAUSATION
Mente et al, Arch Intern Med 2009
*STATISTICS PLAYS A ROLE HERE
www.metcardio.org
LEARNING GOALS
• What are the goals of biostatistics in clinical research?• Is there a hierarchy in
biostatistics?• When is biostatistics not to be
trusted?
www.metcardio.org
EBM HIERARCHY OF EVIDENCE1. N of 1 randomized controlled trial
2. Systematic reviews of homogeneous randomized trials
3. Single randomized trial
4. Systematic review of observational studies addressing patient-important outcomes
5. Single observational study addressing patient-important outcomes
6. Physiologic studies (eg blood pressure, cardiac output, exercise capacity, bone density, and so forth)
7. Unsystematic clinical observations
Guyatt and Rennie, Users’ guide to the medical literature, 2002
www.metcardio.org
PARALLEL HIERARCHY OF RESEARCH
Biondi-Zoccai et al, Ital Heart J 2003
Qualitative reviews
Systematic reviews
Meta-analyses from individual studies
Meta-analyses from individual patient data
Case reports and series
Observational studies
Observational controlled studies
Single center randomized controlled trials
Multicenter randomized controlled trials
MORE FLEXIBLE BUT LESS VALIDMORE FLEXIBLE BUT LESS VALID
LESS FLEXIBLE BUT MORE VALIDLESS FLEXIBLE BUT MORE VALID
www.metcardio.org
IS A RANDOMIZED TRIAL ALWAYS NEEDED?
Smith et al, BMJ 2003
WE WERE UNABLE TO IDENTIFY ANY RCT OF PARACHUTE INTERVENTION FOR GRAVITATIONAL CHALLENGE… WE THINK THAT EVERYONE MIGHT BENEFIT IF THE MOST RADICAL PROTAGONISTS OF EBM ORGANIZED AND PARTICIPATED IN A DOUBLE BLIND, RANDOMIZED, PLACEBO CONTROLLED TRIAL OF PARACHUTE INTERVENTION
www.metcardio.org
LEARNING GOALS
• What are the goals of biostatistics in clinical research?• Is there a hierarchy in
biostatistics?• When is biostatistics not to be
trusted?
www.metcardio.org
THE INAPPROPRIATE USE OF A FANCY STATISTICAL TOOL
Cepeda et al, Am J Epidemiol 2003
PROPENSITY SCORES ARE USELESS IN LARGE STUDIES WITH AN ADEQUATE (>7) NUMBER OF EVENTS PER COVARIATE
www.metcardio.org
THE WEAK STATISTICAL EVIDENCE
Hannan et al, Circulation 2006
DO YOU TRUST A CONFIDENCE INTERVAL REACHING 1.01 (OR A P=0.049)?
www.metcardio.org
THE LARGE CONFIDENCE INTERVAL
Patti et al, Circulation 2005
CONFIDENCE INTERVAL SPANNING FROM 0.15 TO 0.97
www.metcardio.org
THE UNREASONABLY COMPLEX METHOD
Huyhn et al, Circulation 2009
?
www.metcardio.org
THE UNREASONABLY COMPLEX METHOD
Huyhn et al, Circulation 2009
…WE SELECTED NONINFORMATIVE PRIOR DISTRIBUTIONS… THESE INCLUDED NORMAL DENSITIES (MEAN, 0; 0.00001 [VARIANCE OF 105]) FOR THE LOGARITHM OF THE ORS AND (UNIFORM ON THE INTERVAL [0,2]). TO ENSURE CONVERGENCE OF THE GIBBS SAMPLER ALGORITHM, 3 MARKOV MONTE CARLO CHAINS WERE RUN, AND CONVERGENCE WAS ASSESSED AFTER 60000 ITERATIONS. THE FINAL STATISTICS WERE BASED ON 120000 ITERATIONS, 100000 OF THEM FOR BURN-IN.
??
www.metcardio.org
TAKE HOME MESSAGES
www.metcardio.org
TAKE HOME MESSAGES• Biostatistics is merely necessary, but not sufficient
alone to infer causality• Biostatistical comparisons can really be trusted only if
stemming from one or more randomized trials (e.g. in a meta-analysis)
• Sophisticated statistics cannot remedy faulty study designs, fabricated or missing data
• Sophisticated statistics can cloud a manuscript weaknesses, ominously separating clinicians from decision-making
www.metcardio.org
Thank you for your attention
For any correspondence: [email protected]
For these and further slides on these topics feel free to visit the metcardio.org website:http://www.metcardio.org/slides.html