charting a middle road between the ebm movement, and russo- williamson jeremy howick centre for...
TRANSCRIPT
Charting a middle road between the EBM movement, and Russo-
Williamson
Jeremy HowickCentre for Evidence-Based Medicine
University of Oxford
Take home message
1 The EBM movement are wrong to deny mechanistic reasoning any role in supporting efficacy (they have always recognized a role for mechanisms in generalizing).
2 Russo and Williamson are incorrect that mechanistic reasoning is required to support the hypotheses about efficacy.
3 The middle road: high-quality mechanistic reasoning can add evidential weight.
Comparative clinical studies versus mechanistic reasoning
Black BoxBlack BoxI
Comparative clinical research
O
Mechanistic reasoning
Reduce risk
Reduce risk
Serious arrhythmia
Serious arrhythmia
Sudden DeathSudden DeathAnti-arrhythmic drugs
Anti-arrhythmic drugs
Reduce risk Sudden DeathSudden DeathAnti-arrhythmic
drugsAnti-arrhythmic
drugsTherefore
The EBM movement on mechanisms
“[EBM] de-emphasizes clinical expertise and pathophysiologic rationale as sufficient grounds for clinical decision making and stresses the examination of evidence from clinical research”
Level Therapy
1a SR of RCTs
1b Individual RCT
2a SR of cohort studies
2b Individual cohort study
3a SR case-control studiIes
3b Case-Control Study
4 Case-series
5 Expert opinion without explicit critical appraisal, or based on physiology, bench research or "first principles"
Russo and Williamson argue that mechanisms are necessary
“To establish causal claims, scientists need the mutual support of mechanisms and dependencies”
Misplaced rationale for the Russo Williamson claim: The Semmelweis
case
Sodium channels in the heartSodium channels in the heart
Block
Risk
Heart’s action potentialHeart’s action potential
Block
Electric impulseElectric impulse
Slows
Ventricular extra beats (VEBs)Ventricular extra beats (VEBs)
Risk
Risk
Ventricular fibrillationVentricular fibrillation
Sudden deathSudden death
Anti-arrhythmic drugsAnti-arrhythmic drugs
Mechanism for anti-arrhythmic drugs reduce the risk of sudden
death
Risk
More fine-grained mechanism for anti-arrhythmic drugs reduce the
risk of sudden death
Ventricular extra beats (VEBs)Ventricular extra beats (VEBs)
Anti-arrhythmic drugsAnti-arrhythmic drugs
Other examples where mechanistic reasoning led us astray
• Anti-arrhythmic drugs to prevent sudden cardiac death.• Babies to sleep on their front to reduce risk of SIDS• HGH for hypercatabolism• Oxygen for organ failure• ERT for menopausal symptoms• Radical mastectomy for breast cancer• Rest for recovery• Early screening for breast cancer• …
GoiterGoiter ObstructionObstruction Respiratory function
Respiratory function
RadiotherapyRadiotherapy
induces impairs
GoiterGoitershrinks
Evidence (from separate studies)
ObstructionObstruction Respiratory function
Respiratory functionRadiotherapyRadiotherapy
induces impairsGoiterGoiter
shrinks
Mechanistic reasoning
Mechanistic reasoning linking radiotherapy with improved respiratory function
High-quality mechanistic reasoning
1. All the links in the mechanism’s chain must be established as causal.
Reduce risk of by
90%
Reduce risk of by 25%-50%
Serious arrhythmia
Serious arrhythmia
Sudden DeathSudden DeathAnti-arrhythmic drugs
Anti-arrhythmic drugs
Stochastic nature of mechanisms
Sudden DeathSudden DeathAnti-arrhythmic drugs
Anti-arrhythmic drugs Strength???
Paradoxical effects
I2I2
II
c3c3
OOcc
z3z3
zzaa
b3b3
bb
a3a3
c2c2 z2z2b2b2a2a2
¬O¬O
Reduce risk of by
90%
Serious arrhythmia
Serious arrhythmia
Anti-arrhythmic drugs
Anti-arrhythmic drugs
Increase risk of by
10%
Serious arrhythmia
Serious arrhythmia
High-quality mechanistic reasoning
• All the links in the mechanism’s chain must be established as causal.
• The complexity and stochastic nature of mechanisms must be considered before drawing conclusions of the overall effect of the intervention on the clinically relevant outcome.
Why the 2nd criterion is difficult to meet: the unknowns in
biochemical mechanismsPathways shown in blue
represent the potentially beneficial effects of CETP inhibition, those shown in green may remain relatively unaffected, and those shown with dashed lines have potentially reduced activity after CETP inhibition.
Why the 2nd criterion is unlikely to be met, cont’d.
• “It is still unknown, if methyl oxidation at ring B occurs before or after esterification with phytol”
• “In some microorganisms, cystathionine synthesis takes place via O-acetyl-L-homoserine”
• …
Summary
• Mechanisms are not always required, especially when the comparative clinical studies are well-conducted.
• Mechanisms are not always bad.• High-quality mechanistic reasoning can, and
should, be used as evidence.
Other roles for mechanisms
• Generalizability?• Hypothesis generation?• Explanation?• Convincing people?• The quality criteria apply here too!
Other potential roles for mechanisms: Generalizing the results of studies
• This is a different problem than the one I was talking about.
• Generalizability is a real problem (selected population, rarified circumstances…)
• The same applies to mechanisms (as we saw yesterday in Carl’s talk).
• Waving a wand and saying ‘mechanism’ doesn’t solve the problem.
• We need to trace a causal pathway in the individual, and engage in high-quality mechanistic reasoning.
• Mechanisms are not the only way to generalize…
How to combine mechanistic evidence with comparative clinical
studies (a sketch for Lindsay)• Just as comparative clinical studies must reach a
certain threshold of quality in order to be useful, so must mechanistic reasoning.
• All evidence that reaches its respective quality threshold is sufficient to add evidential weight.
• There is no need to get more specific than that.
How to generalize without mechanisms
• We cannot sample from the target population.
• But no need to exclude 60%-90% of potentially eligible participants from trials.
• We can make our exclusion criteria inclusive.• This will increase the variability and hence the
size and cost of the trial.• But this is a cost that is well worth paying.
Role 2: Persuasion
• “The history of medicine presents many cases in which causal claims made solely on the basis of statistics have been rejected until backed by mechanistic … knowledge”
• This is true (Semmelweis, Warren and Marshall).• “It is true that Kuhn’s ideas give a rather
convincing explanation of Semmelweis’s failure, but it is important to remember that such an explanation is not a justification.”
Role 3: Mechanisms for hypothesis generation
• It is true that many of our hypotheses are generated by the basic sciences.
• But are these efficient ways of generating hypotheses?
The exaggerated stability of mechanisms
• “entities and activities organized such that they are productive of regular changes from start or set-up to finish or termination conditions” (MDC)
• “invariant, change-relating generalizations” (Glennan).
• “… the existence of a mechanism provides evidence of the stability of a causal relationship” (Russo and Williamson)
Basic science is not necessarily an efficient way to generate hypotheses
• A study examined 101 major ‘mechanistic’ findings in the top basic science journals
• [INSERT THESE IN TALK]• Science, Nature, Cell, Journal of Experimental Medicine, Journal of
Clinical Investigation, and the Journal of Biological Chemistry between 1979 and 1983.
• By 2002, 27 (25%) of the allegedly promising technologies had been clinically
• 19 (20%) indicated a ‘positive’ benefit• 5 (5%) have been approved for marketing• 1 (1%) has a clinically relevant outcome• There are other ways of generating hypotheses
– Chicken soup for the common cold