vsm chapter 6: harm evidence-based medicine how to practice and teach emb
TRANSCRIPT
VSM
CHAPTER 6: HARMCHAPTER 6: HARM
Evidence-Based MedicineEvidence-Based Medicine
How to Practice and Teach How to Practice and Teach EMBEMB
VSM
HarmHarmConcerns about potentially harmful Concerns about potentially harmful interventions occur dailyinterventions occur dailyWe must make judgments about We must make judgments about whether our patients may be at riskwhether our patients may be at riskEvaluate evidence about causationEvaluate evidence about causation– ValidityValidity– ImportanceImportance– Relevance to our patientsRelevance to our patients
Goal is to avoid false + and false -Goal is to avoid false + and false -
VSM
Types of Studies on Harm/EtiologyTypes of Studies on Harm/Etiology
Systematic reviewsSystematic reviews– Best evidence about effects of therapyBest evidence about effects of therapy
RCTRCT– Seldom large enough to detect rare Seldom large enough to detect rare
adverse events adverse events – Ill suited in size, duration and ethicsIll suited in size, duration and ethics
Cohort, case-control studies, and Cohort, case-control studies, and cross-sectional studies cross-sectional studies
VSM
Is this evidence about harm valid?Is this evidence about harm valid?1.1. Were there clearly defined groups of Were there clearly defined groups of
patients, similar in all important ways patients, similar in all important ways other than exposure to treatment or other than exposure to treatment or other cause?other cause?
2.2. Were treatments/exposures and clinical Were treatments/exposures and clinical outcomes measured in the same ways in outcomes measured in the same ways in both groups?both groups?
3.3. Was the follow-up appropriate?Was the follow-up appropriate?4.4. Do the results of the harm study fulfill Do the results of the harm study fulfill
some of the diagnostic tests for some of the diagnostic tests for causation?causation?
VSM
Is this evidence about harm valid?Is this evidence about harm valid?1.1. Were there clearly defined groups of Were there clearly defined groups of
patients, similar in all impt ways other patients, similar in all impt ways other than exposure to treatment or cause?than exposure to treatment or cause?
2.2. Were treatments/exposures and clinical Were treatments/exposures and clinical outcomes measured in the same ways in outcomes measured in the same ways in both groups?both groups?
3.3. Was the follow-up long enough?Was the follow-up long enough?4.4. Do the results of the harm study fulfill Do the results of the harm study fulfill
some of the diagnostic tests for some of the diagnostic tests for causation?causation?
VSM
Is it Valid?Is it Valid?BestBestSystematic review or RCTSystematic review or RCT– Randomization would make groups Randomization would make groups
identical for all other causesidentical for all other causes– Ill suited (in size, duration and ethics)Ill suited (in size, duration and ethics)
The validity of the study designs The validity of the study designs used to detect harm is inversely used to detect harm is inversely proportional to their feasibility.proportional to their feasibility.Must often look at other types of Must often look at other types of studiesstudies
VSM
Cohort studyCohort studyObservational studyObservational study
Group of pts who are exposed and group Group of pts who are exposed and group of pts not exposed are followed for of pts not exposed are followed for development of outcomedevelopment of outcome
Not randomized—exposure based on Not randomized—exposure based on patient’s/MD’s preferencespatient’s/MD’s preferences– Yields more confoundersYields more confounders– Multivariable analysis useful, but adjustments Multivariable analysis useful, but adjustments
can only be made for known confounderscan only be made for known confounders– Still may not be useful if outcome rareStill may not be useful if outcome rare
VSM
Case-Control StudiesCase-Control Studies““cases” pts with outcome of interestcases” pts with outcome of interest““controls” those withoutcontrols” those withoutProportion of each groups exposure Proportion of each groups exposure assessed retrospectivelyassessed retrospectivelyStudy design easy for exploring possible Study design easy for exploring possible relationships b/t many exposures and relationships b/t many exposures and outcome of interestoutcome of interest– Even greater potential for confoundersEven greater potential for confounders– If a large number of associations are explored, If a large number of associations are explored,
a statistically significant finding may be due to a statistically significant finding may be due to chance alone.chance alone.
VSM
Cross-Sectional StudiesCross-Sectional Studies
Most common for looking at etiologyMost common for looking at etiology
Look at group with outcome and Look at group with outcome and group without, and compare group without, and compare exposureexposure
Exposures and outcomes measured Exposures and outcomes measured at same timeat same time– which came first?which came first?– Lots of confoundersLots of confounders
VSM
Case Series/Case ReportsCase Series/Case Reports
Look at few pts with adverse Look at few pts with adverse outcome while receiving suspected outcome while receiving suspected treatmenttreatment
Useful in rare and dramatic outcomesUseful in rare and dramatic outcomes
Lack comparison groupsLack comparison groups
Usually only sufficient for hypothesis Usually only sufficient for hypothesis generationgeneration
VSM
Is this evidence about harm valid?Is this evidence about harm valid?1.1. Were there clearly defined groups of Were there clearly defined groups of
patients, similar in all impt ways other patients, similar in all impt ways other than exposure to treatment or cause?than exposure to treatment or cause?
2.2. Were treatments/exposures and clinical Were treatments/exposures and clinical outcomes measured in the same ways in outcomes measured in the same ways in both groups?both groups?
3.3. Was the follow-up long enough?Was the follow-up long enough?4.4. Do the results of the harm study fulfill Do the results of the harm study fulfill
some of the diagnostic tests for some of the diagnostic tests for causation?causation?
VSM
Are outcomes and exposures Are outcomes and exposures measured the same way?measured the same way?
RCT’sRCT’s– Best if exposures and outcomes Best if exposures and outcomes
measured in same way in both groupsmeasured in same way in both groups
Cohort studiesCohort studies– Want outcome assessors blinded to Want outcome assessors blinded to
exposureexposure
Case-control studiesCase-control studies– Want pts and assessors blinded to Want pts and assessors blinded to
outcome and study hypothesisoutcome and study hypothesis
VSM
Is this evidence about harm valid?Is this evidence about harm valid?1.1. Were there clearly defined groups of patients, Were there clearly defined groups of patients,
similar in all impt ways other than exposure to similar in all impt ways other than exposure to treatment or cause?treatment or cause?
2.2. Were treatments/exposures and clinical Were treatments/exposures and clinical outcomes measured in the same ways in both outcomes measured in the same ways in both groups?groups?
3.3. Was the follow-up appropriate?Was the follow-up appropriate?4.4. Do the results of the harm study fulfill some of Do the results of the harm study fulfill some of
the diagnostic tests for causation?the diagnostic tests for causation?1.1. Is it clear the exposure preceded the onset of the Is it clear the exposure preceded the onset of the
outcome?outcome?2.2. Is there a dose-response gradient?Is there a dose-response gradient?3.3. Is the association consistent from study to study?Is the association consistent from study to study?4.4. Does the association make biological sense?Does the association make biological sense?
VSM
Was follow-up appropriate?Was follow-up appropriate?
CompleteComplete– Ideally want no pts lost to follow-upIdeally want no pts lost to follow-up– 20% loss to follow-up cut off20% loss to follow-up cut off
Long enoughLong enough– Want sufficient length of time to ensure Want sufficient length of time to ensure
all outcomes includedall outcomes included
VSM
Is this evidence about harm valid?Is this evidence about harm valid?1.1. Were there clearly defined groups of Were there clearly defined groups of
patients, similar in all impt ways other patients, similar in all impt ways other than exposure to treatment or cause?than exposure to treatment or cause?
2.2. Were treatments/exposures and clinical Were treatments/exposures and clinical outcomes measured in the same ways in outcomes measured in the same ways in both groups?both groups?
3.3. Was the follow-up long enough?Was the follow-up long enough?4.4. Do the results of the harm study fulfill Do the results of the harm study fulfill
some of the diagnostic tests for some of the diagnostic tests for causation?causation?
VSM
Do the results of the harm study fulfill some Do the results of the harm study fulfill some of the diagnostic tests for causation?of the diagnostic tests for causation?
1.1. Is it clear the exposure preceded the Is it clear the exposure preceded the onset of the outcome?onset of the outcome?
2.2. Is there a dose-response gradient?Is there a dose-response gradient?
3.3. Is the association consistent from Is the association consistent from study to study?study to study?
4.4. Does the association make biological Does the association make biological sense?sense?
VSM
Are the results important?Are the results important?
If study passes validity test, we must If study passes validity test, we must decide if association is strong enough decide if association is strong enough for us to act uponfor us to act upon
Magnitude of associationMagnitude of association– Use relative risk or odds ratioUse relative risk or odds ratio
Precision of estimate of associationPrecision of estimate of association– Use confidence interval Use confidence interval
VSM
Relative RiskRelative Risk
RCT and cohort studiesRCT and cohort studies
Relative Risk (RR)Relative Risk (RR)
RR= proportion of those with RR= proportion of those with outcome in exposed and not exposed outcome in exposed and not exposed groupsgroups
VSM
Relative RiskRelative RiskExample:Example:Exposure groupExposure group– 1000 pts receive Tx1000 pts receive Tx– 20 develop outcome20 develop outcome– a/(a+b)a/(a+b)– 20/1000= 2%20/1000= 2%
Not exposed groupNot exposed group– 1000 pts no Tx1000 pts no Tx– 2 experience outcome2 experience outcome– c/(c+d)c/(c+d)– 2/1000=0.2%2/1000=0.2%
RR=(a/(a+b))/(c/c+d)RR=(a/(a+b))/(c/c+d)
RR= 2%/0.2%=10RR= 2%/0.2%=10
PresentPresent
casecaseAbsentAbsent
controlscontrols
exposedexposed aa
2020BB
980980a+ba+b
10001000
Not Not exposedexposed
CC
22DD
998998c+dc+d
10001000
Adverse outcome
Patients receiving treatment are 10x more likely to experience the outcome.
VSM
Odds RatioOdds Ratio
Case-control studiesCase-control studies – Can’t calculate RR in studies because Can’t calculate RR in studies because
the investigator selects the people with the investigator selects the people with the outcomes (rather than the exposure)the outcomes (rather than the exposure)
– Can’t calculate “incidence”Can’t calculate “incidence”– Use “odds ratio” or relative oddsUse “odds ratio” or relative odds
OR=ad/bcOR=ad/bc
VSM
Odds Ratio Odds Ratio
Ie: urge incontinenceIe: urge incontinence– 100pt with urge incontinence100pt with urge incontinence
90 with hx of caffeine exp90 with hx of caffeine exp
10 without caffeine exp10 without caffeine exp
– 100pts without urge incontinence100pts without urge incontinence45 with exposure to caffeine45 with exposure to caffeine
55 without caffeine use55 without caffeine use
OR=ad/bcOR=ad/bc– OR=(90x55)/(45x10)OR=(90x55)/(45x10)– OR=11OR=11
casecase controlscontrols
exposedexposed aa
9090BB
4545
Not Not exposedexposed
CC
1010DD
5555
Odds of experiencing urge incontinence for people with caffeine use is 11 times that of those who did not use caffeine
VSM
OR and RROR and RROR and RR >1OR and RR >1– Increased risk of adverse eventIncreased risk of adverse event
OR and RR=1OR and RR=1– Adverse event is no more likely to occurAdverse event is no more likely to occur
OR and RR <1OR and RR <1– Adverse event is less likely to occur Adverse event is less likely to occur
OR and RR approximate each other OR and RR approximate each other when event rate and treatment when event rate and treatment effect are smalleffect are small
VSM
How big should the RR or OR be How big should the RR or OR be for us to be impressed?for us to be impressed?
Strength of study design/validity Strength of study design/validity determinedetermineProfessor IrwigProfessor Irwig– Suggests we compare the unadjusted measure Suggests we compare the unadjusted measure
of association with one in which at least one of association with one in which at least one known confounder has been adjusted outknown confounder has been adjusted out
– If adjustment produces a large decline in RR or If adjustment produces a large decline in RR or OR, be suspicious of spurious associationOR, be suspicious of spurious association
– If adjustment yields stable OR or RR, or if it If adjustment yields stable OR or RR, or if it rises, our confidence in the validity should be rises, our confidence in the validity should be greatergreater
VSM
Number Needed to Harm (NNH)Number Needed to Harm (NNH)
OR and RR tell about strength of OR and RR tell about strength of associationassociation
NNH=number of pts who need to be NNH=number of pts who need to be exposed to the causal agent to exposed to the causal agent to produce one additional harmful eventproduce one additional harmful event
VSM
RCT and Cohort StudiesRCT and Cohort Studies
Can be calculated directly from trials Can be calculated directly from trials and cohort studies and cohort studies
Reciprocal of difference in adverse Reciprocal of difference in adverse eventsevents
NNH= 1/(a/(a+b))-(c/(c+d))NNH= 1/(a/(a+b))-(c/(c+d))
VSM
Case Control StudiesCase Control StudiesCan’t determine incidenceCan’t determine incidence
ComplexComplex
If OR<1If OR<1– 1-(PEER(1-OR))/PEER(1-PEER)(1-OR)1-(PEER(1-OR))/PEER(1-PEER)(1-OR)
If OR>1If OR>1– 1+(PEER(OR-1))/PEER(1-PEER)(OR-1)1+(PEER(OR-1))/PEER(1-PEER)(OR-1)
PEER= patient expected event ratePEER= patient expected event rate
Different PEERs with same OR can lead to Different PEERs with same OR can lead to different NNHdifferent NNH
VSM
Confidence intervalConfidence intervalIn addition to magnitude of OR or RR In addition to magnitude of OR or RR must look at its precisionmust look at its precision
Credibility is highest when entire CI is Credibility is highest when entire CI is narrow and remains within a narrow and remains within a clinically importantly increased riskclinically importantly increased risk
VSM
Can this valid and important evidence about Can this valid and important evidence about harm be applied to our patient?harm be applied to our patient?
1.1. Is our patient so different from Is our patient so different from those included in the study that its those included in the study that its results cannot apply?results cannot apply?
2.2. What is our patient’s risk of benefit What is our patient’s risk of benefit and harm from the agent?and harm from the agent?
3.3. What are our patient’s preferences, What are our patient’s preferences, concerns, and expectations?concerns, and expectations?
4.4. What alternatives are available?What alternatives are available?