superiority, non-inferiority, equivalence studies - what is the difference?
DESCRIPTION
TRANSCRIPT
![Page 1: Superiority, non-inferiority, equivalence studies - what is the difference?](https://reader035.vdocuments.us/reader035/viewer/2022081413/54890b22b47959e70c8b5899/html5/thumbnails/1.jpg)
Superiority-Equivalence-Non-Inferiority Trials: What does it all mean?
Gerald Gartlehner
![Page 2: Superiority, non-inferiority, equivalence studies - what is the difference?](https://reader035.vdocuments.us/reader035/viewer/2022081413/54890b22b47959e70c8b5899/html5/thumbnails/2.jpg)
Is the new treatment better than the established one ?
If not, is it equally effective and preferable for some other reason
What decision-makers want to know
Österreichische Cochrane Zweigstelle (ÖCZ) www.cochrane.at ∙Department für Evidenzbasierte Medizin und Klinische Epidemiologie, Donau-Universität Krems
Gerald Gartlehner
![Page 3: Superiority, non-inferiority, equivalence studies - what is the difference?](https://reader035.vdocuments.us/reader035/viewer/2022081413/54890b22b47959e70c8b5899/html5/thumbnails/3.jpg)
Overview
Österreichische Cochrane Zweigstelle (ÖCZ) www.cochrane.at ∙Department für Evidenzbasierte Medizin und Klinische Epidemiologie, Donau-Universität Krems
Gerald Gartlehner
Definitions and concepts Challenges of non-inferiority trials What does this mean for systematic reviews? Language considerations for comparative
effectiveness reviews
![Page 4: Superiority, non-inferiority, equivalence studies - what is the difference?](https://reader035.vdocuments.us/reader035/viewer/2022081413/54890b22b47959e70c8b5899/html5/thumbnails/4.jpg)
Proving Efficacy
1. Showing superiority of one (new) treatment over another (placebo or active)
2. Showing equivalence or non-inferiority of a new intervention relative to an already existing efficacious treatment
Österreichische Cochrane Zweigstelle (ÖCZ) www.cochrane.at ∙Department für Evidenzbasierte Medizin und Klinische Epidemiologie, Donau-Universität Krems
Gerald Gartlehner
![Page 5: Superiority, non-inferiority, equivalence studies - what is the difference?](https://reader035.vdocuments.us/reader035/viewer/2022081413/54890b22b47959e70c8b5899/html5/thumbnails/5.jpg)
The interpretation of non-inferiority and equivalence can be confusing
Österreichische Cochrane Zweigstelle (ÖCZ) www.cochrane.at ∙Department für Evidenzbasierte Medizin und Klinische Epidemiologie, Donau-Universität Krems
Gerald Gartlehner
![Page 6: Superiority, non-inferiority, equivalence studies - what is the difference?](https://reader035.vdocuments.us/reader035/viewer/2022081413/54890b22b47959e70c8b5899/html5/thumbnails/6.jpg)
Ranking position: 72
Ranking position: 34
![Page 7: Superiority, non-inferiority, equivalence studies - what is the difference?](https://reader035.vdocuments.us/reader035/viewer/2022081413/54890b22b47959e70c8b5899/html5/thumbnails/7.jpg)
Österreichische Cochrane Zweigstelle (ÖCZ) www.cochrane.at ∙Department für Evidenzbasierte Medizin und Klinische Epidemiologie, Donau-Universität Krems
Gerald Gartlehner
![Page 8: Superiority, non-inferiority, equivalence studies - what is the difference?](https://reader035.vdocuments.us/reader035/viewer/2022081413/54890b22b47959e70c8b5899/html5/thumbnails/8.jpg)
Superiority game: the winner takes it all….
Österreichische Cochrane Zweigstelle (ÖCZ) www.cochrane.at ∙Department für Evidenzbasierte Medizin und Klinische Epidemiologie, Donau-Universität Krems
Gerald Gartlehner
![Page 9: Superiority, non-inferiority, equivalence studies - what is the difference?](https://reader035.vdocuments.us/reader035/viewer/2022081413/54890b22b47959e70c8b5899/html5/thumbnails/9.jpg)
Equivalence game – equally good or clearly better
Equivalence margin: tied or less than 1 goal difference
Österreichische Cochrane Zweigstelle (ÖCZ) www.cochrane.at ∙Department für Evidenzbasierte Medizin und Klinische Epidemiologie, Donau-Universität Krems
Gerald Gartlehner
![Page 10: Superiority, non-inferiority, equivalence studies - what is the difference?](https://reader035.vdocuments.us/reader035/viewer/2022081413/54890b22b47959e70c8b5899/html5/thumbnails/10.jpg)
Non-inferiority game– at least not substantially worse….
Non-inferiority margin: can‘t lose with more than 1 goal difference
Österreichische Cochrane Zweigstelle (ÖCZ) www.cochrane.at ∙Department für Evidenzbasierte Medizin und Klinische Epidemiologie, Donau-Universität Krems
Gerald Gartlehner
![Page 11: Superiority, non-inferiority, equivalence studies - what is the difference?](https://reader035.vdocuments.us/reader035/viewer/2022081413/54890b22b47959e70c8b5899/html5/thumbnails/11.jpg)
Definitions
Superiority trialObjective: To determine a clinically relevant difference between two interventions
Equivalence trialObjective: To determine whether a (new) intervention is neither worse nor better than another (established) intervention
Non-inferiority trialObjective: To determine whether a (new) intervention is not inferior to another (established) intervention
Österreichische Cochrane Zweigstelle (ÖCZ) www.cochrane.at ∙Department für Evidenzbasierte Medizin und Klinische Epidemiologie, Donau-Universität Krems
Gerald Gartlehner
![Page 12: Superiority, non-inferiority, equivalence studies - what is the difference?](https://reader035.vdocuments.us/reader035/viewer/2022081413/54890b22b47959e70c8b5899/html5/thumbnails/12.jpg)
Equivalence - Non-Inferiority
The naïve approach: If a head-to-head trial shows no statistically significant
difference, two interventions are “equivalent” Problem – underpowered studies or high variance will
create “equivalent” treatments
The statistical approach: Define a margin of non-inferiority or equivalence If 95% confidence interval of the difference DOES NOT
cross the margin, the new intervention is non-inferior or equivalent
Österreichische Cochrane Zweigstelle (ÖCZ) www.cochrane.at ∙Department für Evidenzbasierte Medizin und Klinische Epidemiologie, Donau-Universität Krems
Gerald Gartlehner
![Page 13: Superiority, non-inferiority, equivalence studies - what is the difference?](https://reader035.vdocuments.us/reader035/viewer/2022081413/54890b22b47959e70c8b5899/html5/thumbnails/13.jpg)
01020
Arthroscopy vs. sham arthoscopy in patients with knee osteoarthritis
Mosley et al. New England J Med, 2002;347:81-88
Knee Specific Pain Scale: difference in points
10 20
MID
favors arthroscopyfavors sham
Österreichische Cochrane Zweigstelle (ÖCZ) www.cochrane.at ∙Department für Evidenzbasierte Medizin und Klinische Epidemiologie, Donau-Universität Krems
Gerald Gartlehner
![Page 14: Superiority, non-inferiority, equivalence studies - what is the difference?](https://reader035.vdocuments.us/reader035/viewer/2022081413/54890b22b47959e70c8b5899/html5/thumbnails/14.jpg)
Non-inferiority margin (d)
The limit of acceptable inferiority:Minimal important differenceClinical judgementStatistical considerations
Österreichische Cochrane Zweigstelle (ÖCZ) www.cochrane.at ∙Department für Evidenzbasierte Medizin und Klinische Epidemiologie, Donau-Universität Krems
Gerald Gartlehner
![Page 15: Superiority, non-inferiority, equivalence studies - what is the difference?](https://reader035.vdocuments.us/reader035/viewer/2022081413/54890b22b47959e70c8b5899/html5/thumbnails/15.jpg)
012
Standard vs. placebo2 points (1.5 to 2.5)
Maximimum non-inferiority margin: dmax = 1.5Fractional preservation of treatment effect: f = 0.5d = dmax x (1 – f)d = 1.5 x ( 1 – 0.5) = 0.75
New vs. standard
Determining the non-inferiority margin statistically
![Page 16: Superiority, non-inferiority, equivalence studies - what is the difference?](https://reader035.vdocuments.us/reader035/viewer/2022081413/54890b22b47959e70c8b5899/html5/thumbnails/16.jpg)
Peculiar issues of non-inferiority trials: the backwardness
Null hypothesis and alternative hypothesis are reversed
Type I and type II errors are reversedPer-protocol analyses can be more important than
ITT analysis (ITT analyses are biased towards finding no difference)
P-value is one-sided (0.025)
![Page 17: Superiority, non-inferiority, equivalence studies - what is the difference?](https://reader035.vdocuments.us/reader035/viewer/2022081413/54890b22b47959e70c8b5899/html5/thumbnails/17.jpg)
Assay Sensitivity and Constancy Assumption
The ability of a trial to distinguish effective from ineffective treatments (depends on the effect size the trial wants to detect).
Österreichische Cochrane Zweigstelle (ÖCZ) www.cochrane.at ∙Department für Evidenzbasierte Medizin und Klinische Epidemiologie, Donau-Universität Krems
Gerald Gartlehner
![Page 18: Superiority, non-inferiority, equivalence studies - what is the difference?](https://reader035.vdocuments.us/reader035/viewer/2022081413/54890b22b47959e70c8b5899/html5/thumbnails/18.jpg)
Assay Sensitivity
Assumptions in non-inferiority trials:
The efficacy of the active control was preserved in the non-inferiority study (i.e. that it had assay sensitivity).
If it was not, equivalence or non-inferiority conclusions are meaningless (The non-inferior drug could have no effect at all).
![Page 19: Superiority, non-inferiority, equivalence studies - what is the difference?](https://reader035.vdocuments.us/reader035/viewer/2022081413/54890b22b47959e70c8b5899/html5/thumbnails/19.jpg)
Constancy Assumption
Active comparator must be well established and have predictable and consistent treatment effects
Participants must be similar to those in trial establishing efficacy
Outcomes must be similar to those in trials establishing efficacy
![Page 20: Superiority, non-inferiority, equivalence studies - what is the difference?](https://reader035.vdocuments.us/reader035/viewer/2022081413/54890b22b47959e70c8b5899/html5/thumbnails/20.jpg)
012
Biocreep
Effective treatment
Still clinically relevant?
Österreichische Cochrane Zweigstelle (ÖCZ) www.cochrane.at ∙Department für Evidenzbasierte Medizin und Klinische Epidemiologie, Donau-Universität Krems
Gerald Gartlehner
![Page 21: Superiority, non-inferiority, equivalence studies - what is the difference?](https://reader035.vdocuments.us/reader035/viewer/2022081413/54890b22b47959e70c8b5899/html5/thumbnails/21.jpg)
Key Points for critical appraisal
Was a non-inferiority margin defined based on clinical considerations and statistical reasoning?
Was it established a priori?Was the study powered based on the non-inferiority
margin?Was an ITT and a per-protocol analysis conducted?Was the trial design (e.g. eligibility criteria) consistent
with placebo controlled trials of the established treatment?
![Page 22: Superiority, non-inferiority, equivalence studies - what is the difference?](https://reader035.vdocuments.us/reader035/viewer/2022081413/54890b22b47959e70c8b5899/html5/thumbnails/22.jpg)
What does this mean for systematic reviews ?
For meta-analyses – data can be used just as from any superiority trial
For qualitative assessments – language considerations
Österreichische Cochrane Zweigstelle (ÖCZ) www.cochrane.at ∙Department für Evidenzbasierte Medizin und Klinische Epidemiologie, Donau-Universität Krems
Gerald Gartlehner
![Page 23: Superiority, non-inferiority, equivalence studies - what is the difference?](https://reader035.vdocuments.us/reader035/viewer/2022081413/54890b22b47959e70c8b5899/html5/thumbnails/23.jpg)
Language considerations
“..there was no statistically significant difference between A and B..”
“..studies failed to show a difference..”Can mean:1) The evidence shows equivalence2) The evidence is inconclusive (because confidence
intervals are wide-lack of precision)
AHRQ guidance: Assessing equivalence and non-inferiority [draft]
Österreichische Cochrane Zweigstelle (ÖCZ) www.cochrane.at ∙Department für Evidenzbasierte Medizin und Klinische Epidemiologie, Donau-Universität Krems
Gerald Gartlehner
![Page 24: Superiority, non-inferiority, equivalence studies - what is the difference?](https://reader035.vdocuments.us/reader035/viewer/2022081413/54890b22b47959e70c8b5899/html5/thumbnails/24.jpg)
Language considerations
Better:“..treatments A and B had similar mortality rates..”
Expressing non-inferiority:
“…treatment A is at least as effective as treatment B for [Outcome or study objective]…”
AHRQ guidance: Assessing equivalence and non-inferiority [draft]Österreichische Cochrane Zweigstelle (ÖCZ) www.cochrane.at ∙Department für Evidenzbasierte Medizin und Klinische Epidemiologie, Donau-Universität Krems
Gerald Gartlehner
![Page 25: Superiority, non-inferiority, equivalence studies - what is the difference?](https://reader035.vdocuments.us/reader035/viewer/2022081413/54890b22b47959e70c8b5899/html5/thumbnails/25.jpg)
Do we need to establish equivalence margins for CERs ?