cinema webinar complete -...
TRANSCRIPT
![Page 1: CINeMA webinar complete - Cochranetraining.cochrane.org/sites/training.cochrane.org/files/public/uploads... · The most critical question raised by patients and clinicians at the](https://reader036.vdocuments.us/reader036/viewer/2022090606/605b02eb298cbb78c31f8c4d/html5/thumbnails/1.jpg)
Georgia Salanti & Theodore PapakonstantinouInstitute of Social and Preventive MedicineUniversity of BernSwitzerland
CINeMA
![Page 2: CINeMA webinar complete - Cochranetraining.cochrane.org/sites/training.cochrane.org/files/public/uploads... · The most critical question raised by patients and clinicians at the](https://reader036.vdocuments.us/reader036/viewer/2022090606/605b02eb298cbb78c31f8c4d/html5/thumbnails/2.jpg)
The most critical question raised by patients and clinicians at the point of care is
“what is the drug of choice for the given condition?”
Del Fiol G et al. Clinical questions raised by clinicians at the point of care: a systematic review. JAMA Intern Med. 2014
![Page 3: CINeMA webinar complete - Cochranetraining.cochrane.org/sites/training.cochrane.org/files/public/uploads... · The most critical question raised by patients and clinicians at the](https://reader036.vdocuments.us/reader036/viewer/2022090606/605b02eb298cbb78c31f8c4d/html5/thumbnails/3.jpg)
`
`
ZOT
ZIP
SER
RIS
QUE
PBOPAL
OLA
LURA
ILO
HAL
CPZ
CLO
ASE
ARIAMI
Leucht S et al. Comparative efficacy and tolerability of 15 antipsychotic drugs in schizophrenia. Lancet 2013
![Page 4: CINeMA webinar complete - Cochranetraining.cochrane.org/sites/training.cochrane.org/files/public/uploads... · The most critical question raised by patients and clinicians at the](https://reader036.vdocuments.us/reader036/viewer/2022090606/605b02eb298cbb78c31f8c4d/html5/thumbnails/4.jpg)
AMI
RIS
Indirecttreatment effectDirecttreatment effect
AMI
OLAOLA
![Page 5: CINeMA webinar complete - Cochranetraining.cochrane.org/sites/training.cochrane.org/files/public/uploads... · The most critical question raised by patients and clinicians at the](https://reader036.vdocuments.us/reader036/viewer/2022090606/605b02eb298cbb78c31f8c4d/html5/thumbnails/5.jpg)
AMI
RIS
Indirecttreatment effectDirecttreatment effect
OLA
Network or Mixedtreatment effect
![Page 6: CINeMA webinar complete - Cochranetraining.cochrane.org/sites/training.cochrane.org/files/public/uploads... · The most critical question raised by patients and clinicians at the](https://reader036.vdocuments.us/reader036/viewer/2022090606/605b02eb298cbb78c31f8c4d/html5/thumbnails/6.jpg)
`
`
ZOT
ZIP
SER
RIS
QUE
PBOPAL
OLA
LURA
ILO
HAL
CPZ
CLO
ASE
ARIAMI
Leucht S et al. Comparative efficacy and tolerability of 15 antipsychotic drugs in schizophrenia. Lancet 2013
![Page 7: CINeMA webinar complete - Cochranetraining.cochrane.org/sites/training.cochrane.org/files/public/uploads... · The most critical question raised by patients and clinicians at the](https://reader036.vdocuments.us/reader036/viewer/2022090606/605b02eb298cbb78c31f8c4d/html5/thumbnails/7.jpg)
1997 1999 2000 2002 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015
Year of publication
Num
ber
of p
ublis
hed
net
wor
ks o
f int
erve
ntio
ns
020
4060
8010
012
0
456 published networks in the medical literature comparing at least 4 medical interventions (March 2015)
(Petropoulou et al. Journal of Clinical Epidemiology 2016, Zarin et al. BMC Medicine 2016)
![Page 8: CINeMA webinar complete - Cochranetraining.cochrane.org/sites/training.cochrane.org/files/public/uploads... · The most critical question raised by patients and clinicians at the](https://reader036.vdocuments.us/reader036/viewer/2022090606/605b02eb298cbb78c31f8c4d/html5/thumbnails/8.jpg)
None of the 456 NMAs published until 3/2015 attempted to evaluate the confidence in NMA results!
![Page 9: CINeMA webinar complete - Cochranetraining.cochrane.org/sites/training.cochrane.org/files/public/uploads... · The most critical question raised by patients and clinicians at the](https://reader036.vdocuments.us/reader036/viewer/2022090606/605b02eb298cbb78c31f8c4d/html5/thumbnails/9.jpg)
Study limitationsIndirectnessInconsistency (heterogeneity, incoherence)ImprecisionPublication bias
Consider the network estimates
CINeMA framework
Rate each network estimate No concernsSome concernsMajor concerns
Networkestimate
Study limitations
Indirectness Inconsistency Imprecision Publication bias
Confidence
Heterogeneity Incoherence
A vs B Some concerns
Some concerns
Major concerns
Some concerns
Some concerns
undetected Very low
A vs C No concerns
No concerns No concerns Major concerns
No concerns suspected Low
….
![Page 10: CINeMA webinar complete - Cochranetraining.cochrane.org/sites/training.cochrane.org/files/public/uploads... · The most critical question raised by patients and clinicians at the](https://reader036.vdocuments.us/reader036/viewer/2022090606/605b02eb298cbb78c31f8c4d/html5/thumbnails/10.jpg)
Methods developed by:Georgia SalantiJulian HigginsAdriani Nikolakopoulou
Web developer:Theodore Papakonstantinou
Project supervision:Matthias Egger
![Page 11: CINeMA webinar complete - Cochranetraining.cochrane.org/sites/training.cochrane.org/files/public/uploads... · The most critical question raised by patients and clinicians at the](https://reader036.vdocuments.us/reader036/viewer/2022090606/605b02eb298cbb78c31f8c4d/html5/thumbnails/11.jpg)
Number of studies 22
Number of treatment nodes 6
Primary outcome Effect of antihypertensives on incidence diabetes mellitus -proportion of patients who developed diabetes
Measurement Binary
Intervention comparison type pharmacological vs placebo
![Page 12: CINeMA webinar complete - Cochranetraining.cochrane.org/sites/training.cochrane.org/files/public/uploads... · The most critical question raised by patients and clinicians at the](https://reader036.vdocuments.us/reader036/viewer/2022090606/605b02eb298cbb78c31f8c4d/html5/thumbnails/12.jpg)
![Page 13: CINeMA webinar complete - Cochranetraining.cochrane.org/sites/training.cochrane.org/files/public/uploads... · The most critical question raised by patients and clinicians at the](https://reader036.vdocuments.us/reader036/viewer/2022090606/605b02eb298cbb78c31f8c4d/html5/thumbnails/13.jpg)
Semi-automated process
Explicit rules that classify each network meta-analysis effect for each domain to
No concerns, Some concerns, Major concernsas described in the documentation
The rules can be overwritten!
CONFIDENCE
MODERATE
MODERATE
MODERATE
LOW
MODERATE
LOW
MODERATE
VERYLOW
MODERATE
MODERATE
VERYLOW
MODERATE
LOW
LOW
LOW
![Page 14: CINeMA webinar complete - Cochranetraining.cochrane.org/sites/training.cochrane.org/files/public/uploads... · The most critical question raised by patients and clinicians at the](https://reader036.vdocuments.us/reader036/viewer/2022090606/605b02eb298cbb78c31f8c4d/html5/thumbnails/14.jpg)
CINeMA
The aim of the webinar is to explain the methods used in
CINeMA and present an alpha version of the web application
pollev.com/gmhbe
![Page 15: CINeMA webinar complete - Cochranetraining.cochrane.org/sites/training.cochrane.org/files/public/uploads... · The most critical question raised by patients and clinicians at the](https://reader036.vdocuments.us/reader036/viewer/2022090606/605b02eb298cbb78c31f8c4d/html5/thumbnails/15.jpg)
q Major concernsq Some concernsq No concerns
![Page 16: CINeMA webinar complete - Cochranetraining.cochrane.org/sites/training.cochrane.org/files/public/uploads... · The most critical question raised by patients and clinicians at the](https://reader036.vdocuments.us/reader036/viewer/2022090606/605b02eb298cbb78c31f8c4d/html5/thumbnails/16.jpg)
Study name Risk of BiasAASK LOW
ALLHAT LOWALPINE LOWANBP-2 LOWASCOT LOWCAPPP MODERATECHARM LOWDREAM LOWEWPHE MODERATEFEVER LOW
HAPPHY HIGHHOPE LOW
INSIGHT LOWINVEST LOW
LIFE LOWMRC LOW
NORDIL LOWPEACE LOWSCOPE MODERATESHEP LOW
STOP-2 MODERATEVALUE MODERATE
Form risk of bias judgements for each study. Consider selection, performance, attrition, detection and reporting bias
CCB vs Diuretics:overall low risk of bias
Plot direct comparisonin green
![Page 17: CINeMA webinar complete - Cochranetraining.cochrane.org/sites/training.cochrane.org/files/public/uploads... · The most critical question raised by patients and clinicians at the](https://reader036.vdocuments.us/reader036/viewer/2022090606/605b02eb298cbb78c31f8c4d/html5/thumbnails/17.jpg)
BB vs PlaceboDiureticsCCBACEARB
Diuretics vs BBCCBACEARB
CCB vs DiureticsACEARB
ACE vs CCBARB
ARB vs ACE
Comparison
0.4 0.7 1.5 21
OR from NMA
Favors first
![Page 18: CINeMA webinar complete - Cochranetraining.cochrane.org/sites/training.cochrane.org/files/public/uploads... · The most critical question raised by patients and clinicians at the](https://reader036.vdocuments.us/reader036/viewer/2022090606/605b02eb298cbb78c31f8c4d/html5/thumbnails/18.jpg)
BB vs PlaceboDiureticsCCBACEARB
Diuretics vs BBCCBACEARB
CCB vs DiureticsACEARB
ACE vs CCBARB
ARB vs ACE
Comparison
0.4 0.7 1.5 21
OR from NMA
Favors first
![Page 19: CINeMA webinar complete - Cochranetraining.cochrane.org/sites/training.cochrane.org/files/public/uploads... · The most critical question raised by patients and clinicians at the](https://reader036.vdocuments.us/reader036/viewer/2022090606/605b02eb298cbb78c31f8c4d/html5/thumbnails/19.jpg)
BB vs PlaceboDiureticsCCBACEARB
CCBACEARB
CCB vs DiureticsACEARB
ACE vs CCBARB
ARB vs ACE
Comparison
0.4 0.7 1.5 21
OR from NMA
What is your judgement about study limitations for this (mixed) OR between CCB vs Diuretics estimated in network meta-analysis?
q Major concernsq Some concernsq No concerns
Go to:
pollev.com/gmhbe
![Page 20: CINeMA webinar complete - Cochranetraining.cochrane.org/sites/training.cochrane.org/files/public/uploads... · The most critical question raised by patients and clinicians at the](https://reader036.vdocuments.us/reader036/viewer/2022090606/605b02eb298cbb78c31f8c4d/html5/thumbnails/20.jpg)
![Page 21: CINeMA webinar complete - Cochranetraining.cochrane.org/sites/training.cochrane.org/files/public/uploads... · The most critical question raised by patients and clinicians at the](https://reader036.vdocuments.us/reader036/viewer/2022090606/605b02eb298cbb78c31f8c4d/html5/thumbnails/21.jpg)
BB vs PlaceboDiureticsCCBACEARB
Diuretics vs BBCCBACEARB
CCB vs DiureticsACEARB
ACE vs CCBARB
ARB vs ACE
0.4 0.7 1.5 21
Studies with high risk of bias contribute to the estimation of the OR CCB vs Diuretics!
!
![Page 22: CINeMA webinar complete - Cochranetraining.cochrane.org/sites/training.cochrane.org/files/public/uploads... · The most critical question raised by patients and clinicians at the](https://reader036.vdocuments.us/reader036/viewer/2022090606/605b02eb298cbb78c31f8c4d/html5/thumbnails/22.jpg)
BB vs PlaceboDiureticsCCBACEARB
Diuretics vs BBCCBACEARB
CCB vs DiureticsACEARB
ACE vs CCBARB
ARB vs ACE
Comparison
0.4 0.7 1.5 21
OR from NMA
What is your judgement about study limitations for this (indirect) OR for ACE vs ARB estimated in NMA?
q Major concernsq Some concernsq No concerns
Favors first
![Page 23: CINeMA webinar complete - Cochranetraining.cochrane.org/sites/training.cochrane.org/files/public/uploads... · The most critical question raised by patients and clinicians at the](https://reader036.vdocuments.us/reader036/viewer/2022090606/605b02eb298cbb78c31f8c4d/html5/thumbnails/23.jpg)
![Page 24: CINeMA webinar complete - Cochranetraining.cochrane.org/sites/training.cochrane.org/files/public/uploads... · The most critical question raised by patients and clinicians at the](https://reader036.vdocuments.us/reader036/viewer/2022090606/605b02eb298cbb78c31f8c4d/html5/thumbnails/24.jpg)
An indirect or mixed treatment effect is a combination of the available direct treatment effects
ACE:BBlocker
ACE:CCB
ACE:Diuretic
ACE:Placebo
ARB:BBlocker
ARB:CCB
ARB:Diuretic
ARB:Placebo
BBlocker:CCB
BBlocker:Diuretic
BBlocker:Placebo
CCB:Diuretic
CCB:Placebo
Diuretic:Placebo
Mixed estimates
ACE:BBlocker 32 10 10 8 6 1 0 4 15 6 2 5 2 0ACE:CCB 10 26 13 11 1 6 0 4 9 1 0 13 6 0ACE:Diuretic 6 7 57 5 0 2 0 2 1 5 0 12 2 2ACE:Placebo 5 7 5 56 3 3 0 6 1 0 2 3 8 2ARB:BBlocker 4 1 0 3 41 21 0 5 19 2 2 2 1 0ARB:CCB 1 2 1 2 8 67 0 6 8 1 0 2 4 0ARB:Diuretic 3 2 11 5 10 27 0 8 0 7 0 25 0 2ARB:Placebo 3 3 2 7 6 15 0 49 0 1 2 2 10 1BBlocker:CCB 6 4 1 1 11 12 0 0 53 4 2 5 2 0BBlocker:Diuretic 10 1 13 2 5 3 0 2 19 20 2 21 0 2BBlocker:Placebo 10 2 2 14 13 3 0 16 16 4 8 1 11 2CCB:Diuretic 2 6 11 3 1 3 0 2 7 6 0 56 3 2CCB:Placebo 2 6 4 12 1 15 0 16 6 0 2 5 28 2Diuretic:Placebo 0 0 20 20 2 7 0 9 0 5 2 17 11 7
Indirect estimatesACE:ARB 10 11 8 16 11 20 0 14 1 1 0 7 2 0
The contribution matrix
![Page 25: CINeMA webinar complete - Cochranetraining.cochrane.org/sites/training.cochrane.org/files/public/uploads... · The most critical question raised by patients and clinicians at the](https://reader036.vdocuments.us/reader036/viewer/2022090606/605b02eb298cbb78c31f8c4d/html5/thumbnails/25.jpg)
An indirect or mixed treatment effect is a combination of the available direct treatment effects
ACE:BBlocker
ACE:CCB
ACE:Diuretic
ACE:Placebo
ARB:BBlocker
ARB:CCB
ARB:Diuretic
ARB:Placebo
BBlocker:CCB
BBlocker:Diuretic
BBlocker:Placebo
CCB:Diuretic
CCB:Placebo
Diuretic:Placebo
Mixed estimates
ACE:BBlocker 32 10 10 8 6 1 0 4 15 6 2 5 2 0ACE:CCB 10 26 13 11 1 6 0 4 9 1 0 13 6 0ACE:Diuretic 6 7 57 5 0 2 0 2 1 5 0 12 2 2ACE:Placebo 5 7 5 56 3 3 0 6 1 0 2 3 8 2ARB:BBlocker 4 1 0 3 41 21 0 5 19 2 2 2 1 0ARB:CCB 1 2 1 2 8 67 0 6 8 1 0 2 4 0ARB:Diuretic 3 2 11 5 10 27 0 8 0 7 0 25 0 2ARB:Placebo 3 3 2 7 6 15 0 49 0 1 2 2 10 1BBlocker:CCB 6 4 1 1 11 12 0 0 53 4 2 5 2 0BBlocker:Diuretic 10 1 13 2 5 3 0 2 19 20 2 21 0 2BBlocker:Placebo 10 2 2 14 13 3 0 16 16 4 8 1 11 2CCB:Diuretic 2 6 11 3 1 3 0 2 7 6 0 56 3 2CCB:Placebo 2 6 4 12 1 15 0 16 6 0 2 5 28 2Diuretic:Placebo 0 0 20 20 2 7 0 9 0 5 2 17 11 7
Indirect estimatesACE:ARB 10 11 8 16 11 20 0 14 1 1 0 7 2 0
The contribution matrix
![Page 26: CINeMA webinar complete - Cochranetraining.cochrane.org/sites/training.cochrane.org/files/public/uploads... · The most critical question raised by patients and clinicians at the](https://reader036.vdocuments.us/reader036/viewer/2022090606/605b02eb298cbb78c31f8c4d/html5/thumbnails/26.jpg)
ACE:BBlocker
ACE:CCB
ACE:Diuretic
ACE:Placebo
ARB:BBlocker
ARB:CCB
ARB:Diuretic
ARB:Placebo
BBlocker:CCB
BBlocker:Diuretic
BBlocker:Placebo
CCB:Diuretic
Mixed estimates
ACE:BBlocker 32 10 10 8 6 1 0 4 15 6 2 5ACE:CCB 10 26 13 11 1 6 0 4 9 1 0 13ACE:Diuretic 6 7 57 5 0 2 0 2 1 5 0 12ACE:Placebo 5 7 5 56 3 3 0 6 1 0 2 3ARB:BBlocker 4 1 0 3 41 21 0 5 19 2 2 2ARB:CCB 1 2 1 2 8 67 0 6 8 1 0 2ARB:Diuretic 3 2 11 5 10 27 0 8 0 7 0 25ARB:Placebo 3 3 2 7 6 15 0 49 0 1 2 2BBlocker:CCB 6 4 1 1 11 12 0 0 53 4 2 5BBlocker:Diuretic 10 1 13 2 5 3 0 2 19 20 2 21BBlocker:Placebo 10 2 2 14 13 3 0 16 16 4 8 1CCB:Diuretic 2 6 11 3 1 3 0 2 7 6 0 56CCB:Placebo 2 6 4 12 1 15 0 16 6 0 2 5Diuretic:Placebo 0 0 20 20 2 7 0 9 0 5 2 17
Indirect estimatesACE:ARB 10 11 8 16 11 20 0 14 1 1 0 7
10 11 8
The contribution matrix
![Page 27: CINeMA webinar complete - Cochranetraining.cochrane.org/sites/training.cochrane.org/files/public/uploads... · The most critical question raised by patients and clinicians at the](https://reader036.vdocuments.us/reader036/viewer/2022090606/605b02eb298cbb78c31f8c4d/html5/thumbnails/27.jpg)
ACE:BBlocker
ACE:CCB
ACE:Diuretic
ACE:Placebo
ARB:BBlocker
ARB:CCB
ARB:Diuretic
ARB:Placebo
BBlocker:CCB
BBlocker:Diuretic
BBlocker:Placebo
CCB:Diuretic
Mixed estimates
ACE:BBlocker 32 10 10 8 6 1 0 4 15 6 2 5ACE:CCB 10 26 13 11 1 6 0 4 9 1 0 13ACE:Diuretic 6 7 57 5 0 2 0 2 1 5 0 12ACE:Placebo 5 7 5 56 3 3 0 6 1 0 2 3ARB:BBlocker 4 1 0 3 41 21 0 5 19 2 2 2ARB:CCB 1 2 1 2 8 67 0 6 8 1 0 2ARB:Diuretic 3 2 11 5 10 27 0 8 0 7 0 25ARB:Placebo 3 3 2 7 6 15 0 49 0 1 2 2BBlocker:CCB 6 4 1 1 11 12 0 0 53 4 2 5BBlocker:Diuretic 10 1 13 2 5 3 0 2 19 20 2 21BBlocker:Placebo 10 2 2 14 13 3 0 16 16 4 8 1CCB:Diuretic 2 6 11 3 1 3 0 2 7 6 0 56CCB:Placebo 2 6 4 12 1 15 0 16 6 0 2 5Diuretic:Placebo 0 0 20 20 2 7 0 9 0 5 2 17
Indirect estimatesACE:ARB 10 11 8 16 11 20 0 14 1 1 0 7
The contribution matrix
![Page 28: CINeMA webinar complete - Cochranetraining.cochrane.org/sites/training.cochrane.org/files/public/uploads... · The most critical question raised by patients and clinicians at the](https://reader036.vdocuments.us/reader036/viewer/2022090606/605b02eb298cbb78c31f8c4d/html5/thumbnails/28.jpg)
What is your judgement about study limitation for this (indirect) OR for ACE vs ARB estimated in NMA?
q Major concernsq Some concernsq No concerns
![Page 29: CINeMA webinar complete - Cochranetraining.cochrane.org/sites/training.cochrane.org/files/public/uploads... · The most critical question raised by patients and clinicians at the](https://reader036.vdocuments.us/reader036/viewer/2022090606/605b02eb298cbb78c31f8c4d/html5/thumbnails/29.jpg)
Some concerns
Major concerns
No concerns
No concerns
No concerns
No concerns
No concerns
No concerns
No concerns
No concerns
Some concerns
Some concerns
Some concerns
Some concerns
Some concerns
![Page 30: CINeMA webinar complete - Cochranetraining.cochrane.org/sites/training.cochrane.org/files/public/uploads... · The most critical question raised by patients and clinicians at the](https://reader036.vdocuments.us/reader036/viewer/2022090606/605b02eb298cbb78c31f8c4d/html5/thumbnails/30.jpg)
q Major concernsq Some concernsq No concerns
![Page 31: CINeMA webinar complete - Cochranetraining.cochrane.org/sites/training.cochrane.org/files/public/uploads... · The most critical question raised by patients and clinicians at the](https://reader036.vdocuments.us/reader036/viewer/2022090606/605b02eb298cbb78c31f8c4d/html5/thumbnails/31.jpg)
§ Considerations similar to those in a pairwise meta-analysis
§ How relevant is the study PICO and setting to the research question?§ Score each study at 3 levels
§ Low indirectness to the research question§ Moderate indirectness to the research question§ High indirectness to the research question
§ Then study-level judgements are summarized within pairwise comparisons and across the network using the contribution matrix exactly as with the Risk of Bias.
§ This also addresses the condition of transitivity!§ If the studies across comparisons have differences in important characteristics (e.g.
effect modifiers) compared to the target population, then the transitivity assumption is challenged
![Page 32: CINeMA webinar complete - Cochranetraining.cochrane.org/sites/training.cochrane.org/files/public/uploads... · The most critical question raised by patients and clinicians at the](https://reader036.vdocuments.us/reader036/viewer/2022090606/605b02eb298cbb78c31f8c4d/html5/thumbnails/32.jpg)
CINeMANow it is time for….
cinema.ispm.ch
![Page 33: CINeMA webinar complete - Cochranetraining.cochrane.org/sites/training.cochrane.org/files/public/uploads... · The most critical question raised by patients and clinicians at the](https://reader036.vdocuments.us/reader036/viewer/2022090606/605b02eb298cbb78c31f8c4d/html5/thumbnails/33.jpg)
q Major concernsq Some concernsq No concerns
![Page 34: CINeMA webinar complete - Cochranetraining.cochrane.org/sites/training.cochrane.org/files/public/uploads... · The most critical question raised by patients and clinicians at the](https://reader036.vdocuments.us/reader036/viewer/2022090606/605b02eb298cbb78c31f8c4d/html5/thumbnails/34.jpg)
§ Traditional GRADE considers, among others, the total sample size available and compares it with the Optimal Information Size
§ The sample size in a NMA relative effect makes little sense (as studies in the network contribute direct and indirect information!)
§ Imprecision relates to the width of the 95% confidence interval:
Does the 95% CI include values that lead to different clinical decisions?
§ Set a ”margin of equivalence”§ the range of relative treatment effect around the no-effect line that do not
signify important differences between the interventions§ Could be set using the Minimum Clinically Important Difference
![Page 35: CINeMA webinar complete - Cochranetraining.cochrane.org/sites/training.cochrane.org/files/public/uploads... · The most critical question raised by patients and clinicians at the](https://reader036.vdocuments.us/reader036/viewer/2022090606/605b02eb298cbb78c31f8c4d/html5/thumbnails/35.jpg)
BB vs PlaceboDiureticsCCBACEARB
Diuretics vs BBCCBACEARB
CCB vs DiureticsACEARB
ACE vs CCBARB
ARB vs ACE
0.4 0.7 1.5 21
Favors first Favors second
Imprecision: Confidence intervals include values that lead into different clinical decisions
Margin of equivalence:OR=1.05 in either directionImprecision when the confidence interval crosses both 0.95 and 1.05
imprecise
imprecise
imprecise
Comparison
NMA estimated odds ratios for diabetes
![Page 36: CINeMA webinar complete - Cochranetraining.cochrane.org/sites/training.cochrane.org/files/public/uploads... · The most critical question raised by patients and clinicians at the](https://reader036.vdocuments.us/reader036/viewer/2022090606/605b02eb298cbb78c31f8c4d/html5/thumbnails/36.jpg)
NMA estimated odds ratios for diabetes
0.4 0.7 1.5 21
Favors first Favors second
BB vs PlaceboDiureticsCCBACEARB
Comparison
Major concernsSome concerns
No concernsNo concerns
Some concerns
![Page 37: CINeMA webinar complete - Cochranetraining.cochrane.org/sites/training.cochrane.org/files/public/uploads... · The most critical question raised by patients and clinicians at the](https://reader036.vdocuments.us/reader036/viewer/2022090606/605b02eb298cbb78c31f8c4d/html5/thumbnails/37.jpg)
CINeMANow it is time for….
![Page 38: CINeMA webinar complete - Cochranetraining.cochrane.org/sites/training.cochrane.org/files/public/uploads... · The most critical question raised by patients and clinicians at the](https://reader036.vdocuments.us/reader036/viewer/2022090606/605b02eb298cbb78c31f8c4d/html5/thumbnails/38.jpg)
q Major concernsq Some concernsq No concerns
q Major concernsq Some concernsq No concerns
HETEROGENEITY INCOHERENCE
![Page 39: CINeMA webinar complete - Cochranetraining.cochrane.org/sites/training.cochrane.org/files/public/uploads... · The most critical question raised by patients and clinicians at the](https://reader036.vdocuments.us/reader036/viewer/2022090606/605b02eb298cbb78c31f8c4d/html5/thumbnails/39.jpg)
Incoherencedisagreement between
different sources of evidence
Heterogeneitybetween-study
variance within a comparison
![Page 40: CINeMA webinar complete - Cochranetraining.cochrane.org/sites/training.cochrane.org/files/public/uploads... · The most critical question raised by patients and clinicians at the](https://reader036.vdocuments.us/reader036/viewer/2022090606/605b02eb298cbb78c31f8c4d/html5/thumbnails/40.jpg)
§The major driver in judging heterogeneity is whether it impacts on clinical decisions
§Heterogeneity is represented by the predictive intervals: the intervals within which we expect to find the true effect size of a new study
§They are extensions of the confidence intervals
![Page 41: CINeMA webinar complete - Cochranetraining.cochrane.org/sites/training.cochrane.org/files/public/uploads... · The most critical question raised by patients and clinicians at the](https://reader036.vdocuments.us/reader036/viewer/2022090606/605b02eb298cbb78c31f8c4d/html5/thumbnails/41.jpg)
BB vs PlaceboDiureticsCCBACEARB
Diuretics vs BBCCBACEARB
CCB vs DiureticsACEARB
ACE vs CCBARB
ARB vs ACE
Treatment Effect
0.4 0.7 1.5 21
Favors first Favors second
INCONSISTENCY HETEROGENEITY
![Page 42: CINeMA webinar complete - Cochranetraining.cochrane.org/sites/training.cochrane.org/files/public/uploads... · The most critical question raised by patients and clinicians at the](https://reader036.vdocuments.us/reader036/viewer/2022090606/605b02eb298cbb78c31f8c4d/html5/thumbnails/42.jpg)
BB vs Placebo
Diuretics
CCB
ACE
ARB
Treatment EffectPrediction interval:Where is the true effect in a new study?
Heterogeneity changes conclusions!
0.4 0.7 1.5 21
Favors first Favors second
INCONSISTENCY HETEROGENEITY
![Page 43: CINeMA webinar complete - Cochranetraining.cochrane.org/sites/training.cochrane.org/files/public/uploads... · The most critical question raised by patients and clinicians at the](https://reader036.vdocuments.us/reader036/viewer/2022090606/605b02eb298cbb78c31f8c4d/html5/thumbnails/43.jpg)
BB vs PlaceboDiureticsCCBACEARB
Diuretics vs BBCCBACEARB
CCB vs DiureticsACEARB
ACE vs CCBARB
ARB vs ACE
Treatment Effect
0.4 0.7 1.5 21
INCONSISTENCY HETEROGENEITY
Favors first Favors second
Accounting for heterogeneity leads into different clinical decisions!
Heterogeneity does not changes conclusions!
![Page 44: CINeMA webinar complete - Cochranetraining.cochrane.org/sites/training.cochrane.org/files/public/uploads... · The most critical question raised by patients and clinicians at the](https://reader036.vdocuments.us/reader036/viewer/2022090606/605b02eb298cbb78c31f8c4d/html5/thumbnails/44.jpg)
Margin of equivalence
Prediction interval Confidence interval
INCONSISTENCY HETEROGENEITY
Rules implemented in the software
![Page 45: CINeMA webinar complete - Cochranetraining.cochrane.org/sites/training.cochrane.org/files/public/uploads... · The most critical question raised by patients and clinicians at the](https://reader036.vdocuments.us/reader036/viewer/2022090606/605b02eb298cbb78c31f8c4d/html5/thumbnails/45.jpg)
§ The major driver or our decisions is whether the heterogeneity impacts on clinical decisions
§ Heterogeneity is represented by the predictive intervals: the intervals within which we expect to find the true effect size of a new study
§ They are extensions of the confidence intervals
§ Pairwise meta-analysis heterogeneity variances τ2 can be estimated§ But their estimation makes sense when you have enough studies § The observed values of τ2 are can be compared with the expected values from empirical
evidence (Turner et al Int J Epidemiol. 2012, Rhodes et al. J Clin Epidemiol. 2015)§ The expected values depend on the nature of the outcome and the treatments being
compared
![Page 46: CINeMA webinar complete - Cochranetraining.cochrane.org/sites/training.cochrane.org/files/public/uploads... · The most critical question raised by patients and clinicians at the](https://reader036.vdocuments.us/reader036/viewer/2022090606/605b02eb298cbb78c31f8c4d/html5/thumbnails/46.jpg)
Separate Direct from Indirect Evidence test
![Page 47: CINeMA webinar complete - Cochranetraining.cochrane.org/sites/training.cochrane.org/files/public/uploads... · The most critical question raised by patients and clinicians at the](https://reader036.vdocuments.us/reader036/viewer/2022090606/605b02eb298cbb78c31f8c4d/html5/thumbnails/47.jpg)
Heterogeneitybetween-study
variance within a comparison
Incoherencedisagreement between
different sources of evidence
We consider prediction intervals for the impact of heterogeneity in clinical decision making
![Page 48: CINeMA webinar complete - Cochranetraining.cochrane.org/sites/training.cochrane.org/files/public/uploads... · The most critical question raised by patients and clinicians at the](https://reader036.vdocuments.us/reader036/viewer/2022090606/605b02eb298cbb78c31f8c4d/html5/thumbnails/48.jpg)
Heterogeneitybetween-study
variance within a comparison
Incoherencedisagreement between
different sources of evidence
We consider prediction intervals for the impact of heterogeneity in clinical decision making
Separate Direct from Indirect Evidence test (node-splitting): Compare direct and indirect relative treatment effects using a Z-test : one test for each treatment comparisons
Design-by-treatment test X2
: one test for the network
![Page 49: CINeMA webinar complete - Cochranetraining.cochrane.org/sites/training.cochrane.org/files/public/uploads... · The most critical question raised by patients and clinicians at the](https://reader036.vdocuments.us/reader036/viewer/2022090606/605b02eb298cbb78c31f8c4d/html5/thumbnails/49.jpg)
Separate Direct from Indirect Evidence test
ACE
Placebo
Compare!
Dias et al. Checking consistency in mixed treatment comparison meta-analysis Stat Med 2010
![Page 50: CINeMA webinar complete - Cochranetraining.cochrane.org/sites/training.cochrane.org/files/public/uploads... · The most critical question raised by patients and clinicians at the](https://reader036.vdocuments.us/reader036/viewer/2022090606/605b02eb298cbb78c31f8c4d/html5/thumbnails/50.jpg)
Design-by-treatment X2 test
Does the assumption of coherence hold for the
entire network?
χ2 =19.325 (13 df)P-value=0.113
White et al. Consistency and inconsistency in network meta-analysis. Res Synth Meth 2012
![Page 51: CINeMA webinar complete - Cochranetraining.cochrane.org/sites/training.cochrane.org/files/public/uploads... · The most critical question raised by patients and clinicians at the](https://reader036.vdocuments.us/reader036/viewer/2022090606/605b02eb298cbb78c31f8c4d/html5/thumbnails/51.jpg)
Design-by-treatment interaction model
p-value>0.1 0.01<p-value<0.1 p-value<0.01
SIDE
p-value>0.1 No concerns No concerns Some concerns
0.01<p-value<0.1 Some concerns Some concerns Major concerns
p-value<0.01 Some concerns Major concerns Major concerns
Treatment comparisons that take at least 90% of the information from direct evidence have no concerns for incoherenceFor comparisons with at least 10% of information derived from indirect evidence we use the following rules
![Page 52: CINeMA webinar complete - Cochranetraining.cochrane.org/sites/training.cochrane.org/files/public/uploads... · The most critical question raised by patients and clinicians at the](https://reader036.vdocuments.us/reader036/viewer/2022090606/605b02eb298cbb78c31f8c4d/html5/thumbnails/52.jpg)
Salanti G. Indirect and mixed-treatment comparison, network, or multiple-treatments meta-analysis. Res Synth Meth 2012
![Page 53: CINeMA webinar complete - Cochranetraining.cochrane.org/sites/training.cochrane.org/files/public/uploads... · The most critical question raised by patients and clinicians at the](https://reader036.vdocuments.us/reader036/viewer/2022090606/605b02eb298cbb78c31f8c4d/html5/thumbnails/53.jpg)
q Suspectedq Undetected
![Page 54: CINeMA webinar complete - Cochranetraining.cochrane.org/sites/training.cochrane.org/files/public/uploads... · The most critical question raised by patients and clinicians at the](https://reader036.vdocuments.us/reader036/viewer/2022090606/605b02eb298cbb78c31f8c4d/html5/thumbnails/54.jpg)
CINeMANow it is time for….
![Page 55: CINeMA webinar complete - Cochranetraining.cochrane.org/sites/training.cochrane.org/files/public/uploads... · The most critical question raised by patients and clinicians at the](https://reader036.vdocuments.us/reader036/viewer/2022090606/605b02eb298cbb78c31f8c4d/html5/thumbnails/55.jpg)
You are welcome to use CINeMA with the understanding that it is still under development§We will improve the data input module§We will fix some known bugs in the calculations§For some calculations CINeMA the netmeta package in R, so
updates/debugging in netmeta affect CINeMA too§Please notify us for any problems you come across
[email protected]§ If you use it in a publication you can cite CINeMA: Confidence in Network Meta-Analysis [Software]. University of Bern 2017. Available from cinema.ispm.ch