parvin tajik, md phd candidate department of clinical epidemiology & biostatistics
DESCRIPTION
FHCRC 2014 Risk Prediction Symposium June 11, 2014. Illustration of the evaluation of risk prediction models in randomized trials Examples from women’s health studies. Parvin Tajik, MD PhD candidate Department of Clinical Epidemiology & Biostatistics Department of Obstetrics & Gynecology - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Parvin Tajik, MD PhD candidate Department of Clinical Epidemiology & Biostatistics](https://reader036.vdocuments.us/reader036/viewer/2022081506/5681581b550346895dc581c9/html5/thumbnails/1.jpg)
Illustration of the evaluation of risk prediction models in randomized trialsExamples from women’s health studies
Parvin Tajik, MDPhD candidateDepartment of Clinical Epidemiology & BiostatisticsDepartment of Obstetrics & GynecologyAcademic Medical Center, University of Amsterdam, the Netherlands
FHCRC 2014 Risk Prediction SymposiumJune 11, 2014
![Page 2: Parvin Tajik, MD PhD candidate Department of Clinical Epidemiology & Biostatistics](https://reader036.vdocuments.us/reader036/viewer/2022081506/5681581b550346895dc581c9/html5/thumbnails/2.jpg)
Clinical Problem I
Pre-eclampsia
![Page 3: Parvin Tajik, MD PhD candidate Department of Clinical Epidemiology & Biostatistics](https://reader036.vdocuments.us/reader036/viewer/2022081506/5681581b550346895dc581c9/html5/thumbnails/3.jpg)
fullPIERS model
Lancet, 2011
![Page 4: Parvin Tajik, MD PhD candidate Department of Clinical Epidemiology & Biostatistics](https://reader036.vdocuments.us/reader036/viewer/2022081506/5681581b550346895dc581c9/html5/thumbnails/4.jpg)
Development Method
• Patients: • 2000 women admitted in hospital for pre-eclapmsia
(260 event)
• Outcome: • Maternal mortality or other serious complications of
pre-eclampsia
• Logistic regression model with stepwise backward elimination
![Page 5: Parvin Tajik, MD PhD candidate Department of Clinical Epidemiology & Biostatistics](https://reader036.vdocuments.us/reader036/viewer/2022081506/5681581b550346895dc581c9/html5/thumbnails/5.jpg)
Final model
Logit P(D) = 2.68 – (0.054 × gestational age at eligibility) + (1.23 × chest pain or dyspnoea) – (0.027 × creatinine) + (0.21 × platelets) + (0.00004 × platelets2) + (0.01 × AST) – (0.000003 × AST2) + (0.00025 × creatinine × platelet) – (0.00007 × platelets × AST) – (0.0026 × platelets × SpO2)
![Page 6: Parvin Tajik, MD PhD candidate Department of Clinical Epidemiology & Biostatistics](https://reader036.vdocuments.us/reader036/viewer/2022081506/5681581b550346895dc581c9/html5/thumbnails/6.jpg)
Performance of full-PIERS model
Reported good risk discrimination and calibration
![Page 7: Parvin Tajik, MD PhD candidate Department of Clinical Epidemiology & Biostatistics](https://reader036.vdocuments.us/reader036/viewer/2022081506/5681581b550346895dc581c9/html5/thumbnails/7.jpg)
Online calculator
![Page 8: Parvin Tajik, MD PhD candidate Department of Clinical Epidemiology & Biostatistics](https://reader036.vdocuments.us/reader036/viewer/2022081506/5681581b550346895dc581c9/html5/thumbnails/8.jpg)
HYPITAT trial (2005-2008)
• PP Women at 36-41 wks of pregnancy with mild pre-eclampsia (n=750)
• I I Early Induction of labor (LI)
• C C Expectant monitoring (EM)
• O O Composite measure of adverse maternal outcomes
![Page 9: Parvin Tajik, MD PhD candidate Department of Clinical Epidemiology & Biostatistics](https://reader036.vdocuments.us/reader036/viewer/2022081506/5681581b550346895dc581c9/html5/thumbnails/9.jpg)
HYPTAT Results
(relative risk 0.71, 95% CI 0.59–0.86, p<0·0001)
ManagementManagement Adverse maternal Adverse maternal outcomesoutcomes
TotalTotal
Labor induction 117 (31%) 377Expectant monitoring 166 (44 %) 379
![Page 10: Parvin Tajik, MD PhD candidate Department of Clinical Epidemiology & Biostatistics](https://reader036.vdocuments.us/reader036/viewer/2022081506/5681581b550346895dc581c9/html5/thumbnails/10.jpg)
Modeling
Logit P(D=1|T,Y) = β0 + β1T + β2Y + β3TY
•D = 1 Adverse maternal outcome•Y = fullPIERS score•T = Treatment
• 1 Labor induction • 0 Expectant monitoring
![Page 11: Parvin Tajik, MD PhD candidate Department of Clinical Epidemiology & Biostatistics](https://reader036.vdocuments.us/reader036/viewer/2022081506/5681581b550346895dc581c9/html5/thumbnails/11.jpg)
FullPIERS for guiding labor induction
P for interaction: 0.93
fullPIERS score
![Page 12: Parvin Tajik, MD PhD candidate Department of Clinical Epidemiology & Biostatistics](https://reader036.vdocuments.us/reader036/viewer/2022081506/5681581b550346895dc581c9/html5/thumbnails/12.jpg)
Clinical Problem II
Preterm birth
![Page 13: Parvin Tajik, MD PhD candidate Department of Clinical Epidemiology & Biostatistics](https://reader036.vdocuments.us/reader036/viewer/2022081506/5681581b550346895dc581c9/html5/thumbnails/13.jpg)
Cervical pessary• Medical device inserted to vagina• to provide structural support to cervix
![Page 14: Parvin Tajik, MD PhD candidate Department of Clinical Epidemiology & Biostatistics](https://reader036.vdocuments.us/reader036/viewer/2022081506/5681581b550346895dc581c9/html5/thumbnails/14.jpg)
ProTWIN trial (2009-2012)
• P Women with multiple pregnancy (twin or triplet) between 12 & 20 weeks pregnancy
• I Cervical Pessary (n = 403)• C Control (n = 410)
• O Primary: Composite Adverse perinatal outcome
![Page 15: Parvin Tajik, MD PhD candidate Department of Clinical Epidemiology & Biostatistics](https://reader036.vdocuments.us/reader036/viewer/2022081506/5681581b550346895dc581c9/html5/thumbnails/15.jpg)
ProTWIN Results
(relative risk 0.98, 95% CI 0.69–1.39)
ManagementManagement Composite adverse Composite adverse perinatal outcomeperinatal outcome
TotalTotal
Pessary 53 (13%) 401No pessary 55 (14 %) 407
![Page 16: Parvin Tajik, MD PhD candidate Department of Clinical Epidemiology & Biostatistics](https://reader036.vdocuments.us/reader036/viewer/2022081506/5681581b550346895dc581c9/html5/thumbnails/16.jpg)
Pre-specified subgroup analysis
Cervical length (<38 mm vs >= 38 mm)
![Page 17: Parvin Tajik, MD PhD candidate Department of Clinical Epidemiology & Biostatistics](https://reader036.vdocuments.us/reader036/viewer/2022081506/5681581b550346895dc581c9/html5/thumbnails/17.jpg)
Pre-specified subgroup analysis
Trial Conclusion: Clinicians should consider a cervical pessary in women with a multiple pregnancy and a short
cervical length.
Cervical length Pessary group
Control group
RR (95%CI)
CxL < 38 mm 12% 29% 0.42 (0.19-0.91)CxL >= 38 mm 13% 10% 1.26 (0.74-2.15)
(P for interaction 0.01)
![Page 18: Parvin Tajik, MD PhD candidate Department of Clinical Epidemiology & Biostatistics](https://reader036.vdocuments.us/reader036/viewer/2022081506/5681581b550346895dc581c9/html5/thumbnails/18.jpg)
Other Markers
1. Obstetric history (parity) • Nulliparous• Parous with no previous preterm birth• Parous with at least one previous preterm birth
2. Chorionicity• Monochorionic• Dichorionic
3. Number of fetuses• Twin• Triplet
![Page 19: Parvin Tajik, MD PhD candidate Department of Clinical Epidemiology & Biostatistics](https://reader036.vdocuments.us/reader036/viewer/2022081506/5681581b550346895dc581c9/html5/thumbnails/19.jpg)
One marker at a time analysis
Other Potential Treatment Selection Factors
% Poor Outcome Odds Ratio (95% CI)
Odds Ratio (95% CI)
Int. P-value
Pessary Control
Cervical length
< 38 mm 11.54 29.09 0.32 (0.13-0.79) 0.010
≥ 38mm 12.85 10.13 1.31 (0.75-2.30)
Chorionicity
Monochorionic 13.79 26.00 0.46 (0.21-0.97) 0.015
Dichorionic 13.06 9.51 1.43 (0.86-2.37) Obstetric history
Nulliparous 13.12 18.30 0.67 (0.40-1.13) 0.212
Parous with no previous preterm birth 9.93 8.28 1.22 (0.56-2.66)
Parous with at least one previous preterm birth
31.03 3.85 11.25 (1.31-96.4) 0.012
Number of foetuses
Twin 12.50 13.32 0.98 (0.61-1.41) 0.301
Triplet 44.44 22.22 2.8 (0.36-21.73)
![Page 20: Parvin Tajik, MD PhD candidate Department of Clinical Epidemiology & Biostatistics](https://reader036.vdocuments.us/reader036/viewer/2022081506/5681581b550346895dc581c9/html5/thumbnails/20.jpg)
Modeling
Logit P(D=1|T,Y) = β0 + β1T + Σ βiYi + Σ βjTYj
•D = 1 composite poor perinatal outcome•Y = Markers•T = Treatment
• 1 pessary• 0 control
- Internal validation by bootstrapping
![Page 21: Parvin Tajik, MD PhD candidate Department of Clinical Epidemiology & Biostatistics](https://reader036.vdocuments.us/reader036/viewer/2022081506/5681581b550346895dc581c9/html5/thumbnails/21.jpg)
Multi-marker modelPredictor OR (95% CI) Beta*
P-value
Intercept
-2.08
<0.001 Main terms Pessary 1.13 (0.57-2.24) 0.12 0.426
Cervical length <38 mm 2.20 (1.09-4.46) 0.79 <0.001
Monochorionic 2.44 (1.33-4.47) 0.89 <0.001
Parous with no previous preterm birth 0.53 (0.27-1.06) -0.63 0.031
Parous with at least one previous preterm birth 0.34 (0.04- 2.63) -1.09 0.165
Triplet 1.49 (0.28- 8.05) 0.40 0.010
Interaction terms
Pessary × Cervical length <38 mm 0.52 (0.19-1.42) -0.65 0.058
Pessary × Monochorionic 0.41 (0.16-1.05) -0.89 0.009
Pessary × Parous with no previous preterm birth 1.52 (0.58-3.98) 0.42 0.312
Pessary × Parous with at least one previous preterm birth 7.24 (0.78-67.65) 1.98 0.020
* Shrunken with an average shrinkage factor of 0.76c-stat : 0,71 (95%CI: 0,66-0,77); optimism-corrected c-stat: 0,69 (95%CI: 0,63-0,74)
![Page 22: Parvin Tajik, MD PhD candidate Department of Clinical Epidemiology & Biostatistics](https://reader036.vdocuments.us/reader036/viewer/2022081506/5681581b550346895dc581c9/html5/thumbnails/22.jpg)
How can the model be used in practice?
![Page 23: Parvin Tajik, MD PhD candidate Department of Clinical Epidemiology & Biostatistics](https://reader036.vdocuments.us/reader036/viewer/2022081506/5681581b550346895dc581c9/html5/thumbnails/23.jpg)
Predicted benefit from pessary
050
100
150
200
250
Stu
dy
Pa
rtic
ipa
nts
, %
* ** *** ** ** ** **** * ** ** **** *** **** ***** * *** ** *** **** * ** *** ** **** * ***** ***** ** ** ** ** ***** ****
-0.2 -0.1 0.0 0.1 0.2
Predicted Difference (Control-Pessary) in Poor Perinatal Outcome
Favors Control Favors Pessary
![Page 24: Parvin Tajik, MD PhD candidate Department of Clinical Epidemiology & Biostatistics](https://reader036.vdocuments.us/reader036/viewer/2022081506/5681581b550346895dc581c9/html5/thumbnails/24.jpg)
Calibration of the predicted benefit
-30 -20 -10 0 10 20 30 40
-30
-20
-10
01
02
03
04
0
Expected Treatment Effect
Ob
serv
ed
Tre
atm
en
t Effe
ct
![Page 25: Parvin Tajik, MD PhD candidate Department of Clinical Epidemiology & Biostatistics](https://reader036.vdocuments.us/reader036/viewer/2022081506/5681581b550346895dc581c9/html5/thumbnails/25.jpg)
Model performance
-30 -20 -10 0 10 20 30 40
-30
-20
-10
01
02
03
04
0
Expected Treatment Effect
Ob
serv
ed
Tre
atm
en
t Effe
ct
![Page 26: Parvin Tajik, MD PhD candidate Department of Clinical Epidemiology & Biostatistics](https://reader036.vdocuments.us/reader036/viewer/2022081506/5681581b550346895dc581c9/html5/thumbnails/26.jpg)
Conclusion
• Common assumption for application of risk prediction models for treatment selection:“Being at higher risk of outcome implies a
larger benefit from treatment” • Not necessarily true
• Developing models using trial data and modeling the interaction between markers and treatment might be a more optimal strategy
![Page 27: Parvin Tajik, MD PhD candidate Department of Clinical Epidemiology & Biostatistics](https://reader036.vdocuments.us/reader036/viewer/2022081506/5681581b550346895dc581c9/html5/thumbnails/27.jpg)
Open Research Questions
• Optimal modeling strategy?
• Optimal algorithm for variable selection?
• Optimal method for optimism correction?
![Page 28: Parvin Tajik, MD PhD candidate Department of Clinical Epidemiology & Biostatistics](https://reader036.vdocuments.us/reader036/viewer/2022081506/5681581b550346895dc581c9/html5/thumbnails/28.jpg)
Thanks!Any Questions?
![Page 29: Parvin Tajik, MD PhD candidate Department of Clinical Epidemiology & Biostatistics](https://reader036.vdocuments.us/reader036/viewer/2022081506/5681581b550346895dc581c9/html5/thumbnails/29.jpg)
Multimarker vs. CxL only
Multimarker + Multimarker -
Short cervix 174 9
Long cervix 120 505
![Page 30: Parvin Tajik, MD PhD candidate Department of Clinical Epidemiology & Biostatistics](https://reader036.vdocuments.us/reader036/viewer/2022081506/5681581b550346895dc581c9/html5/thumbnails/30.jpg)
Two examples