session 3 managing uncertainty in the post- launch phase · 1. evidence generation timeline 2....
TRANSCRIPT
![Page 1: Session 3 Managing uncertainty in the post- launch phase · 1. Evidence generation timeline 2. Uncertanity of evidence 3. Managing uncertainty ... technology should be weighed against](https://reader033.vdocuments.us/reader033/viewer/2022060523/605299fbc1316e11ff4793e1/html5/thumbnails/1.jpg)
Session 3 – Managing uncertainty in the post-launch phase Piotr Szymanski Associate Professor of Cardiology European Society of Cardiology
![Page 2: Session 3 Managing uncertainty in the post- launch phase · 1. Evidence generation timeline 2. Uncertanity of evidence 3. Managing uncertainty ... technology should be weighed against](https://reader033.vdocuments.us/reader033/viewer/2022060523/605299fbc1316e11ff4793e1/html5/thumbnails/2.jpg)
Managing uncertainty in post-launch phase – key points
1. Evidence generation timeline
2. Uncertanity of evidence
3. Managing uncertainty
![Page 3: Session 3 Managing uncertainty in the post- launch phase · 1. Evidence generation timeline 2. Uncertanity of evidence 3. Managing uncertainty ... technology should be weighed against](https://reader033.vdocuments.us/reader033/viewer/2022060523/605299fbc1316e11ff4793e1/html5/thumbnails/3.jpg)
Managing uncertainty in post-launch phase – key points
1. Evidence generation timeline 2. Uncertainty of evidence
3. Managing uncertainty
![Page 4: Session 3 Managing uncertainty in the post- launch phase · 1. Evidence generation timeline 2. Uncertanity of evidence 3. Managing uncertainty ... technology should be weighed against](https://reader033.vdocuments.us/reader033/viewer/2022060523/605299fbc1316e11ff4793e1/html5/thumbnails/4.jpg)
Launch of technology
CE MARK
AVAILABLE NOT REIMBURSED
HTA
AVAILABLE REIMBURSED
![Page 5: Session 3 Managing uncertainty in the post- launch phase · 1. Evidence generation timeline 2. Uncertanity of evidence 3. Managing uncertainty ... technology should be weighed against](https://reader033.vdocuments.us/reader033/viewer/2022060523/605299fbc1316e11ff4793e1/html5/thumbnails/5.jpg)
Launch of technology
CE MARK
AVAILABLE NOT REIMBURSED
HTA
AVAILABLE REIMBURSED
![Page 6: Session 3 Managing uncertainty in the post- launch phase · 1. Evidence generation timeline 2. Uncertanity of evidence 3. Managing uncertainty ... technology should be weighed against](https://reader033.vdocuments.us/reader033/viewer/2022060523/605299fbc1316e11ff4793e1/html5/thumbnails/6.jpg)
Launch of technology
CE MARK HTA
AVAILABLE REIMBURSED
![Page 7: Session 3 Managing uncertainty in the post- launch phase · 1. Evidence generation timeline 2. Uncertanity of evidence 3. Managing uncertainty ... technology should be weighed against](https://reader033.vdocuments.us/reader033/viewer/2022060523/605299fbc1316e11ff4793e1/html5/thumbnails/7.jpg)
Evidence generation
CE MARK
AVAILABLE NOT REIMBURSED
HTA
AVAILABLE REIMBURSED
EVIDENCE EVIDENCE
EVIDENCE
![Page 8: Session 3 Managing uncertainty in the post- launch phase · 1. Evidence generation timeline 2. Uncertanity of evidence 3. Managing uncertainty ... technology should be weighed against](https://reader033.vdocuments.us/reader033/viewer/2022060523/605299fbc1316e11ff4793e1/html5/thumbnails/8.jpg)
Evidence generation
CE MARK
AVAILABLE NOT REIMBURSED
HTA
AVAILABLE REIMBURSED
EVIDENCE EVIDENCE
EVIDENCE
![Page 9: Session 3 Managing uncertainty in the post- launch phase · 1. Evidence generation timeline 2. Uncertanity of evidence 3. Managing uncertainty ... technology should be weighed against](https://reader033.vdocuments.us/reader033/viewer/2022060523/605299fbc1316e11ff4793e1/html5/thumbnails/9.jpg)
Evidence generation
CE MARK HTA
AVAILABLE REIMBURSED
EVIDENCE
EVIDENCE
![Page 10: Session 3 Managing uncertainty in the post- launch phase · 1. Evidence generation timeline 2. Uncertanity of evidence 3. Managing uncertainty ... technology should be weighed against](https://reader033.vdocuments.us/reader033/viewer/2022060523/605299fbc1316e11ff4793e1/html5/thumbnails/10.jpg)
Evidence generation timeline vs source
CE MARK
AVAILABLE NOT REIMBURSED
HTA
AVAILABLE REIMBURSED
EVIDENCE EVIDENCE
EVIDENCE
NON-COMMERCIAL REASERCH
INDUSTRY SPONSORED RESEARCH
![Page 11: Session 3 Managing uncertainty in the post- launch phase · 1. Evidence generation timeline 2. Uncertanity of evidence 3. Managing uncertainty ... technology should be weighed against](https://reader033.vdocuments.us/reader033/viewer/2022060523/605299fbc1316e11ff4793e1/html5/thumbnails/11.jpg)
Source of evidence
non-commercial : commercial
1:3
11
1/3 of clinical trials
non-commercial
![Page 12: Session 3 Managing uncertainty in the post- launch phase · 1. Evidence generation timeline 2. Uncertanity of evidence 3. Managing uncertainty ... technology should be weighed against](https://reader033.vdocuments.us/reader033/viewer/2022060523/605299fbc1316e11ff4793e1/html5/thumbnails/12.jpg)
Managing uncertainty in post-launch phase – key points
1. Evidence generation
2. Uncertainty of evidence 3. Managing uncertainty
![Page 13: Session 3 Managing uncertainty in the post- launch phase · 1. Evidence generation timeline 2. Uncertanity of evidence 3. Managing uncertainty ... technology should be weighed against](https://reader033.vdocuments.us/reader033/viewer/2022060523/605299fbc1316e11ff4793e1/html5/thumbnails/13.jpg)
Evidence pyramid
![Page 14: Session 3 Managing uncertainty in the post- launch phase · 1. Evidence generation timeline 2. Uncertanity of evidence 3. Managing uncertainty ... technology should be weighed against](https://reader033.vdocuments.us/reader033/viewer/2022060523/605299fbc1316e11ff4793e1/html5/thumbnails/14.jpg)
Evidence pyramid
„Benchmark” – randomized
controlled trial
![Page 15: Session 3 Managing uncertainty in the post- launch phase · 1. Evidence generation timeline 2. Uncertanity of evidence 3. Managing uncertainty ... technology should be weighed against](https://reader033.vdocuments.us/reader033/viewer/2022060523/605299fbc1316e11ff4793e1/html5/thumbnails/15.jpg)
Quality of Evidence
15 JAMA.
2014;311:368-77
JAMA. 2014 Jan 22-29;311(4):368-77
![Page 16: Session 3 Managing uncertainty in the post- launch phase · 1. Evidence generation timeline 2. Uncertanity of evidence 3. Managing uncertainty ... technology should be weighed against](https://reader033.vdocuments.us/reader033/viewer/2022060523/605299fbc1316e11ff4793e1/html5/thumbnails/16.jpg)
Quality of Evidence
16 JAMA.
2014;311:368-77
Randomized trials in 9/10 cases (in cancer -
5/10 cases)
![Page 17: Session 3 Managing uncertainty in the post- launch phase · 1. Evidence generation timeline 2. Uncertanity of evidence 3. Managing uncertainty ... technology should be weighed against](https://reader033.vdocuments.us/reader033/viewer/2022060523/605299fbc1316e11ff4793e1/html5/thumbnails/17.jpg)
Quality of Evidence
17 JAMA.
2014;311:368-77
Surrogate outcomes in ½ of
trials
![Page 18: Session 3 Managing uncertainty in the post- launch phase · 1. Evidence generation timeline 2. Uncertanity of evidence 3. Managing uncertainty ... technology should be weighed against](https://reader033.vdocuments.us/reader033/viewer/2022060523/605299fbc1316e11ff4793e1/html5/thumbnails/18.jpg)
Quality of Evidence
18 JAMA.
2014;311:368-77
Populations studied and duration of treatment are disproportionately small
compared to real life numbers
![Page 19: Session 3 Managing uncertainty in the post- launch phase · 1. Evidence generation timeline 2. Uncertanity of evidence 3. Managing uncertainty ... technology should be weighed against](https://reader033.vdocuments.us/reader033/viewer/2022060523/605299fbc1316e11ff4793e1/html5/thumbnails/19.jpg)
Quality of Evidence – Medical Devices
19 JAMA. 2017;318(7):619-
625.
![Page 20: Session 3 Managing uncertainty in the post- launch phase · 1. Evidence generation timeline 2. Uncertanity of evidence 3. Managing uncertainty ... technology should be weighed against](https://reader033.vdocuments.us/reader033/viewer/2022060523/605299fbc1316e11ff4793e1/html5/thumbnails/20.jpg)
Quality of Evidence – Medical Devices
20 JAMA. 2017;318(7):619-
625.
less than half of clinical studies submitted for approval of high-risk
medical devices were randomized
![Page 21: Session 3 Managing uncertainty in the post- launch phase · 1. Evidence generation timeline 2. Uncertanity of evidence 3. Managing uncertainty ... technology should be weighed against](https://reader033.vdocuments.us/reader033/viewer/2022060523/605299fbc1316e11ff4793e1/html5/thumbnails/21.jpg)
Evidence available at technology approval
Randomized controlled trial
is a „benchmark”
![Page 22: Session 3 Managing uncertainty in the post- launch phase · 1. Evidence generation timeline 2. Uncertanity of evidence 3. Managing uncertainty ... technology should be weighed against](https://reader033.vdocuments.us/reader033/viewer/2022060523/605299fbc1316e11ff4793e1/html5/thumbnails/22.jpg)
Quality of Evidence
22 PLoS Med.
2005;2:e124.
Randomized controlled trial is a „benchmark”
– but may be false
![Page 23: Session 3 Managing uncertainty in the post- launch phase · 1. Evidence generation timeline 2. Uncertanity of evidence 3. Managing uncertainty ... technology should be weighed against](https://reader033.vdocuments.us/reader033/viewer/2022060523/605299fbc1316e11ff4793e1/html5/thumbnails/23.jpg)
Quality of Evidence
23
power
True/not true
bias example PPV
PLoS Med.
2005;2:e124.
Even with good quality RCT, due to issues related
to power, bias, pre-test probability, the proportion
of true to false results is 1:1
![Page 24: Session 3 Managing uncertainty in the post- launch phase · 1. Evidence generation timeline 2. Uncertanity of evidence 3. Managing uncertainty ... technology should be weighed against](https://reader033.vdocuments.us/reader033/viewer/2022060523/605299fbc1316e11ff4793e1/html5/thumbnails/24.jpg)
Quality of Evidence
24
power
True/not true
bias example PPV
PLoS Med.
2005;2:e124.
True/not true
In case of underpowered RCT the risk rises 5-fold
![Page 25: Session 3 Managing uncertainty in the post- launch phase · 1. Evidence generation timeline 2. Uncertanity of evidence 3. Managing uncertainty ... technology should be weighed against](https://reader033.vdocuments.us/reader033/viewer/2022060523/605299fbc1316e11ff4793e1/html5/thumbnails/25.jpg)
Quality of Evidence
25 JAMA. 2005;294:218-28.
![Page 26: Session 3 Managing uncertainty in the post- launch phase · 1. Evidence generation timeline 2. Uncertanity of evidence 3. Managing uncertainty ... technology should be weighed against](https://reader033.vdocuments.us/reader033/viewer/2022060523/605299fbc1316e11ff4793e1/html5/thumbnails/26.jpg)
Quality of Evidence
26 JAMA. 2005;294:218-28.
1/4 of 49 highly cited clinical
studies remained largely unchallenged
by subsequent studies
![Page 27: Session 3 Managing uncertainty in the post- launch phase · 1. Evidence generation timeline 2. Uncertanity of evidence 3. Managing uncertainty ... technology should be weighed against](https://reader033.vdocuments.us/reader033/viewer/2022060523/605299fbc1316e11ff4793e1/html5/thumbnails/27.jpg)
Quality of evidence and postmarket safety
27 JAMA. 2017;317:1854-1863
![Page 28: Session 3 Managing uncertainty in the post- launch phase · 1. Evidence generation timeline 2. Uncertanity of evidence 3. Managing uncertainty ... technology should be weighed against](https://reader033.vdocuments.us/reader033/viewer/2022060523/605299fbc1316e11ff4793e1/html5/thumbnails/28.jpg)
Quality of evidence and postmarket safety
28 JAMA. 2017;317:1854-1863
The risk of postmarket safety events is over two-fold higher with
accelerated approval
![Page 29: Session 3 Managing uncertainty in the post- launch phase · 1. Evidence generation timeline 2. Uncertanity of evidence 3. Managing uncertainty ... technology should be weighed against](https://reader033.vdocuments.us/reader033/viewer/2022060523/605299fbc1316e11ff4793e1/html5/thumbnails/29.jpg)
Quality of evidence and postmarket safety – medical devices
29 BMJ. 2016
;353:i3323.
The risk of postmarket
safety events was two-fold
higher in EU vs US
![Page 30: Session 3 Managing uncertainty in the post- launch phase · 1. Evidence generation timeline 2. Uncertanity of evidence 3. Managing uncertainty ... technology should be weighed against](https://reader033.vdocuments.us/reader033/viewer/2022060523/605299fbc1316e11ff4793e1/html5/thumbnails/30.jpg)
Missing evidence in the postapproval phase
30 BMJ 2017;357:j1680
![Page 31: Session 3 Managing uncertainty in the post- launch phase · 1. Evidence generation timeline 2. Uncertanity of evidence 3. Managing uncertainty ... technology should be weighed against](https://reader033.vdocuments.us/reader033/viewer/2022060523/605299fbc1316e11ff4793e1/html5/thumbnails/31.jpg)
Missing evidence in the postapproval phase
31 BMJ 2017;357:j1680
no postapproval studies were performed for 43 of the 123
(35%) indications approved on the basis of limited evidence
(single pivotal trial or surrogate endpoints)
![Page 32: Session 3 Managing uncertainty in the post- launch phase · 1. Evidence generation timeline 2. Uncertanity of evidence 3. Managing uncertainty ... technology should be weighed against](https://reader033.vdocuments.us/reader033/viewer/2022060523/605299fbc1316e11ff4793e1/html5/thumbnails/32.jpg)
EVIDENCE
CE MARK
AVAILABLE NOT REIMBURSED
HTA
AVAILABLE REIMBURSED
MISSING EVIDENCE EVIDENCE
Missing evidence for HTA
![Page 33: Session 3 Managing uncertainty in the post- launch phase · 1. Evidence generation timeline 2. Uncertanity of evidence 3. Managing uncertainty ... technology should be weighed against](https://reader033.vdocuments.us/reader033/viewer/2022060523/605299fbc1316e11ff4793e1/html5/thumbnails/33.jpg)
Managing uncertainity in post-launch phase – key points
1. Evidence generation
2. Uncertainity of evidence
3. Managing uncertainity
![Page 34: Session 3 Managing uncertainty in the post- launch phase · 1. Evidence generation timeline 2. Uncertanity of evidence 3. Managing uncertainty ... technology should be weighed against](https://reader033.vdocuments.us/reader033/viewer/2022060523/605299fbc1316e11ff4793e1/html5/thumbnails/34.jpg)
Clinical registries – technology comparisons
34
An integrated clinical-data surveillance system used to conduct a prospective, propensity-matched analysis of the safety of the Mynx vascular-closure device, as compared with alternative approved vascular-closure devices, with data from the CathPCI Registry of the National Cardiovascular Data Registry.
Vascular complications more frequent with Mynx vs comparators
N Engl J Med 2017; 376:526
![Page 35: Session 3 Managing uncertainty in the post- launch phase · 1. Evidence generation timeline 2. Uncertanity of evidence 3. Managing uncertainty ... technology should be weighed against](https://reader033.vdocuments.us/reader033/viewer/2022060523/605299fbc1316e11ff4793e1/html5/thumbnails/35.jpg)
Postmarket surveillance
35 J Card Electrophysiol. 2018;29:5
![Page 36: Session 3 Managing uncertainty in the post- launch phase · 1. Evidence generation timeline 2. Uncertanity of evidence 3. Managing uncertainty ... technology should be weighed against](https://reader033.vdocuments.us/reader033/viewer/2022060523/605299fbc1316e11ff4793e1/html5/thumbnails/36.jpg)
Postmarket surveillance
36 J Card Electrophysiol. 2018;29:5
Risk of malfunction
several dozen higher post- as
compared to pre-FDA approval
![Page 37: Session 3 Managing uncertainty in the post- launch phase · 1. Evidence generation timeline 2. Uncertanity of evidence 3. Managing uncertainty ... technology should be weighed against](https://reader033.vdocuments.us/reader033/viewer/2022060523/605299fbc1316e11ff4793e1/html5/thumbnails/37.jpg)
ESC Atlas of Cardiology database, 2017
Variations in uptake may result in
variable learning curves, differences in complication
rates, cost-effectiveness, etc.
Variations in annual implant rates for CRTD/per million
![Page 38: Session 3 Managing uncertainty in the post- launch phase · 1. Evidence generation timeline 2. Uncertanity of evidence 3. Managing uncertainty ... technology should be weighed against](https://reader033.vdocuments.us/reader033/viewer/2022060523/605299fbc1316e11ff4793e1/html5/thumbnails/38.jpg)
Quality of Evidence
38
power
True/not true
bias example PPV
PLoS Med.
2005;2:e124.
True/not true
observational < randomized
![Page 39: Session 3 Managing uncertainty in the post- launch phase · 1. Evidence generation timeline 2. Uncertanity of evidence 3. Managing uncertainty ... technology should be weighed against](https://reader033.vdocuments.us/reader033/viewer/2022060523/605299fbc1316e11ff4793e1/html5/thumbnails/39.jpg)
Registry-based randomized trial
40 Courtesy: Jonas Oldgren, UCR
![Page 40: Session 3 Managing uncertainty in the post- launch phase · 1. Evidence generation timeline 2. Uncertanity of evidence 3. Managing uncertainty ... technology should be weighed against](https://reader033.vdocuments.us/reader033/viewer/2022060523/605299fbc1316e11ff4793e1/html5/thumbnails/40.jpg)
A change in guidelines
41 Courtesy: Jonas Oldgren, UCR
CLASS III – DO NOT
use
CLASS IIa –USE
![Page 41: Session 3 Managing uncertainty in the post- launch phase · 1. Evidence generation timeline 2. Uncertanity of evidence 3. Managing uncertainty ... technology should be weighed against](https://reader033.vdocuments.us/reader033/viewer/2022060523/605299fbc1316e11ff4793e1/html5/thumbnails/41.jpg)
A change in clinical practice
42 Courtesy: Jonas Oldgren, UCR
![Page 42: Session 3 Managing uncertainty in the post- launch phase · 1. Evidence generation timeline 2. Uncertanity of evidence 3. Managing uncertainty ... technology should be weighed against](https://reader033.vdocuments.us/reader033/viewer/2022060523/605299fbc1316e11ff4793e1/html5/thumbnails/42.jpg)
Conclusions
• The benefits of introduction of a new technology should be weighed against inevitable uncertainty of evidence
• Approval of new technologies on the basis of limited evidence should result in risk management plan, including systematic collection of real world data and pragmatic clinical trials, to reduce uncertainity.
43
![Page 43: Session 3 Managing uncertainty in the post- launch phase · 1. Evidence generation timeline 2. Uncertanity of evidence 3. Managing uncertainty ... technology should be weighed against](https://reader033.vdocuments.us/reader033/viewer/2022060523/605299fbc1316e11ff4793e1/html5/thumbnails/43.jpg)
Conclusions
• Adequate quality real world data are fundamental to judge the clinical benefits related to the use of new health technologies compared to existing ones
• Joint clinical assessments should be coordinated, but not aligned with CE marking, to allow for the collection of real world data
• A coordinated effort should be undertaken to improve the quality of data available for HTA
44