biostatistics in development of medical devices by t. mudde - clinquest (qserve conference 2013)
Post on 19-Oct-2014
604 views
DESCRIPTION
TRANSCRIPT
Fulfilling the Promise of Medicine Together
Clinquest
CLINQUEST SERVICES
Biostatistics in
Development of Medical Devices
2 Clinquest Services Qserve Conference
OUTLINE
3 Clinquest Services Qserve Conference
• Guidelines
• Clinical Investigations
(Statistical Section) Protocol
• Precision and Bias
• Adaptive Designs
EUROPEAN GUIDELINES
Clinquest Services Qserve Conference 4
“Clinical data must be provided for ALL medical devices. This may be
literature review or clinical investigation depending on device class and
use.”
When must/should a clinical investigation be undertaken?
To ensure a high level of safety and performance, demonstration of
compliance with the general safety and performance requirements
should be based on clinical data that, for class III medical devices and
implantable medical devices should, as a general rule, be sourced
from clinical investigations to be carried out under the responsibility of
a sponsor who can be the manufacturer or another legal or natural
person taking responsibility for the clinical investigation.
Depending on clinical claims, risk management outcome and on the
results of the clinical evaluation, clinical investigations may also have
to be performed for non-implantable medical devices of classes I, IIa
and IIb.
FDA GUIDELINES
Clinquest Services Qserve Conference 5
The collection and evaluation of sound clinical data are the basis of the approval process for many medical devices.
Goals for device studies (including IVD studies) are:
• Producing valid scientific evidence, demonstrating
reasonable assurance of the safety and effectiveness of the
product
• Protecting the rights and welfare of study subjects
FDA GUIDELINES
Clinquest Services Qserve Conference 6
Valid scientific evidence is defined as:
“Evidence from well-controlled investigations, partially controlled
studies, studies and objective trials without matched controls,
well-documented case histories conducted by qualified experts,
and reports of significant human experience with a marketed
device, from which it can fairly and responsibly be concluded by
qualified experts that there is a reasonable assurance of the
safety and effectiveness of a device under its conditions of use.”
GUIDELINES
Clinquest Services Qserve Conference 7
Clinical Investigation Plan
The clinical investigation plan (CIP) shall define:
• Rationale of the clinical investigation
• Study objectives
• Study design
• In/exclusion criteria
• Endpoint choice
• Proposed analyses: statistical considerations
• Monitoring
• Conduct and record-keeping: data management
GUIDELINES
Clinquest Services Qserve Conference 8
Statistical Section Clinical Investigation Plan
The statistical section should at least include:
• Analysis populations used
• Missing data strategy
• Definition of endpoints
• Hypotheses to be tested (efficacy/safety endpoints)
• Planned statistical methodology
• Sample size justification
• Sensitivity analyses planned
• Planned interim analyses
• Detailed Statistical Analysis Plan
Clinquest Services Qserve Conference 9
INTERACTION SPONSOR- BIOSTATISTICIAN
Study Design Endpoint Choice
Sample Size Hypotheses
Study Objective
Bias
Precision
PRECISION / BIAS
Clinquest Services Qserve Conference 10
Precision small
Bias large Precision large
Bias large
Precision small
Bias small
Precision large
Bias small
X x X x x x
x x x x x x
X x X x x x
x x x x x x
PRECISION / BIAS
Clinquest Services Qserve Conference 11
• Precision: random fluctuations
Maximizing precision by:
- work with large samples
- work with homogeneous samples
- use paired designs (patient/sample is its own control)
- use extra information, e.g. baseline measurements
• Bias: systematic deviate from true result
Minimizing bias by:
- randomization (decreases selection bias and impact of confounding)
- blinding (decreases observation bias)
- use extra information, e.g. baseline measurements (decreases impact of confounding)
- avoid missing data
N
ses ..
EXAMPLE STENT STUDY
Clinquest Services Qserve Conference 12
Comparison of Stenting versus Balloon Angioplasty (PTA) for the
treatment of below the knee artery disease (Class III).
Efficacy Study Objective: - Does our Stent perform better than balloon angioplasty?
Ethical, control standard care
Design: - Parallel Arm, Multiple Centers
- Superiority Trial
- Randomized? Yes
- Blinded? No
- Duration FU? Long enough for acceptable evaluation of
performance and safety (1 year)
- Interim Analyses? No
Primary efficacy endpoint: - Binary in-segment stenosis at 12 months by angiography
STENT STUDY
Clinquest Services Qserve Conference 13
Possible bias:
- Objective primary endpoint
- Observation bias, blinding not possible
- Use ITT analysis population in case of
randomized study
- Confounding possible?
- Missing data
ANALYSIS DATASETS
Clinquest Services Qserve Conference 14
• ITT Analysis set: All randomized patients; analyzed as
randomized.
• Per Protocol set: Set of patients with minimal violations against
the protocol as: errors in treatment assignment,
use of excluded medication, poor compliance,
loss to follow-up and missing data; analyzed as
treated.
To be set before database lock.
• Full Analysis set: The analysis set that is as complete as
possible and as close as possible to the
intention to treat ideal of including all
randomized subjects (ICH).
ANALYSIS DATA SETS
Clinquest Services Qserve Conference 15
• ITT 50 patients on stent 50 patients on angioplasty
• PP 55 patients on stent 45 patients on angioplasty
PP analysis generally used as sensitivity analysis
R
Angioplasty N1=50
5 patients too severe
Stent N2=50
5 patients go over on
stenting
Hospital
A
B
Total
Died 63 16 79
Survived 2037 784 2821
Total
Mortality rate
2100
3.0%
800
2.0%
2900
BIAS BY CONFOUNDING VARIABLES
Clinquest Services Qserve Conference 16
Mortality rate comparison
Relative risk hosp. B vs. hosp. A = 0.66,
34% lower risk of dying in hosp. B than in
hosp. A
Good
A
Health
B
Total
Poor
A
Health
B
Total
Died 6 8 14 57 8 65
Survived 594 592 1186 1443 192 1635
Total
Mortality rate
600
1.0%
600
1.3%
1200 1500
3.8%
200
4.0%
1700
Relative risk hosp. B vs. hosp. A = 1.33,
33% higher risk of dying in hosp. B than
in hosp. A
Relative risk hosp. B vs. hosp. A = 1.05,
5% higher risk of dying in hosp. B than
in hosp. A
BIAS BY CONFOUNDING VARIABLES
Clinquest Services Qserve Conference 17
Good
A
Health
B
Total
Poor
A
Health
B
Total
Died 6 8 14 57 8 65
Survived 594 592 1186 1443 192 1635
Total
Mortality rate
600
1.0%
600
1.3%
1200 1500
3.8%
200
4.0%
1700
Mortality rate confounded by health status because :
- Health status has impact on outcome
- Health status not equally distributed over the two hospitals
Statistical analysis: Adjusted relative risk hospital B vs. hospital A = 1.14,
14% higher risk of dying in hospital B than in hospital A, taking health status
into account.
BIAS BY CONFOUNDING VARIABLES
Clinquest Services Qserve Conference
18
• Identical distribution of prognostic factors over treatment groups
(stratified randomization)
• Subgroup analyses (problem of multiple testing and low power)
• Retrospective control for prognostic/confounding factors during
statistical analysis
HOW TO COPE WITH PROGNOSTIC/CONFOUNDING FACTORS ?
FACTORS?
Confounding variable is a background factor which:
– is not equally distributed over the risk groups
– influences the outcome variable(s)
Primary efficacy endpoint:
Try to reject H0
-If p-value < 0.05 reject H0: there is a statistically significant therapy effect
-If p-value ≥ 0.05 accept H0: there is no statistically significant therapy effect.
H0 : π A = π B no stent therapy effect π A π B 0
HA : π A π B stent therapy effect π A π B 0
STENT STUDY, HYPOTHESES TO BE TESTED
Clinquest Services Qserve Conference 19
STENT STUDY, STATISTICAL INFERENCE
Clinquest Services Qserve Conference 20
ITT Population
-40.0%
-30.0%
-20.0%
-10.0%
0.0%
10.0%
20.0%
30.0%
40.0%
50.0%P
erc
en
tag
e w
ith
S
ten
osis
Stent Treatment
PTA Treatment
Difference
STENT STUDY, STATISTICAL INFERENCE
Clinquest Services Qserve Conference 21
Diabetes
-60.0%
-50.0%
-40.0%
-30.0%
-20.0%
-10.0%
0.0%
10.0%
20.0%
30.0%
40.0%
50.0%
60.0%
StentTreatment
PTATreatment
-60.0%
-50.0%
-40.0%
-30.0%
-20.0%
-10.0%
0.0%
10.0%
20.0%
30.0%
40.0%
50.0%
60.0%
StentTreatment
PTATreatment
Non-diabetes
p-value<0.001
p-value=0.313
Stent PTA
Diabetes 64.6% 64.4%
Diabetes: prognostic factor, not confounding
STENT STUDY, MISSING DATA
Clinquest Services Qserve Conference 22
Therapy
Stent (N=113)
PTA (N=115)
Total
Restenosis 15 (22.4%) 31 (41.9%) 46
No Restenosis 52 43 95
Total
Missing
67
46 (40.7%)
74
41 (35.7%)
141
87 (38.2%)
worst case
best case
Restenosis:
23 Stent,
21 PTA
STENT STUDY, SAMPLE SIZE CALCULATION
Clinquest Services Qserve Conference 23
Stent study:
With 61 lesions in each therapy arm, a clinically meaningful difference of 25% in 12 months in-segment restenosis rate can be detected with a power of 80% at a two sided significance level of 5%, assuming a 12 months restenosis rate of 45% in the group of subjects who only receive balloon angioplasty. The total sample size was increased to almost 110 lesions per therapy arm to account for a 45% drop out rate.
Therapy
Stent (N=113)
PTA (N=115)
Total
Restenosis 15 (22.4%) 31 (41.9%) 46
No Restenosis 52 43 95
Total
Missing
67
46
74
41
141
87
Drop out rate: 38.2%
Power to detect 25% difference = 85.8%
STENT STUDY, SAMPLE SIZE CALCULATION
Clinquest Services Qserve Conference 24
• Performed on the basis of assumptions concerning:
- How accurate can the primary endpoint be measured?
- What is the clinically significant difference you like to see
(HA)?
- What do you expect as outcome for the control group?
• Performed on the basis of agreements about:
- significance level: (e.g. 5%)
- power: 1- (e.g. 90%)
STENT STUDY, SAMPLE SIZE CALCULATION
Clinquest Services Qserve Conference 25
Hypothesis Testing Type I error / Type II error
(Type I error)= Significance level P(reject H0 | H0 is true) = P(false positive)
of the test
(1 - ) = Power of the test: P(reject H0 | HA is true)
(Type II error): P(accept H0 | HA is true) = P(false negative)
Power: Probability to accept correctly, by a given , the alternative
hypothesis.
H0 HA
ADAPTIVE DESIGNS
Clinquest Services Qserve Conference 26
Adaptive Design Clinical Study: a study that includes a
prospectively planned opportunity for modification of one or more
specified aspects of the study design and hypotheses based on
analysis of data (usually interim data) from subjects in the study
(FDA).
Goal: to make the study more efficient: shorter duration, fewer
patients, more information
Possible problems:
Operational bias: revisions not previously planned and made or proposed after an un-blinded interim analysis raise major concerns about study integrity.
Multiple testing: control of type I error rate
Regulatory concerns: FDA communication/review needed
ADAPTIVE DESIGNS
Clinquest Services Qserve Conference 27
Planned Adaptive Design Fixed Sample Design
• Allocation Rule
- Can be fixed but can change based - Randomization remains
on accruing data fixed throughout the study
• Sampling Rule
- How many subjects sampled at next - Only one stage
stage (cohort)
- Sample size recalculation based on - Fixed sample size
interim results
• Stopping Rule
- When to stop a trial: efficacy, futility - No early stopping
• Decision Rule
- E.g. dropping study arm - No changes
EXAMPLE CIN-EVENT STUDY
Clinquest Services Qserve Conference 28
Comparison experimental device system with standard hydration
protocol for preventing the incidence of CIN (Contrast Induced
Nephropathy), after the administration of contrast media. (Class
IIb).
Efficacy Study Objective: - Evaluation of the comparability between experimental device and
standard care in CIN events within 3 days post contrast administration.
Design: - Parallel Arm, multiple centers
- Superiority Trial
- Randomized? Yes, stratified by Y/N NSTEMI
- Blinded? No
- Duration FU? 3 days for primary efficacy endpoint, 90 days for safety
- Interim Analysis? Yes
Primary efficacy endpoint: - CIN event within 3 days; missing information for CIN: failure
CIN-EVENT STUDY, SAMPLE SIZE CALCULATION
Clinquest Services Qserve Conference 29
Assumptions:
3-day CIN event rate control arm: 15%
Assuming 80% power and with a 1:1 randomization allocation ratio, 155 subjects in each group are required to demonstrate the expected difference of 10% in the incidence of CIN between groups at a significance level of 0.05 based on a 2-sided test.
The study will randomize a total of 326 patients to account for a 5% lost to follow-up but based on the interim analysis, could randomize up to a maximum of twice the initial sample size, or 652 patients.
CIN-EVENT STUDY, SAMPLE SIZE RECALCULATION
Clinquest Services Qserve Conference 30
1
1ˆ Measured difference at interim analysis
Expected difference at final analysis
CIN-EVENT STUDY, SAMPLE SIZE RECALCULATION
Clinquest Services Qserve Conference 31
Sample size recalculation will be performed based on the method of modification of sample size in group sequential clinical trials employing conditional power and maintaining type I error rate. The sample size will be adjusted such that the calculated conditional power obtained is 80%.
The trial can be stopped for reasons of overwhelming efficacy (p-value<0.003) or futility (conditional power under the current trend <20%).
QUESTIONS
Clinquest Services Qserve Conference 32
?