incorporating virtual patients into clinical studies€¦ · human •integrate the virtual patient...

1

Incorporating Virtual Patients into Clinical Studies

Adam Himes, Tarek Haddad, Medtronic

Laura Thompson 1 , Telba Irony2 , Rajesh Nair1

1CDRH / FDA, 2CBER / FDA

on behalf of MDIC working group colleagues:

Dawn Bardot, MDIC / Medtronic

Anita Bestelmeyer, BD

Dan Cooke, Boston Scientific

Mark Horner, ANSYS

Russ Klehn, St. Jude Medical

Tina Morrison, OSEL / FDA

Kyle Myers, OSEL / FDA

Val Parvu, BD

MDIC.org

2 http://www.nejm.org/doi/full/10.1056/NEJMra1512592

“If it can be shown that these virtual patients are similar, in a precisely defined way, to

real patients, future trials may be able to rely partially on virtual-patient information,

thus lessening the burden of enrolling additional real patients.”

3

Outline

• Motivation

• Virtual patients: physical + probabilistic modeling

• Combining virtual patients with clinical data

• Test drive: mock submission

• Review

4

Motivation

Rising Demands:

• Duration

• Size

• Data

New Factors:

• Diseases

• Markets

• Cost ModelsJ. Diabetes Sci Tech (2009) 3(1) 44-55

Murbach, et.al. 2017 BMES/FDA Frontiers

Lee, et.al. 2017 BMES/FDA Frontiers

Dharia, et.al. 2016 BMES/FDA Frontiers

Kuntz, Insights on Global Healthcare Trends (2013)

Global demand:C

OS

T

15% year/year

TIME

Modeling advances:

Our ability to simulate clinical

outcomes has never been better

Demand for clinical evidence has

never been higher

Yet it is still challenging to incorporate prior information into new studies

5

Virtual Patients as a New Source of Evidence

Bench

Animal

Human

Computer

Traditional

Virtual Patient

Computer

Bench

Animal

Human

• Integrate the virtual patient in clinical study design

• Use Bayesian statistical methods

• Build on 2010 FDA Guidance

• Maintain clinical and statistical rigor

Future

6

What Makes a Virtual Patient?

Physical Modeling

Probabilistic Modeling

Clinically Relevant

Predictions

Variability:

• Age

• Gender

• Activity level

• Implant factors

• Physical tolerances

Uncertainty:

• Sample size

• Measurement error/bias

• Model bias

Well Characterized Physics:

• Structural

• Electrical

• Heat transfer / fluid flow

Knowledge of Biology /

Physiology:

• Local device ↔ tissue

interactions

• Failure modes

• Insulin response

Clinically Relevant End

Points:

• ICD lead fracture

• Orthopedic implant

survival

• Coronary artery flow

• MRI heating

• Cardiac rhythm

detection

• Blood glucose level

Like running a

virtual clinical

study!

7

Sources of information for virtual

patient models

Virtual patient outcomes have to be exchangeable with

something you’d measure in a clinical study.

• Historical data

−Pilot studies, other geograhies, similar predicate products

• Real-world data

−Electronic medical records, claims data, observational studies

• Engineering and physiological models

Credibility is the biggest hurdle.

8

Virtual Patient Example: ICD Lead Fracture

• Medical devices are particularly well suited to virtual patient modeling

− Local vs. systemic, often iterative, method of action is usually well understood

• Many applicable models, implantable defibrillator lead fracture is a good example

− Relevant, simple, public domain examples

Haddad, et.al., Reliability Engineering and System Safety, 123 (2014): 145-157.

Swerdlow, et.al., JACC, 67 (2016): 1358-1368.

9

Virtual Patient Example: ICD Lead Fracture

• Simulate many combinations of virtual patients

& clinical trial

• Propagate variability and uncertainty to

predict survival with confidence bounds

Field data

Projection with

95% Confidence

Interval

in-vivo

bending

patient

activity

fatigue

strength

INPUT OUTPUT

Haddad, et.al., Reliability Engineering and System Safety,

123 (2014): 145-157

10

Bayesian Statistical Methods

• How much influence is given to

the prior data?

• What if the clinical study data

disagrees?

Challenges: Solution:

• Method developed by MDIC

working group

• Influence of prior data determined

by agreement with study data

• Maintain statistical power with

fewer patients

influen

ce

disagree agreedisagree agree

(ideal state)

prior data

study data

discount

function

Provide a way to incorporate prior data into analysis of a clinical study

Haddad, et.al. (2017). J. Biopharm Stat, 27(6), 1089-1103.

DOI: 10.1080/10543406.2017.1300907

11

Using a Discount Function to Control

the Influence of Virtual Patients

• Influence determined by agreement

between real patients and virtual patients

− Defined before starting the clinical study

− Maximum depends on model maturity

− Shape of function depends on desired characteristics

• If virtual and real patients disagree:

− Number of virtual patients decreases

− Eventually converts to a traditional study

• If virtual and real patients agree:

− Number of virtual patients increases up to pre-

specified maximum

Vir

tual patient

weig

ht

Statistical

agreement

agreedisagree

more

conservative

less

conservative


DOI: 10.1080/10543406.2017.1300907

12

Incorporate Virtual Patients: Step By Step

1. Compare virtual

patient and current data

2. Compute strength of

prior using discount function

3. Combine virtual

patient and current data

4. Statistical analysis

using combined data

virtual patient data

current data

𝑝

𝑝

𝛼0

𝑛𝑡𝑜𝑡𝑎𝑙 = 𝑛𝑐𝑢𝑟𝑟𝑒𝑛𝑡 + 𝛼0𝑛𝑉𝑃

combined data

This part

is new


DOI: 10.1080/10543406.2017.1300907

13

Implementation: Mock Submission

• Collaboration between MDIC and FDA CDRH Division of Cardiovascular Devices

• Demonstrate the engineering and statistical framework for virtual patients

• MDIC sponsor team includes industry and FDA

• FDA review team, just like for a real device

http://mdic.org/computer-modeling/virtual-patients/

2014 2015 2016

MDIC

working

group

formed

Mock

submission

team identified

Mock

submission

informational

meeting at FDA

2nd Mock

submission meeting

(engineering model)

at FDA

3rd Mock

submission

meeting (clinical

study) at FDA

FDA

commitment 2017

MDIC project

initiated to

develop methods

for historical data


14

Mock submission details

• Hypothetical new ICD lead

− Changes to insulation thickness and conductor

material, both affect fatigue life

− Expected fracture rate <1% at 18 months

− Single anatomical zone, single failure mode

• Clinical study design

− Objectives

• Fracture rate at 18 months < 3%, type I error < 10%

− Enrollment

• 200 initial patients, interim look every 30, maximum 400

patients. Up to 160 virtual patients (40%)

− Analysis

• No virtual patients, fixed amount, and with a discount

function



15

Results

• Traditional study with no virtual

patients is underpowered

− 60% power / 3% type I error

• Fixed number of virtual patients

has unacceptable type I error

− 96% power / 25% type I error

• Studies using virtual patients with

a discount function have

acceptable power AND type I error

− Function #1: 80% power / 5% type I error

− Function #2: 85% power / 10% type I error

Power = % chance of success that you deserve

Type I error = % chance of success you don’t deserve

lower bound of virtual

patient model

performance

performance

requirement

Type I error

suffers without

discount

functionPower suffers

without virtual

patients



16

What did we learn?

• Three year process, required a very high level of collaboration

• Stakeholders have to engage in a different way

− Statistician, engineer, clinician, regulator all have a role in the study design

− The bandwidth of the regulatory communication process is a challenge – engage early!

• Agreement on credibility of the prior is the most important topic

− Discount function allows for scaling according to prior credibility

− Introduces a different lens for evaluating historical data and engineering models

• All material available at http://mdic.org/computer-modeling/virtual-patients

2014 2015 2016

MDIC

working

group

formed

Mock

submission

team identified

Mock

submission

informational

meeting at FDA

2nd Mock

submission meeting

(engineering model)

at FDA

3rd Mock

submission

meeting (clinical

study) at FDA

FDA

commitment 2017

MDIC project

initiated to

develop methods

for historical data

http://mdic.org/computer-modeling/virtual-patients

17

Model Credibility

• Level of trust in the model is driven by credibility.

• Credibility comes from verification and validation.

• ASME V&V40 risk-informed credibility

assessment methodology

− Model influence: the contribution of the model to

the decision relative to other available evidence

− Decision consequence: the significance of an

incorrect decision (related to the device)

− Model Risk: combination of model influence and

decision consequence for a context of use

18

Summary

• Virtual patients can improve the clinical decision process while exposing

fewer patients to clinical trials

• Bayesian design with a discount function controls the influence of virtual

patients

• The statistical methods are ready – we just need the right applications!

• Model credibility is the most important thing

incorporating virtual patients into clinical studies€¦ · human •integrate the virtual patient...

Documents