mike fino miracosta college statistical issues in biomanufacturing: clinical trials and lot...

MIKE FINOMIRACOSTA COLLEGE

Statistical Issues in Biomanufacturing:

Clinical Trials and Lot Acceptance

1

The motivation for our discussion

http://news.sciencemag.org/sciencenow/2009/10/30-01.html

SCIENCE covered the results of the 2009 AIDS Thai trial, which resulted in an analysis between the vaccine and placebo group with a p-value of 0.04

A prominent AIDS vaccine researcher declared

The probability that this vaccine didn’t work was only 4%

2

Which of these correctly state the p-value?

There's less than a 5% chance that the results are misleading

A p-value of 0.05 means that the null hypothesis has a probability of only 5%

Given a result with p ≤ 0.05, we state that there is a 95% or greater chance that the null hypothesis is incorrect

There's a less than 5% chance that I'm going to say this worked when it really didn't.

The p-value cut off of 5% means we have 95% confidence that this is real.

3

A Phase II Clinical Trial

A Study to Investigate the Neuroprotective Effect of PROCRIT (Epoetin Alfa) Versus Placebo in Cancer Patients Who Develop

Chemotherapy-induced Peripheral Neuropathy

The purpose of this study is to evaluate the neuroprotective effect of PROCRIT (epoetin alfa, a glycoprotein that stimulates red blood cell production) versus placebo in patients with cancer who develop chemotherapy-induced peripheral neuropathy due to combination Taxane and Platinum-Based treatment.

SOURCE: http://www.clinicaltrials.gov/ct2/show/NCT00267007

4

Study Design5

The Prevention

Peripheral neuropathy is a debilitating disease of the nerves which can be a dose-limiting toxicity of chemotherapeutic agents.

The symptoms of peripheral neuropathy can lead to considerable patient distress and discomfort, discontinuation of chemotherapy, and limitations regarding the selection of future chemotherapeutic regimens.

Symptoms such as numbness, weakness, burning pain (especially at night), and loss of reflexes may take months before they improve and permanent deficits may remain.

Epoetin alfa, already used in the treatment of chemotherapy-induced anemia, has been shown to have neuroprotective effects in preclinical studies.

6

Primary Measure -- Results

Measure Title The Number of Patients Who Developed Peripheral

Sensory Neuropathy (National Cancer Institute Common Toxicity Criteria (NCI CTC) Score >= 1) at Week 12.

Measure Description NCI CTC neuropathy: a descriptive terminology used

to grade the severity of AEs in cancer subjects on a 0-5 scale. A higher score indicates worse peripheral neuropathy.

7

Results

PROCRIT Placebo Totals

Developed Neuropathy

5 6 11

No Neuropathy 14 5 19

Totals 19 11 30

PROCRIT Placebo


26% 55%

8

Did PROCRIT have an effect?

Play the skeptic for a second. What could you say about this data? Do you think the researchers could have just

been “lucky” – that just by chance they got these results?

PROCRIT Placebo


26% 55%

9

ACTUAL RESULTS



5 6 11


Totals 19 11 30

NO EFFECTNO EFFECTRESULTSRESULTS


Developed Neuropathy 77 44 11


Totals 19 11 30

What we would expect, on average, by chance alone

Does the 5/6 split we got seem very different from the 7/4 expected?

10

Resampling Activity

We want to know: if the PROCRIT is truly not effective in helping, how often would the random assignment process alone would lead to such a difference between the treatment groups?

We will answer this question by replicating the randomization process all over again, but in a situation where we play the skeptic and assume that PROCRIT therapy is not effective.

We’ll start with 11 “afflicted” and 19 non-afflicted, and we’ll randomly assign 19 of these 30 subjects to the PROCRIT therapy group and the remaining 11 to the placebo group.

11

Resampling Activity

Everybody will receive a deck of cards. These cards match the patient outcomes if we believed that the therapy was NOT effective. 11 got neuropathy, 19 didn’t – what group they end up in is random and the “treatment” in the group does nothing to affect their outcome.

Pick 11 FACE CARDS to represent those with neuropathy

Pick 19 NUMBER CARDS to represent no neuropathy

We want to simulate what the skeptic believes. That is, how many people might we expect – BY CHANCE ALONE – to be in the PROCRIT Group that developed neuropathy. This is the number that you’re going to be tracking through FIVE rounds of resampling.

12

Resampling Activity: Efficacy

1. Count out 11 face cards and 19 non-face cards. Set the remaining cards aside.

2. You will only be resampling the PROCRIT group because the number of PROCRIT-Neuropathy is the only number we’re going to pay attention to.

3. Shuffle, shuffle, then shuffle some more.4. Deal 19 cards out – this will represent the PROCRIT

Therapy group.5. Count how many have neuropathy (face cards) in the

PROCRIT group. Write this number down.6. Shuffle, shuffle, then shuffle some more.7. Repeat (3) through (5) until you have five numbers.

8. Go to the front of the class and mark your numbers on the class graph.

13

Resampling Activity

We want to know, if the null hypothesis is true, how often would the random assignment process alone lead to such a large difference between the treatment groups?

14

Results with 10,000 resamplings15

Safety Data with PROCRIT Study


Serious Adverse Event Reported

6 1 7

No SAE 13 10 19

Totals 19 11 30

PROCRIT Placebo

SAE Reported 32% 9%

Do we have a safety problem here?

16

ACTUAL RESULTS


Serious Adverse Event Reported

6 1 7

No SAE 13 10 23

Totals 19 11 30



Serious Adverse Event Reported 4.44.4 2.62.6 7

No SAE 14.614.6 8.48.4 23

Totals 19 11 30


Does the 6/1 split we got seem very different from the 4.4/2.6 expected?

17

Resampling Activity: Safety

1. Count out 7 face cards and 23 non-face cards. Set the remaining cards aside.

2. You will only be resampling the PROCRIT group because the number of PROCRIT-SAE events is the only number we’re going to pay attention to.

3. Shuffle, shuffle, then shuffle some more.4. Deal 19 cards out – this will represent the PROCRIT

Therapy group.5. Count how many have an SAE (face cards) in the

PROCRIT group. Write this number down.6. Shuffle, shuffle, then shuffle some more.7. Repeat (3) through (5) until you have five numbers.

8. Go to the front of the class and mark your numbers on the class graph.

18

The Outcome20

Effect of Erythropoietin (EPO) in Kidney After Cardiac Surgery (EPO-CABG)

The purpose of this study is to determine whether erythropoietin is effective in preventing acute kidney dysfunction after coronary artery bypass grafting surgery.

SOURCE: http://www.clinicaltrials.gov/ct2/show/NCT00654992

21

EPO and AKI

Acute kidney injury (AKI) occurs in 7% to 40% of patients undergoing cardiac surgery, depending on the definition of AKI used.

Even small increments in serum creatinine have been shown to be associated with increased mortality after cardiac surgery.

However, there are no proven interventions to prevent AKI after cardiac surgery.

Erythropoietin (EPO) has been shown to have tissue-protective effects in various experimental models.

In this prospective, placebo-controlled, randomized trial, we evaluated the effectiveness of EPO in the prevention of AKI after coronary artery bypass grafting (CABG).

22

Study Design23

Study Results

EPO Placebo Totals

AKI Observed 3 10 13

No AKI 33 25 58

Totals 36 35 71

EPO Placebo

AKI Observed 8% 29%

24

Here we go again?

EPO Placebo

AKI Observed 8% 29%

PROCRIT Placebo

SAE Reported 32% 9%

Similar disparity between groups.Are we expecting a similar

conclusion?

25

ACTUAL RESULTS

EPO Placebo Totals

AKI Observed 3 10 13

No AKI 33 25 58

Totals 36 35 71


EPO Placebo Totals

AKI Observed 6.66.6 6.46.4 13

No AKI 29.429.4 28.628.6 58

Totals 36 35 71


Does the 3/10 split we got seem very different from the 6.6/6.4 expected?

26

The Approach to Hypothesis Testing27

Model what the data would like, if the null were true

Compare our actual results to the nullAsk whether our data would be expected or

unexpected in the model Expected data consistent with null (e.g. p-value

greater than 5%) Unexpected data rejects the null (e.g. p-value less

than 5%)

29

Quality Control for a Cell-Based Therapeutic

BREAK

Tissue Engineering30

Processes which combine

living cells and biomaterials

to generate tissue in vitro or in vivo

for therapeutic implantation

or as diagnostic reagents

Process Overview31

Diabetic Ulcers32

People with advanced diabetes generally suffer from poor circulation in their extremities

Small cuts cannot resolve themselves due to inadequate delivery of nutrients

Neuropathy, an attendant complication, often allows the patient to “live with” the ulcer

It is not uncommon for patients to finally visit the doctor after having the ulcer for many months – sometimes years!

Angiogenic Foundation33

0

200

400

600

800

1000

Mic

rove

ssel

Mic

rove

ssel

Den

sity

Den

sity

(ves

sels

/ s

qu

are

mm

)(v

esse

ls /

sq

uar

e m

m) *p < 0.05*p < 0.05

Dermagra

ft

Dermagra

ft

NonNon--V

iable

ViableDerm

agraft

Dermagra

ft

ShamSham

0

200

400

600

800

1000**

Mic

rove

ssel

Mic

rove

ssel

Den

sity

Den

sity

(ves

sels

/ s

qu

are

mm

)(v

esse

ls /

sq

uar

e m

m) *p < 0.05*p < 0.05

Dermagra

ft

Dermagra

ft

NonNon--V

iable

ViableDerm

agraft

Dermagra

ft

ShamSham

DERMAGRAFT® Growth Time Course34

2 Days

1 Day

8 Days

3 Hrs

DERMAGRAFT® & Diabetic Foot Ulcers

35

Week 0

Week 5

Week 1

Week 8

Week 3

Week 10

Dermagraft implanted weekly for eight weeks

Batch Production36

Scale-up

Physician cutting piece

of skin

Manifold: 96 units

Each batch: Four manifolds

Dermagraft Production Overview37

Product Quality38

The FDA mandates that quality attributes are tested against pre-determined specifications to ensure

Purity Potency Consistency Identity Stability

Safety

Efficacy

Product Quality39

Safety and efficacy were both proven during the final (Phase III) clinical trial

Quality Control is the group that’s responsible for conducting the laboratory tests to demonstrate the product made today is the same as the Phase III material

Assays Related to Efficacy40

MTT A metabolic assay that’s generally viewed as a

measure of the number of viable cells

Sirius Red Colorimetric stain measuring collagen content – an

essential EM protein

DNA For a populated neo-dermis this is a relatively stable

measure of cell count

Assays Related to Safety41

Sterility A measure of asepsis

Endotoxin A measure of LPS from gram (-) bacteria

Mycoplasma Cell culture contaminant

Assay Specifications42

Each assay had its own specified limits of what’s acceptable

To measure the product against the specifications acceptance sampling is used A composite percent defective score from the

ANSI/ASQ Z1.9 Acceptance Sampling for Variables is calculated

This standard is a measure that’s making a check on the batch’s central tendency and its variability relative to the defined limits

Acceptance Sampling43

The idea is rooted in inferential statistics Get a random sample from a large population (batch

of product) Analyze it (assays) Make a decision on the properties of the larger batch

based on the sample

So what’s involved44

Determine sample size code based on batch size

Determine the maximum allowable percent defective, M (based on AQL)

Randomly sample and analyzeCompute sample mean and std devCompute quality indices: QU and QL

Find their associated percent defective: pU and pL

Compare pU + pL to M

QU and QL45

s

XUQU

s

LXQL

U is the upper limit of the specification for the assay

L is the lower limit of the specification for the assay

X is the sample average

s is the sample std deviation

mike fino miracosta college statistical issues in biomanufacturing: clinical trials and lot...

Documents

worse peripheral neuropathy

results measure title

cancer patients

treatment of chemotherapy

cancer subjects

greater chance

number of patients

placebo group