statistical approaches to meeting emerging usp guidelines ... · 7/6/2010 · d. lansky...
TRANSCRIPT
©
D. Lansky
Introduction
Equivalence
Scaling
Summary
Statistical Approaches to MeetingEmerging USP Guidelines for BioassayDevelopment, Analysis, and Validation
David Lansky, Ph.D.
Burlington, Vermont, USA
May 12, 2010
1 / 29
©
D. Lansky
Introduction
Equivalence
Scaling
Summary
Bioassay Background
I Parallel Line: Shapes similar assume:- same cmpd⇒ common slope & asymptotes- interpret horizontal shift=log potency
I Slope Ratio: Intercepts similar assume:- same cmpd ⇒ common y intercept- interpret slope ratio=potency
Parallel Line Dose on log scale
Re
sp
on
se
-1.5
-1.0
-0.5
0.0
0.5
5 10 20 40 80 160 320 640
R R RR
R
RR
R
TT T
T
T
T
TT
Slope Ratio Dose on arithmentic scale
Re
sp
on
se
6
8
10
12
14
16
1 2 3 4 5 6
R
RR
R
RR
T
T
T
T
T
T
2 / 29
©
D. Lansky
Introduction
Equivalence
Scaling
Summary
Similarity in assay
Distance between curves
1 2 3 4 5 6 7 8 9 10 11 12
3 / 29
©
D. Lansky
Introduction
Equivalence
Scaling
Summary
Similarity in assay
Distance between curves
1 2 3 4 5 6 7 8 9 10 11 12
Do the two curves have the same shape?
4 / 29
©
D. Lansky
Introduction
Equivalence
Scaling
Summary
Similarity in assay
Distance between curves
1 2 3 4 5 6 7 8 9 10 11 12
Do the two curves have the same shape?
5 / 29
©
D. Lansky
Introduction
Equivalence
Scaling
Summary
Potency SD and Dose Range
I SDlog potency � SDlog ED50
I PGSD (= 100(eSD − 1)) 5%, 6x-15xED50 range 68%-216%
I product potency spec. often 0.71-1.41I Generally need 3 doses on steep partI doses often 1:2 dilutionsI SDlog ED50 & potency range: need 5
dilutions
6 / 29
©
D. Lansky
Introduction
Equivalence
Scaling
Summary
Transform not Weight
I Weight on dilution-specific variances?I SDlog ED50 inflates SD at middle dilutionsI Weighting confounds nonlinear mixed
model
0.5
1.0
1.5
2 4 6 8 10 12 7 / 29
©
D. Lansky
Introduction
Equivalence
Scaling
Summary
Outliers
I After transform (or weight)I Fit model (w/Design Structure) to all dataI Avoid shape assumptionsI Separate outlier detection from model
adequacy
8 / 29
©
D. Lansky
Introduction
Equivalence
Scaling
Summary
Design Structure
I Old recommendation: keep designsimple, drive bias into variance
I New: recognize design structureI Grouped (multi-channnel) and serial
dilution commonI pseudo-replicates (multiple aliquots from a
preparation)I strip-plot designs appearI (incomplete) block designs efficient
9 / 29
©
D. Lansky
Introduction
Equivalence
Scaling
Summary
Strip-Plot Design
ABCDEFGH
1 2 3 4 5 6 7 8 9101112
1.9 1.81.81.6 1.41.2 1.2 1.11.1
1.9 1.91.81.61.4 1.31.2 1.1 1.11.1
1.9 1.91.81.71.6 1.31.2 1.1 1.11
1.9 1.91.81.81.6 1.41.3 1.2 1.11.1
1.8 1.91.81.81.7 1.61.3 1.2 1.11.1
1.9 1.91.81.81.8 1.61.4 1.3 1.11.1
1.6
1
1 2 3 4 5 6 7 8 9101112
1.9 1.91.8 1.81.6 1.4 1.31.21.11.1
1.9 1.91.8 1.71.4 1.3 1.11.11.11.1
1.9 1.81.8 1.71.5 1.3 1.21.11 1
1.9 1.91.8 1.81.7 1.5 1.21.21.11.1
1.9 1.81.9 1.81.8 1.5 1.31.21.11.1
1.9 1.91.9 1.91.8 1.7 1.41.21.11.1
2
1 2 3 4 5 6 7 8 9101112
ABCDEFGH
1.91.91.9 1.81.61.5 1.3 1.21.21.1
1.91.91.8 1.71.51.3 1.2 1.11.11.1
1.91.91.8 1.71.61.4 1.3 1.21.11.1
1.91.91.9 1.81.61.5 1.3 1.21.21.1
1.921.9 1.91.71.6 1.4 1.21.21.1
1.91.91.9 1.91.81.6 1.5 1.31.11.1
3
1 2 3 4 5 6 7 8 9101112
ABCDEFGH
1.91.9 1.91.8 1.6 1.51.31.21.11.1
1.81.9 1.81.7 1.4 1.31.21.11.11.2
1.91.9 1.81.6 1.5 1.31.31.21.11
1.91.9 1.81.8 1.6 1.41.21.21.21.1
1.91.9 1.81.8 1.6 1.51.31.31.21.1
1.91.9 1.91.8 1.8 1.61.41.31.11.2
4
1 2 3 4 5 6 7 8 9101112
1.91.91.91.71.61.4 1.31.2 1.11
1.91.91.71.61.41.3 1.21.1 11.1
21.91.81.71.61.4 1.21.1 1.11.1
1.91.91.81.81.71.4 1.31.2 1.11.1
1.91.91.91.81.71.5 1.41.2 1.11
21.921.91.81.7 1.3 1.21.11.7
5
1 2 3 4 5 6 7 8 9101112
ABCDEFGH
1.91.9 1.81.81.6 1.41.21.1 1.1 1.1
1.91.9 1.71.61.4 1.21.11.1 1 1.1
1.91.9 1.81.71.5 1.31.21.1 1.1 1.1
1.91.9 1.71.81.6 1.41.21.1 1.1 1.1
1.91.9 1.91.81.7 1.51.31.2 1.1 1.1
1.91.9 1.91.81.8 1.61.41.2 1.1 1.1
6
10 / 29
©
D. Lansky
Introduction
Equivalence
Scaling
Summary
Strip-Plot Response
x
yhat
1.2
1.4
1.6
1.8
-10 -8 -6 -4 -2 0
R
R
RR
R
R
RR
RR
HH
H
H
H
HH
HHH
hh
hh
h
h
hh
hh
111
1
1
1
1111
ss
sss
s
s
ss
s
SSSS
S
S
S
S
SS
1
RRR
R
R
R
RR
RR
HH
H
H
H
H
HHHH
hh
hh
h
h
hhhh
1111
1
1
11
11
sssss
s
s
sss
SS
SS
SS
S
S
SS
2
-10 -8 -6 -4 -2 0
RRRR
R
R
R
RRR
HHH
H
H
H
HHHH
hhh
h
h
h
hh
hh
1111
1
1
111
1
ss
ss
s
s
s
sss
SSS
SS
S
S
S
SS
3
RRRR
R
R
R
RRR
HH
HH
H
HH
HHH
hhh
h
h
hhh
hh
1111
1
1
1111
ss
ss
s
s
ss
ss
SSS
SS
S
S
S
SS
4
-10 -8 -6 -4 -2 0
RRR
RR
R
R
RR
R
HH
HH
H
HH
HH
H
hh
hh
h
h
hhh
h
11
11
1
1
11
11
ss
ss
s
s
s
s
ss
SSS
SS
SS
S
SS
5
1.2
1.4
1.6
1.8
RRR
R
R
R
RR
RR
HH
H
H
H
H
HH
HH
hhh
h
h
h
hhhh
1111
1
1
1111
ssss
s
s
s
sss
SSS
SS
S
S
S
SS
6
11 / 29
©
D. Lansky
Introduction
Equivalence
Scaling
Summary
Strip-Plot Model
yijk =A + ak + aik − D
1 + e−B(xijk−(Ci+ck+cjk))+ D + εijk
independent : εijk ∼ iid N(0, σ2
)ak ∼ iid N
(0, σ2
ablock
), aik ∼ iid N
(0, σ2
arow
)ck ∼ iid N
(0, σ2
ablock
), cjk ∼ iid N
(0, σ2
acol
)with i for sample (row in block), j for dilution(column in block), and k block.
12 / 29
©
D. Lansky
Introduction
Equivalence
Scaling
Summary
Combine log Potencies
I ”weighted” and ”semi-weighted” assumebetween-assay σpotency = 0 (Finney, 1978)
I sampling SD of log potency safeI Link properties of reported value to
clinical need
13 / 29
©
D. Lansky
Introduction
Equivalence
Scaling
Summary
Validation Params & Methods
I Relative AccuracyI ”Linearity” of log potencyI Bias limit at each targetI Bias trend limit
I PrecisionI Components (repeatability, intermediate
precision, reproducibility)I Predict for various ”formats”
I SpecificityI (Robustness)I Equivalence used broadly
USP <1032>, <1033>, and <1034> will appear in PF 36(4)(early July). These and <111> athttp://www.usp.org/meetings/workshops/2010Bioassay.html
14 / 29
©
D. Lansky
Introduction
Equivalence
Scaling
Summary
Why Equivalence?
I Statistical similarity testedI Biological similarity assumed (stat.
similarity necessary, not sufficient)
I Assume critical differences known:I SlopeI Upper asymptoteI Lower asymptote
I Equivalence tests: ”Are reference andtest sufficiently similar”
I USP -> equivalenceI Curve parameters have meaning
I Critical quality attributesI Separate equivalence intervals
15 / 29
©
D. Lansky
Introduction
Equivalence
Scaling
Summary
Similarity: What is Needed
I Slope ONLY: lot releaseI Asymptote of max activity:
I compare standards (i.e.; new lot)I change productionI stability
I Asymptote of min activity: checks onlymatrix effects
16 / 29
©
D. Lansky
Introduction
Equivalence
Scaling
Summary
Difference testing for Similarity
I Practical ProblemsI Assays w/low variance fail parallelismI Assays w/high variance pass parallelism
I Theoretical Problem (one parameter)I Difference test
I H0 : βReference = βTest
I α (Type I) controls P(Falsely rejecting H0)I β∗ (Type II) controls P(Accepting H0|δ∗)
I Equivalence testI H0 :| βReference − βTest |> δ∗
I α controls Type I error of H0 : for δ∗
I In Practice: What is δ∗?
17 / 29
©
D. Lansky
Introduction
Equivalence
Scaling
Summary
Difference test for Similarity
Incorrectly sensitive to variance
Log(dose)
Response
-1
0
1
20 40 80 160 320
R
R
R
R
T
T
T
Tp< 0.03612
assay
R
R
R
R
T
T
T
T
p< 0.00577
assay
R
R
R
R
T
T
T
Tp< 0.53529
assay
20 40 80 160 320
-1
0
1
RR
R
R
T
T
T
Tp< 0.07369
assay
18 / 29
©
D. Lansky
Introduction
Equivalence
Scaling
Summary
Equivalence test for Similarity
Correctly sensitive to variance90% CI inside indifference zone⇒ equivalence95% CI does not include 0 ⇒ difference
Test slope as % of Reference
1
2
3
4
50 100 200
Log(dose)
Response
-101
20 40 80 160 320
RR
RR
TT
TTp< 0.03612
assay
-101
RR
RR
TT
TT
p< 0.00577
assay
-101
RR R
R
T TT
Tp< 0.53529
assay
-101
R RR
R
TT
T Tp< 0.07369
assay
19 / 29
©
D. Lansky
Introduction
Equivalence
Scaling
Summary
What can we say about δ∗?
I What if we knew δ∗?I Assay dependentI Scale dependentI Parameter link to quality attribute weakI Standardize meaning of δ?
20 / 29
©
D. Lansky
Introduction
Equivalence
Scaling
Summary
Seeking Scale Invariance
I Assays have different responsesI Asymptotes in response units
I Slope units logit(response)log(dose)
21 / 29
©
D. Lansky
Introduction
Equivalence
Scaling
Summary
Simple Scale Invariance
y ∗ =Ai
1 + e−Bi (log(x)−Ci )+ Di
with C = log (ED50) and i = [Ref|Test](Ratkowsky & Reedy, 1986)
Range= A: %∆A = 100ATest−ARef
ARef
Lower Asy= D: %∆D = 100DTest−DRef
ARef
Slope= B : %∆B = 100BTest−BRef
BRef
Concerns:I Is meaning consistent?I Are these useful across assays?I Variances of %∆A, %∆B , and %∆D
22 / 29
©
D. Lansky
Introduction
Equivalence
Scaling
Summary
Interpretation of %∆A:B :D
A
B
D
A in {2A/3,A, 3A/2}, %∆A = 10B in {B/3,B , 3B}, %∆B = 50D in {2D/3,D, 3D/2}, %∆D = 10
23 / 29
©
D. Lansky
Introduction
Equivalence
Scaling
Summary
Interpretation of %∆A:B :D
I scaling mostly worksI Requires some explaining
24 / 29
©
D. Lansky
Introduction
Equivalence
Scaling
Summary
%∆A:B :D 10:50:10
D=-10
{ -50 }{ -10 }
{ 0 }{ -10 }
{ 50 }{ -10 }
{ -50 }{ 0 }
{ 0 }{ 0 }
{ 50 }{ 0 }
{ -50 }{ 10 }
{ 0 }{ 10 }
{ 50 }{ 10 }
D=0
{ -50 }{ -10 }
{ 0 }{ -10 }
{ 50 }{ -10 }
{ -50 }{ 0 }
{ 0 }{ 0 }
{ 50 }{ 0 }
{ -50 }{ 10 }
{ 0 }{ 10 }
{ 50 }{ 10 }
D=10
{ -50 }{ -10 }
{ 0 }{ -10 }
{ 50 }{ -10 }
{ -50 }{ 0 }
{ 0 }{ 0 }
{ 50 }{ 0 }
{ -50 }{ 10 }
{ 0 }{ 10 }
{ 50 }{ 10 }
50% slope change seems small in comparison
25 / 29
©
D. Lansky
Introduction
Equivalence
Scaling
Summary
%∆A:B :D 5:35:5
D=-5
{ -35 }{ -5 }
{ 0 }{ -5 }
{ 35 }{ -5 }
{ -35 }{ 0 }
{ 0 }{ 0 }
{ 35 }{ 0 }
{ -35 }{ 5 }
{ 0 }{ 5 }
{ 35 }{ 5 }
D=0
{ -35 }{ -5 }
{ 0 }{ -5 }
{ 35 }{ -5 }
{ -35 }{ 0 }
{ 0 }{ 0 }
{ 35 }{ 0 }
{ -35 }{ 5 }
{ 0 }{ 5 }
{ 35 }{ 5 }
D=5
{ -35 }{ -5 }
{ 0 }{ -5 }
{ 35 }{ -5 }
{ -35 }{ 0 }
{ 0 }{ 0 }
{ 35 }{ 0 }
{ -35 }{ 5 }
{ 0 }{ 5 }
{ 35 }{ 5 }
35% slope change small?5% range small vs. 5% on lower asy?
26 / 29
©
D. Lansky
Introduction
Equivalence
Scaling
Summary
Experience with %∆A:B :D
I excellent assays can use 5:35:5I many cell fail 5:35:5, ok w/10:50:10I noisy assays struggle with 15:50:15I Equiv. in linear: longer subsets
27 / 29
©
D. Lansky
Introduction
Equivalence
Scaling
Summary
Summary
I TransformI Detect outliers (w/smooth) modelI Use Design StructureI Assess similarity with equivalenceI Asymptote sim. needed at timesI Assess Validation with equivalence
28 / 29
©
D. Lansky
Introduction
Equivalence
Scaling
Summary
Acknowledgements
I Consulting clientsI USP and USP bioassay panel membersI Carrie WagerI NSF EPSCoRI NIH SBIR 3R44RR02198-03S1
29 / 29