A Reviewer’s Perspective on the Use of Constrained Versus Unconstrained Models to Calculate Relative PotencyEvangelos Bakopanos, Ph.D.Senior Biologist/Evaluator, Monoclonal Antibodies DivisionCentre for Evaluation of Radiopharmaceuticals and BiotherapeuticsBiologics & Genetics Therapies Directorate
CASSS BIOASSAYS 2016April 4-5, 2016Silver Spring, Maryland
• The views expressed in this presentation are those of the presenter and do not convey official Health Canada policy.
Disclaimer:
• Scope of the issue
• Relative Potency & Similarity
• Comparison of constrained and unconstrained analysis of data sets:
o Case Study # 1o Case Study # 2o Case Study # 3
• Conclusions
Outline
Analysis of Bioassay Data
Data
Fit Statistical Model
Assess System Suitability
Assess Sample Suitability
Calculate Relative Potency
Reportable Value
Analysis of Bioassay Data
Data
Fit Statistical Model
Assess System Suitability
Assess Sample Suitability
Calculate Relative Potency
Reportable Value
𝐘𝐘 =𝐀𝐀 − 𝐃𝐃
(𝟏𝟏 + 𝐗𝐗𝐂𝐂
𝐁𝐁 + 𝑫𝑫
• Unconstrained (full, unrestricted): fit independent 4PL curves to the dose-response data.
• Constrained (reduced, restricted): fit 4PL curves having common A, D and B parameters
ρ = 𝑬𝑬𝑬𝑬𝟓𝟓𝟓𝟓 𝑹𝑹𝑹𝑹𝑬𝑬𝑬𝑬𝟓𝟓𝟓𝟓 𝑹𝑹
Unconstrained vs Constrained 4PL models
• USP <1034> Analysis of Biological Assay:
For those Test samples in the assay that meet the criterion for similarity to the Standard (i.e., sufficiently similar concentration–response curves or similar straight-line subsets of concentrations), calculate relative potency estimates assuming similarity between Test and Standard, i.e., by analyzing the Test and Standard data together using a model constrained to have exactly parallel lines or curves, or equal intercepts.
Calculation of Relative Potency
• Relative potency is calculated using an unconstrained model.
• The use of the unconstrained model may yield variable & questionable relative potency estimates.
• Precludes any reasonable assessment of the potency data contained in the submission.
• Requires re-analysis of the potency data which hinders the review and approval process.
Recurring Issue with Submissions
Approval Year # of NDS affectedby Unconstrained
Issue
% of Approved NDS
2013 1 13%
2014 2 14%
2015*2* 13%
Monoclonal Antibodies Division
*Also had one SNDS case (excluded from table).
• Unable to find research comparing constrained and unconstrained analysis of data sets and there appears to be little consideration of the effect constraining has on the data.
Typical Feedback from Sponsors
• Scope
• Relative Potency & Similarity
• Comparison of constrained and unconstrained analysis of data sets:
o Case Study # 1o Case Study # 2o Case Study # 3
• Conclusions
Outline
• Can only be strictly defined between two perfectly parallel curves.
Relative Potency
Unconstrained model
Constrain curves in order to enforce parallelism
Unconstrained Constrained
RP = 127 % RP = 95 %
• Hypothesis Testing:o F-Testo Chi-Square
• Equivalence Testing:o Ratio of parameters (A, B, D).
Criteria based on acceptable range for ratio.
Criteria based on acceptable range for Confidence Intervals of ratio.
Assessment of Similarity (parallelism)
67% of
cases
33% of unconstrained cases
• Scope
• Relative Potency & Similarity
• Comparison of constrained and unconstrained analysis of data sets:
o Case Study # 1o Case Study # 2o Case Study # 3
• Conclusions
Outline
• ELISA assay using unconstrained model to estimate relative potency.
• Sponsor was requested to implement the use of constrained model & re-analyse method validation data.
• Equivalence study was performed to statistically evaluate the difference between the constrained and unconstrained model after both the sample data and reference data pass parallelism.
• Study included method validation, comparability, tech transfer, release & stability data (~200 samples).
Case Study # 1
Case Study # 1 ResultsHISTOGRAMS OF RELATIVE POTENCY RESULTS
• The distributions of relative potency results generated by constrained and unconstrained methods were almost identical with very small variability.
Case Study # 1 Conclusion
An assay displaying very small variability
RP = 109 % RP = 106 %
Case Study # 2• Bioassay using unconstrained 4PL model to calculate
relative potency.
• Three replicate values (i.e., plates) for each sample.
• Observed increased bioassay variability over time.
• More frequent results close to control limits.
• 60 valid assays were selected randomly and re-analyzed using constrained 4PL model.
Case Study # 2 Results
HISTOGRAMS OF MEAN RELATIVE POTENCIES
Case Study # 2 Results (continued)
HISTOGRAMS OF %CVs OF MEAN RELATIVE POTENCIES
• The averages of the distributions were comparable (108% and 106% for constrained and unconstrained respectively).
• The variability of the results from the constrained analysis was 39 % lower than the variability of results from the unconstrained analysis (standard deviations of 8.6 and 14.2 respectively).
• The use of the constrained analysis reduced the inter-plate variability.
Case Study # 2 Conclusions
• Health Canada’s concerns:
o Bioassay using unconstrained 4PL model to calculate relative potency.
o HC’s analysis of raw assay data (provided as part of the consistency lot testing assessment) using a constrained 4PL model yielded significantly different results.
Case Study # 3
• HC’s analysis of the raw assay data.
Case Study # 3
• Sponsor’s Response:o Bioassay data were reprocessed using a constrained
curve fit for method validation results, drug substance and drug product release lots, as well as drug product stability results.
o Based on comparison of the data, an approximate two-fold change in the method’s precision capability was observed using the constrained curve fit (i.e. % CV of < 21 for constrained versus < 11 for unconstrained).
o The curve fitting model used in the validated potency method with the appropriate system suitability criteria generates accurate relative potency values.
Case Study # 3
• Decision:o Sponsor had already developed a new bioassay to
replace the current one & intended to file a post approval supplement immediately after NDS approval.
o New bioassay was validated using a constrained model.
o Bridging study demonstrated that the relative potency values reported by the current (unconstrained) & new method were statistically equivalent.
o Product assigned to Lot Release Evaluation Group 3; required to assess potency using the current method (constrained & unconstrained) as well as the new method.
Case Study # 3
• Bioassay parallelism criteria:
o Acceptable range for upper asymptote ratio
o Acceptable range for slope ratio
o Acceptable range for effective asymptote ratio
CASE STUDY # 3
• Hypothesis Testing:o F-Testo Chi-Square
• Equivalence Testing:o Ratio of parameters (A, B, D).
Criteria based on acceptable range for ratio.
Criteria based on acceptable range for Confidence Intervals of ratio.
Assessment of Similarity (parallelism)
• HC’s analysis of the raw assay data.
Case Study # 3
?
CASE STUDY # 3
Constrained
RP = 101% RP = 62 %
CASE STUDY # 3
RP = 101%
CASE STUDY # 3
Constrained
RP = 70 % RP = 38 %
CASE STUDY # 3
RP = 70 %
• Relative potency estimates are based on the assumption that test samples and standard behave similarly in the assay system.
• This assumption is verified by assessing the similarity or parallelism of the dose response curves.
• For those test samples that meet the criterion for similarity to the standard, relative potency estimates are calculated using a constrained model in order to enforce similarity.
Conclusions
• The effect that constraining has on a given data set highly depends on the variability of the assay and quality of the acceptance criteria used to define similarity. Therefore, it should be evaluated on a case by case basis & during method development.
• When data suggests that dose-response curves of a test sample and standard are not similar, regarding them as similar and estimating relative potency accordingly may lead to incorrect conclusions about the potency of the sample.
Conclusions (continued)