EMA DRAFT GUIDELINE ON SUBGROUPSDISCUSSION

April 2014Kristian Windfeld, Biometrics, H. Lundbeck A/S

NOT FOR PROMOTIONAL USE 2

Scope

Guidance to assessors in interpretation of subgroup analyses and associated regulatory decision making

Possible implications for indication and inclusion of data in clinical studies section of SmPC

“Enjoy the result you have found by exploratory data analysis, for you will not find it again”

Confuseus

NOT FOR PROMOTIONAL USE 3

Highlights of draft guideline

Frame for discussing and prioritising subgroups in protocol

Focus on exploratory subgroups rather than those part of confirmatory test strategy

Heterogeneity of target population

Methodological challenges (false positives/negatives)

Credibility of subgroup findingsPlausibility

Pre-specification

Replication

Common scenariosOverall positive study with adverse subgroup findings

Overall borderline/negative study, possible ID of subgroup with positive benefit/risk

NOT FOR PROMOTIONAL USE 4

Definition of subgroups

Based on pre-randomization intrinsic/extrinsic factors

Factor typesUnordered categorical (e.g. region)

Ordered categorical (e.g. disease severity)

Based on continuous measures

Usually subgroups based on single factor. Combinations may be of interest

Risk scores

Based on biological measures – possible misclassification

Discussion:How to pre-specify and justify cutpoints for continuous measures? Should you use cutpoints in the first place for continuous variables?

Is it reasonable to study subgroups based on one factor at a time? When/how to study factor combinations?

NOT FOR PROMOTIONAL USE 5

The multiplicity problem

Multiple testing problemRisk of false ”positives” is recognized – but no excuse for not investigating subgroups…

Risk of approving drug in subgroups not benefitting also important

”Cautionary principle”: replicated evidence cannot be required to confirm credibility of an untoward effect of the experimental treatment

Discussion:How can we achieve adequate protection against false ‘positive’ findings while satisfying the need to study homogeneity of effect in population?

What do you think about prioritization of sensitivity of the investigation (by not adjusting for multiplicity) and the credibility considerations provided to ensure specificity of the approach? Is there a reasonable balance?

Effective in subgroup

Yes No

Yes

No

Decision to approvein subgroup

E1

E2

NOT FOR PROMOTIONAL USE 6

Statistical methodsStatistical interaction tests appear not to be encouraged (said not to be well understood…)

(unadjusted) p-values + plots with estimates and CIs (e.g. Forest plots)Visual inspection of Forest plots; some guidance text about when to be concerned given (Subgroups with CIs 2x or 3x width of overall effect that are not overlapping with overall effect CI…)

Shrunk estimates may be used to reduce the problem of extreme random subgroup findings by chance

P-value adjustment not recommended because the subgroup analyses are triggers for further investigation

Commonly used scale versus scale relevant for B-R decision (relative vs absolute)

Discussion:What is your opinion about analyzing the same outcome on both relative and absolute scale? (Statistical versus regulatory (B/R) considerations…)

How do you find the interpretability of unadjusted p-values/CIs in Forest-plots? How should difference between plausible vs exploratory subgroups be addressed?

Do you see a role for ‘data mining’ methods such as recursive partitioning, lasso, etc.?

NOT FOR PROMOTIONAL USE 7

Prioritising the exploratory analyses

Addressing multiplicity by prioritization:Key subgroup (used for stratification, plausibility)

Truly exploratory (demographic, disease characteristics,…)

Maximize a priori discussion to minimize a posteriori discussion…

Potential disincentive for sponsor to plan subgroup analyses, arguing for non-plausibility

Discussion:Why should a sponsor pre-specify and prioritize potentially prognostic and predictive factors with a potential label restriction as a consequence? Is it better to not pre-specify?

How do you see the relevance of multiplicity correction for the plausible factor based subgroup investigations?

NOT FOR PROMOTIONAL USE 8

Credibility of subgroup findings

Key considerations regarding credibility of subgroup findings:Plausibility (+pre-specification)

Replication

Possibility to look at ≥2 sources of evidence better than a pooled estimate(!)

Sponsor may use lack of pre-specification to argue for lack of credibility – not accepted (cf sponsor incentive discussion…)

Discussion:How do you see the apparent preference for doing subgroup analyses by study rather than pooled, given the replication consideration?

NOT FOR PROMOTIONAL USE 9

DSBS consolidated comments

We plan to submit DSBS consolidated comments via EFPIS

Please email your (company/group consolidated) comments to krwi@lundbeck.com by 30 April 2014

EFSPI deadline: April/May

EMA deadline: 31 July 2014