interlaboratory performance metrics from the mam ......interlaboratory performance metrics from the...

Interlaboratory Performance Metrics from the MAM Consortium New Peak Detection Round Robin Study

Trina Mouchahoir1,2, Rich Rogers3, John Schiel1,2

15th Symposium on the Practical Applications of Mass Spectrometry in the Biotechnology Industry

San Francisco, CASeptember 12, 2018

1National Institute of Standards and Technology (NIST)2Institute for Bioscience and Biotechnology Research (IBBR)3Just Biotherapeutics, Inc.

Multi-Attribute Method (MAM)

EEQYNSTYRG0F, G1F, G2F

Glycan

DTLMISROxidation

SLSLSPG(K)C-term Lys-loss

GFYPSDIAVEWESNGQPENNYKDeamidation

SCDKTHTCPPCPAPELLGGPSVFLFPPKPKGlycationQVTLR

N-term pyro-Glu

SGGTAALGCLVKClipping

ESGPALVKPTQTLTLTCTFSGFSLSTAGMSVGWIRIsomerization

0 5 10 15 20 25 30 35 40 45 50 55 60 65

65.86

10.6329.47 31.109.92 50.92 62.60

23.02 42.8655.8249.7239.51 51.8229.05 63.4418.23

19.5126.28

46.71

60.82 67.785.02 53.98

20

40

60

80

100

Time (min)

Rela

tive A

bundance

Enzymatic digestion of biopharmaceutical protein

Separation of peptides by liquid chromatography

Analysis of peptides by mass spectrometry

Interrogation of data for pre-defined Product Quality Attributes (e.g. N-terminal pyro-Glu; glycopeptide profile….)

New Peak Detection: interrogation of data for new, missing or changed peaks (e.g. increased deamidation, oxidation; host cell peptides…)

2

MAM: New Peak Detection

NISTmAb (3 µg)

NISTmAb (3 µg)+ exogenous peptides (0.5 pmol each)

3

TVAAPSVFIFPPSDEQLK

TVAAPSVFIFPPSDEQLK

NISTmAb (3 µg)

NISTmAb (3 µg)+ exogenous peptides (0.5 pmol each)

MAM: New Peak Detection

4

TVAAPSVFIFPPSDEQLK

TVAAPSVFIFPPSDEQLK

LSSEAPALFQFDLK

MAM: New Peak Detection

TVAAPSVFIFPPSDEQLK

TVAAPSVFIFPPSDEQLK

TVAAPSVFIFPPSDEQLK

TVAAPSVFIFPPSDEQLK

TVAAPSVFIFPPSDEQLK (+K)

TVAAPSVFIFPPSDEQLK (+K)

LSSEAPALFQFDLK

LSSEAPALFQFDLK

NISTmAb (3 µg)

NISTmAb (3 µg)+ exogenous peptides (0.5 pmol each)

EIC

5

MAM: New Peak Detection

NISTmAb (3 µg)

NISTmAb (3 µg)+ exogenous peptides (0.5 pmol each)

RT

(min

)R

T (m

in)

m/z 6

Round Robin New Peak Detection Study

7

MAM Consortium Round Robin New Peak Detection Study

• Evaluate interlaboratory reproducibility of MAM

• Understand qualitative and quantitative capabilities of the platform

• Establish performance metrics

• Assess the state-of-the-industry regarding MAM

8

MAM Consortium Round Robin New Peak Detection Study

• Defray some of the inherent risk associated with evaluating and moving forward with new platforms

• Encourage integration of MAM within the industry

• Enable the platform to move forward in regulatory filings

9

MAM Consortium Round Robin New Peak Detection StudySample Kit

System Suitability Peptide Mix

NISTmAb Digests

pH StressReference Spiked Unknown

10

MAM Consortium Round Robin New Peak Detection StudyProtocol

• Samples analyzed using same: Column Gradient Injection sequence

• LC and MS instruments vary with each participant• Participants use preferred data analysis software and platforms• Returned data regarding System Suitability Peptide Mix• Reported data regarding identification and quantification of: Pre-defined peptides and quality attributes (reference digest) New, missing or changed peaks (reference vs sample)

• Datasets are anonymized

SampleInjection

VolumeInjection Quantity Data acquisition

Blank 12 µL NA MS1 only

Blank 12 µL NA MS1 only

System Suitability Mix 2 µL 1 pmol/peptide MS1 only

Reference 12 µL 3 µg IgG MS1 only

Spike 12 µL 3 µg IgG; 0.5 pmol/SSM peptide MS1 only

pH Stress 12 µL 3 µg IgG MS1 only

Unknown 12 µL 3 µg IgG MS1 only

System Suitability Mix 2 µL 1 pmol/peptide MS1 only

Reference 12 µL 3 µg IgG MS/MS

Spike 12 µL 3 µg IgG; 0.5 pmol/SSM peptide MS/MS

pH Stress 12 µL 3 µg IgG MS/MS

Unknown 12 µL 3 µg IgG MS/MS

System Suitability Mix 2 µL 1 pmol/peptide MS1 only

Blank 12 µL NA MS1 only

11

Preliminary Numbers

• 28 sets of processed data

• 16 sets of raw data

0

5

10

15

20

25

30

35

40

45

QTOF LIT-Orbitrap Q-Orbitrap

% o

f Pa

rtic

ipan

ts

Instrument Types

12

Outlier: More than 1.5x above the upper quartile, shape coded by participant

Outlier: More than 1.5x below the lower quartile, shape coded by participant

Minimum: Least value, excluding outliers

Maximum: Greatest value, excluding outliers

Median

Upper Quartile

Lower Quartile

Mean

13

0

10

20

30

40

50

60

Ave

rage

RT

(min

)

System Suitability Peptide Retention Time

*not detected by Increasing RT (min)

14

0.0

0.5

1.0

1.5

2.0

2.5

RT

St D

ev

(min

)

System Suitability Peptide Retention Time Variation(Between-Lab)

*not detected by

Increasing RT (min)15

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

RT

St D

ev

(min

)System Suitability Peptide Retention Time Variation

(Within-Lab)

*not detected byNot included due to aligned RT values:

Increasing RT (min)16

Increasing RT (min)17

0

2

4

6

8

10

12

14|P

PM

|

System Suitability Peptide Mass Accuracy

*not detected by *not detected by

Note: This slide has been corrected from a previous version.

0

5

10

15

20

25

Ave

RA

(%

)

System Suitability Peptide Relative Abundance

Not included due to lack of 15 peptides:Not included due to errors in RA reporting:

Increasing RT (min)

6.67

18

0.0

0.5

1.0

1.5

2.0

2.5

3.0

3.5

4.0

4.5

RA

St

De

v (%

)System Suitability Peptide Relative Abundance Variation

(Between Lab)

Not included due to lack of 15 peptides:Not included due to errors in RA reporting:

Increasing RT (min)19

0

0.5

1

1.5

2

2.5

3

RA

St

De

v (%

)System Suitability Peptide Relative Abundance Variation

(Within-Lab)

Not included due to lack of 15 peptides: Not included due to lack of third injection: Not included due to errors in RA reporting:

Increasing RT (min)20

0.0

0.5

1.0

1.5

2.0

2.5R

T St

De

v (m

in)

NISTmAb Reference Peptide Retention Time Variation(Between-Lab)

*not detected bypQ = Gln -> pyro-Gluk = Lys-loss

Increasing RT (min)21

20

30

40

50

60

70

80

90

0

2

4

6

8

10

12

14

|PP

M|

NISTmAb Reference Peptide Mass Accuracy

*not detected bypQ = Gln -> pyro-Gluk = Lys-loss

Increasing RT (min)22

90

92

94

96

98

100

102

QVTLR*(Gln->pyro-glu)

RA

(%

)

Pyro-glutamination

*neither modified nor unmodified detected by

65

70

75

80

85

90

95

100

SLSLSPGK(Lys-loss)

RA

(%

)

Lys-loss

0

0.5

1

1.5

2

2.5

3

DTLMISR(Met Oxidation)

RA

(%

)

Met Oxidation

0

0.5

1

1.5

2

2.5

3

3.5

4

GFYPSDIAVEWESNGQPENNYK(Deamidation)

GFYPSDIAVEWESNGQPENNYK(Ammonia-loss)

RA

(%

)

Asparagine Modification

NISTmAb Reference Peptide Relative Abundance

23

0

10

20

30

40

50

60

70

80

90

A2G0F A2G1F A2G2F

RA

(%

)

Major EEQYNSTYR Glycopeptides

not included due to lack of detection of A2G1F and A2G2Fand not included due to lack of detection of A2G2Fnot included due to lack of detection of A2G0F, A2G1F and A2G2F

NISTmAb Reference Peptide Relative Abundance

24

0.0

1.0

2.0

3.0

4.0

5.0

6.0

7.0

8.0

9.0EE

QYN

STYR

+ A

2G

0F

†

EEQ

YNST

YR +

A2

G1

F †

EEQ

YNST

YR +

A2

G2

F †

QV

TLR

*(G

ln->

pyr

o-g

lu)

DTL

MIS

R(M

et O

xid

atio

n)

SLSL

SPG

K(L

ys-l

oss

)

GFY

PSD

IAV

EWES

NG

QP

ENN

YK(D

eam

idat

ion

)

GFY

PSD

IAV

EWES

NG

QP

ENN

YK(A

mm

on

ia-l

oss

)

RA

St

De

v (%

)

NISTmAb Reference Peptide Modification Relative Abundance Variation(Between-Lab)

* not detected by † not included due to lack of detection of A2G2F1F and A2G2F:

not included due to lack of detection of A2G: and not included due to lack of detection of A2G0F, A2G1F and A2G2F: Increasing RT (min)

25

0

5

10

15

20

25

30

35

40

45

50

55

60

65

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28

Nu

mb

er o

f P

eaks

Participant

Spiked Digest New Peak Detection

Plasticizer

Unknown Species

NISTmAb Peptides

Spiked Peptide Impurities

Modified Spiked Peptides

Spiked Peptides

26

0

5

10

15

20

25

30

35

% R

epo

rted

Survey of New, Missing, Changed Peak Modifications(pH Stress Digest)

27

0

50

100

150

Nu

mb

er

of

Pea

ks

Participant

pH Stress Digest New Peak Detection

False Positive

Missing/Decreased Peaks

New/Increased Peaks

450

500

550

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28

28

0

5

10

15

20

25

30

35

40

45

50

55

60

65

70

75

80

85

90

95

100

105

Nu

mb

er o

f P

eaks

Participant

Unknown Digest New Peak Detection

False Positives

New, Missing or Changed Peaks

29

“Passing” Results

30

0

20

40

60

80

100

120

Spike pH Unknown

% o

f P

arti

cip

ants

Sample

Pass

Fail

Spike pH Unknown

Individual “Pass/Fail” Status

12 datasets “pass” for all three analyses

31Note: This slide has been updated from a previous version.

Conclusions and Future Directions

• Out of 28 datasets, 13 were considered “passing”• Identify sources of false positives• Identify sources of false negatives• Identify ways to improve reproducibility Community specifications System suitability criteria for MAM platform

• Analyze 16 raw data sets with harmonized software platforms

32

AbbVie:Aaron AmmermannAnton V. ManuilovBo YanBrian SchmidtChris M. ChumsaeTaro FujimoriTom RobinsonXiaoxiao LiXinbi Li

Agilent:Gregory StaplesJordy Hsiao

Amgen:Aria VeachDa Ren

MedImmune:Ben NiuJihong Wang

BioAnalytix:Dongdong WangWael Yared

Biogen:Li ZangYan WangZoran Sosic

Bristol-Myers Squibb:Li TaoPeiran LiuRichard ludwigWei Wu

Charles River Labs:Andrew HannemanAhmet Cansizoglu

FDA:Sarah RogstadXiaoshi Wang

Fujifilm Diosynth:Greg AdamsHunter WalkerIrina PerdivaraMargo Wilson

Genedata:Arnd BrandenburgDavid BushJoe Shambaugh

Genentech:Chi-Yan TamChristopher YuMelissa Alvarez

GSK:Chun ShaoLi Cao

Janssen:Andrew MahanHirsh NandaKristen Nields

JUST:Nancy Nightlinger

Lonza:Helena Maria BaryszMichael Jahn

Merck:Angelo PalmeseBhumit PatelDouglas RichardsonFrancesca CutilloGabriella LeoNunzio SepeYan-Hui LiuYi Wang

NIST:Benjamin PlaceDaniela Tizabi (UMBC IMET)

Ryan EvansTony Kearsely

Novavax:Oleg BorisovYali LuErnest Maynard

Novo Nordisk:Albrecht GruhlerCarsten P. SönksenKim F. HaselmannLisbet Lone HansenThomas N. Krogh

Pfizer:Anastasiya ManuilovAndrew DawdyCarly DanielsDavid RipleyHimakshi PatelJason RouseJosh WoodsJustin SperryKeith JohnsonKristin BoggioMatthew ThompsonOlga FrieseSean ShenSimon LetarteThomas PowersWenqin NiYing Zhang

Protein Metrics:Eric CarlsonIlker SenSt John Skilton

Sanofi:Anders LundHelena AwadMartha StapelsMichelle Busch

SCIEX:Eva DuchoslavHarini KaluarachchiSean McCarthySibylle HeidelbergerXu Guo

Teva:Ying ZhouJing ZhenJohn Kim

Waters:Jing FangWeibin ChenYing Qing Yu

Zoetis:Hua YuanJohn G. HoogerheideRebecca Scott

Thank OU

*Disclaimer: Certain commercial equipment, instruments, or materials are identified in this poster to specify

adequately the experimental procedure. Such identification does not imply recommendation or endorsement by the

National Institute of Standards and Technology, nor does it imply that the materials or equipment identified are

necessarily the best available for the purpose. Note: This slide has been updated from a previous version.