Interlaboratory Performance Metrics from the MAM Consortium New Peak Detection Round Robin Study
Trina Mouchahoir1,2, Rich Rogers3, John Schiel1,2
15th Symposium on the Practical Applications of Mass Spectrometry in the Biotechnology Industry
San Francisco, CASeptember 12, 2018
1National Institute of Standards and Technology (NIST)2Institute for Bioscience and Biotechnology Research (IBBR)3Just Biotherapeutics, Inc.
Multi-Attribute Method (MAM)
EEQYNSTYRG0F, G1F, G2F
Glycan
DTLMISROxidation
SLSLSPG(K)C-term Lys-loss
GFYPSDIAVEWESNGQPENNYKDeamidation
SCDKTHTCPPCPAPELLGGPSVFLFPPKPKGlycationQVTLR
N-term pyro-Glu
SGGTAALGCLVKClipping
ESGPALVKPTQTLTLTCTFSGFSLSTAGMSVGWIRIsomerization
0 5 10 15 20 25 30 35 40 45 50 55 60 65
65.86
10.6329.47 31.109.92 50.92 62.60
23.02 42.8655.8249.7239.51 51.8229.05 63.4418.23
19.5126.28
46.71
60.82 67.785.02 53.98
20
40
60
80
100
Time (min)
Rela
tive A
bundance
Enzymatic digestion of biopharmaceutical protein
Separation of peptides by liquid chromatography
Analysis of peptides by mass spectrometry
Interrogation of data for pre-defined Product Quality Attributes (e.g. N-terminal pyro-Glu; glycopeptide profile….)
New Peak Detection: interrogation of data for new, missing or changed peaks (e.g. increased deamidation, oxidation; host cell peptides…)
2
MAM: New Peak Detection
NISTmAb (3 µg)
NISTmAb (3 µg)+ exogenous peptides (0.5 pmol each)
3
TVAAPSVFIFPPSDEQLK
TVAAPSVFIFPPSDEQLK
NISTmAb (3 µg)
NISTmAb (3 µg)+ exogenous peptides (0.5 pmol each)
MAM: New Peak Detection
4
TVAAPSVFIFPPSDEQLK
TVAAPSVFIFPPSDEQLK
LSSEAPALFQFDLK
MAM: New Peak Detection
TVAAPSVFIFPPSDEQLK
TVAAPSVFIFPPSDEQLK
TVAAPSVFIFPPSDEQLK
TVAAPSVFIFPPSDEQLK
TVAAPSVFIFPPSDEQLK (+K)
TVAAPSVFIFPPSDEQLK (+K)
LSSEAPALFQFDLK
LSSEAPALFQFDLK
NISTmAb (3 µg)
NISTmAb (3 µg)+ exogenous peptides (0.5 pmol each)
EIC
5
MAM: New Peak Detection
NISTmAb (3 µg)
NISTmAb (3 µg)+ exogenous peptides (0.5 pmol each)
RT
(min
)R
T (m
in)
m/z 6
Round Robin New Peak Detection Study
7
MAM Consortium Round Robin New Peak Detection Study
• Evaluate interlaboratory reproducibility of MAM
• Understand qualitative and quantitative capabilities of the platform
• Establish performance metrics
• Assess the state-of-the-industry regarding MAM
8
MAM Consortium Round Robin New Peak Detection Study
• Defray some of the inherent risk associated with evaluating and moving forward with new platforms
• Encourage integration of MAM within the industry
• Enable the platform to move forward in regulatory filings
9
MAM Consortium Round Robin New Peak Detection StudySample Kit
System Suitability Peptide Mix
NISTmAb Digests
pH StressReference Spiked Unknown
10
MAM Consortium Round Robin New Peak Detection StudyProtocol
• Samples analyzed using same: Column Gradient Injection sequence
• LC and MS instruments vary with each participant• Participants use preferred data analysis software and platforms• Returned data regarding System Suitability Peptide Mix• Reported data regarding identification and quantification of: Pre-defined peptides and quality attributes (reference digest) New, missing or changed peaks (reference vs sample)
• Datasets are anonymized
SampleInjection
VolumeInjection Quantity Data acquisition
Blank 12 µL NA MS1 only
Blank 12 µL NA MS1 only
System Suitability Mix 2 µL 1 pmol/peptide MS1 only
Reference 12 µL 3 µg IgG MS1 only
Spike 12 µL 3 µg IgG; 0.5 pmol/SSM peptide MS1 only
pH Stress 12 µL 3 µg IgG MS1 only
Unknown 12 µL 3 µg IgG MS1 only
System Suitability Mix 2 µL 1 pmol/peptide MS1 only
Reference 12 µL 3 µg IgG MS/MS
Spike 12 µL 3 µg IgG; 0.5 pmol/SSM peptide MS/MS
pH Stress 12 µL 3 µg IgG MS/MS
Unknown 12 µL 3 µg IgG MS/MS
System Suitability Mix 2 µL 1 pmol/peptide MS1 only
Blank 12 µL NA MS1 only
11
Preliminary Numbers
• 28 sets of processed data
• 16 sets of raw data
0
5
10
15
20
25
30
35
40
45
QTOF LIT-Orbitrap Q-Orbitrap
% o
f Pa
rtic
ipan
ts
Instrument Types
12
Outlier: More than 1.5x above the upper quartile, shape coded by participant
Outlier: More than 1.5x below the lower quartile, shape coded by participant
Minimum: Least value, excluding outliers
Maximum: Greatest value, excluding outliers
Median
Upper Quartile
Lower Quartile
Mean
13
0
10
20
30
40
50
60
Ave
rage
RT
(min
)
System Suitability Peptide Retention Time
*not detected by Increasing RT (min)
14
0.0
0.5
1.0
1.5
2.0
2.5
RT
St D
ev
(min
)
System Suitability Peptide Retention Time Variation(Between-Lab)
*not detected by
Increasing RT (min)15
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
RT
St D
ev
(min
)System Suitability Peptide Retention Time Variation
(Within-Lab)
*not detected byNot included due to aligned RT values:
Increasing RT (min)16
Increasing RT (min)17
0
2
4
6
8
10
12
14|P
PM
|
System Suitability Peptide Mass Accuracy
*not detected by *not detected by
Note: This slide has been corrected from a previous version.
0
5
10
15
20
25
Ave
RA
(%
)
System Suitability Peptide Relative Abundance
Not included due to lack of 15 peptides:Not included due to errors in RA reporting:
Increasing RT (min)
6.67
18
0.0
0.5
1.0
1.5
2.0
2.5
3.0
3.5
4.0
4.5
RA
St
De
v (%
)System Suitability Peptide Relative Abundance Variation
(Between Lab)
Not included due to lack of 15 peptides:Not included due to errors in RA reporting:
Increasing RT (min)19
0
0.5
1
1.5
2
2.5
3
RA
St
De
v (%
)System Suitability Peptide Relative Abundance Variation
(Within-Lab)
Not included due to lack of 15 peptides: Not included due to lack of third injection: Not included due to errors in RA reporting:
Increasing RT (min)20
0.0
0.5
1.0
1.5
2.0
2.5R
T St
De
v (m
in)
NISTmAb Reference Peptide Retention Time Variation(Between-Lab)
*not detected bypQ = Gln -> pyro-Gluk = Lys-loss
Increasing RT (min)21
20
30
40
50
60
70
80
90
0
2
4
6
8
10
12
14
|PP
M|
NISTmAb Reference Peptide Mass Accuracy
*not detected bypQ = Gln -> pyro-Gluk = Lys-loss
Increasing RT (min)22
90
92
94
96
98
100
102
QVTLR*(Gln->pyro-glu)
RA
(%
)
Pyro-glutamination
*neither modified nor unmodified detected by
65
70
75
80
85
90
95
100
SLSLSPGK(Lys-loss)
RA
(%
)
Lys-loss
0
0.5
1
1.5
2
2.5
3
DTLMISR(Met Oxidation)
RA
(%
)
Met Oxidation
0
0.5
1
1.5
2
2.5
3
3.5
4
GFYPSDIAVEWESNGQPENNYK(Deamidation)
GFYPSDIAVEWESNGQPENNYK(Ammonia-loss)
RA
(%
)
Asparagine Modification
NISTmAb Reference Peptide Relative Abundance
23
0
10
20
30
40
50
60
70
80
90
A2G0F A2G1F A2G2F
RA
(%
)
Major EEQYNSTYR Glycopeptides
not included due to lack of detection of A2G1F and A2G2Fand not included due to lack of detection of A2G2Fnot included due to lack of detection of A2G0F, A2G1F and A2G2F
NISTmAb Reference Peptide Relative Abundance
24
0.0
1.0
2.0
3.0
4.0
5.0
6.0
7.0
8.0
9.0EE
QYN
STYR
+ A
2G
0F
†
EEQ
YNST
YR +
A2
G1
F †
EEQ
YNST
YR +
A2
G2
F †
QV
TLR
*(G
ln->
pyr
o-g
lu)
DTL
MIS
R(M
et O
xid
atio
n)
SLSL
SPG
K(L
ys-l
oss
)
GFY
PSD
IAV
EWES
NG
QP
ENN
YK(D
eam
idat
ion
)
GFY
PSD
IAV
EWES
NG
QP
ENN
YK(A
mm
on
ia-l
oss
)
RA
St
De
v (%
)
NISTmAb Reference Peptide Modification Relative Abundance Variation(Between-Lab)
* not detected by † not included due to lack of detection of A2G2F1F and A2G2F:
not included due to lack of detection of A2G: and not included due to lack of detection of A2G0F, A2G1F and A2G2F: Increasing RT (min)
25
0
5
10
15
20
25
30
35
40
45
50
55
60
65
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28
Nu
mb
er o
f P
eaks
Participant
Spiked Digest New Peak Detection
Plasticizer
Unknown Species
NISTmAb Peptides
Spiked Peptide Impurities
Modified Spiked Peptides
Spiked Peptides
26
0
5
10
15
20
25
30
35
% R
epo
rted
Survey of New, Missing, Changed Peak Modifications(pH Stress Digest)
27
0
50
100
150
Nu
mb
er
of
Pea
ks
Participant
pH Stress Digest New Peak Detection
False Positive
Missing/Decreased Peaks
New/Increased Peaks
450
500
550
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28
28
0
5
10
15
20
25
30
35
40
45
50
55
60
65
70
75
80
85
90
95
100
105
Nu
mb
er o
f P
eaks
Participant
Unknown Digest New Peak Detection
False Positives
New, Missing or Changed Peaks
29
“Passing” Results
30
0
20
40
60
80
100
120
Spike pH Unknown
% o
f P
arti
cip
ants
Sample
Pass
Fail
Spike pH Unknown
Individual “Pass/Fail” Status
12 datasets “pass” for all three analyses
31Note: This slide has been updated from a previous version.
Conclusions and Future Directions
• Out of 28 datasets, 13 were considered “passing”• Identify sources of false positives• Identify sources of false negatives• Identify ways to improve reproducibility Community specifications System suitability criteria for MAM platform
• Analyze 16 raw data sets with harmonized software platforms
32
AbbVie:Aaron AmmermannAnton V. ManuilovBo YanBrian SchmidtChris M. ChumsaeTaro FujimoriTom RobinsonXiaoxiao LiXinbi Li
Agilent:Gregory StaplesJordy Hsiao
Amgen:Aria VeachDa Ren
MedImmune:Ben NiuJihong Wang
BioAnalytix:Dongdong WangWael Yared
Biogen:Li ZangYan WangZoran Sosic
Bristol-Myers Squibb:Li TaoPeiran LiuRichard ludwigWei Wu
Charles River Labs:Andrew HannemanAhmet Cansizoglu
FDA:Sarah RogstadXiaoshi Wang
Fujifilm Diosynth:Greg AdamsHunter WalkerIrina PerdivaraMargo Wilson
Genedata:Arnd BrandenburgDavid BushJoe Shambaugh
Genentech:Chi-Yan TamChristopher YuMelissa Alvarez
GSK:Chun ShaoLi Cao
Janssen:Andrew MahanHirsh NandaKristen Nields
JUST:Nancy Nightlinger
Lonza:Helena Maria BaryszMichael Jahn
Merck:Angelo PalmeseBhumit PatelDouglas RichardsonFrancesca CutilloGabriella LeoNunzio SepeYan-Hui LiuYi Wang
NIST:Benjamin PlaceDaniela Tizabi (UMBC IMET)
Ryan EvansTony Kearsely
Novavax:Oleg BorisovYali LuErnest Maynard
Novo Nordisk:Albrecht GruhlerCarsten P. SönksenKim F. HaselmannLisbet Lone HansenThomas N. Krogh
Pfizer:Anastasiya ManuilovAndrew DawdyCarly DanielsDavid RipleyHimakshi PatelJason RouseJosh WoodsJustin SperryKeith JohnsonKristin BoggioMatthew ThompsonOlga FrieseSean ShenSimon LetarteThomas PowersWenqin NiYing Zhang
Protein Metrics:Eric CarlsonIlker SenSt John Skilton
Sanofi:Anders LundHelena AwadMartha StapelsMichelle Busch
SCIEX:Eva DuchoslavHarini KaluarachchiSean McCarthySibylle HeidelbergerXu Guo
Teva:Ying ZhouJing ZhenJohn Kim
Waters:Jing FangWeibin ChenYing Qing Yu
Zoetis:Hua YuanJohn G. HoogerheideRebecca Scott
Thank OU
*Disclaimer: Certain commercial equipment, instruments, or materials are identified in this poster to specify
adequately the experimental procedure. Such identification does not imply recommendation or endorsement by the
National Institute of Standards and Technology, nor does it imply that the materials or equipment identified are
necessarily the best available for the purpose. Note: This slide has been updated from a previous version.