improving the quality of protein samples for better data...

Dr. André MatagneCentre for Protein Engineering

Protein Factory & Robotein facilitiesUNIVERSITY OF LIEGE

Joint initiative on the quality of recombinant proteins

Improving the quality of protein samples for better data reproducibility

Implementation of a standardized sample quality control workflow

Les Journées Jeunes Chercheurs Transfrontalières URCA, Reims, April 2nd, 3rd, 2019 - Some slides prepared by Bertrand Raynal, Pasteur Institute, Paris

– Some slides prepared by Mario Lebendiker, Hebrew University of Jerusalem

QC teams of ARBRE-MOBIEU and P4EUNick Berrow - Institute for research in Medicine, Barcelona, Spain (P4EU)

Maria Garcia-Alai - EMBL Hamburg, Germany (ARBRE-MOBIEU)

Stefan Knauer - Bayreuth University, Germany (ARBRE-MOBIEU)

Mario Lebendiker - Hebrew University of Jerusalem, Israel (P4EU)

Blanca Lopez-Mendez – University of Copenhagen, Denmark (ARBRE-MOBIEU)

Ario de Marco - University of Nova Gorica, Slovenia (P4EU and ARBRE-MOBIEU)

André Matagne – University of Liège, Belgium (ARBRE-MOBIEU

Annabel Parret - EMBL Hamburg, Germany (P4EU)

Bertrand Raynal - Pasteur Institute, Paris, France (ARBRE-MOBIEU)

Kim Remans - EMBL Heibelberg, Germany (P4EU)

Stephan Uebel - MPI for Biochemistry Munich, Germany (ARBRE-MOBIEU)

Titel der Präsentation (Editierbar im Folienmaster), Ort, 18.04.2019 (Editierbar im Folienmaster)Sabine Suppmann, Biochemistry Core Facility 9th P4EU meeting, Nov30

Network of ca. 100 members form > 40 protein facilities, mainly but not exclusively in Europe

Started in 2010 (Hüseyin Besir, EMBL)

Share expertise and information

Exchange materials & protocols

New tools & technologies

Establish standards for P related work

Benchmarking and SOPs

Enable collaboration

Access to external facilities

Training of staff/usersfrom Hüseyin Besir, CTLS 2014

(2015)

and the related COST Action

MOBIEU (Molecular Biophysics in Europe)

Topic: “Integrating Molecular Biophysics Approaches for Biology and

Healthcare” (Grant period 2016-2020)

A large communityhas already joined

More than 135

resource labs/infrastructures/facilities

from 29 different european countries

working in very different contexts ,

some more "research topic oriented"

and others

more "technology oriented"

Dedicated website http://arbre-mobieu.eu/

http://arbre-mobieu.eu/

WG2Joint research projects

WG1Communication and

membership

WG3Training and human capacity

development

WG7External synergies

WG6Partnership with instrument

developers and manufacturing companies

WG5Transnational access to

instrumentation and expertise

WG4Optimization of data quality

7 working groups

+ a Short Term Scientific Mission (STSM)

coordination

Joint initiative on the quality of recombinant proteins

Improving the time-efficiency and quality of your resultsImplementation of a standardized sample quality control workflow (guidelines, best practice recommendations)

CA15126

• A lot of time is spent on poor quality samples

Improving the quality of the samples is essential to improve the quality (i.e. accuracy, reproducibility) of the results we produce

• The best experiments in the world will turn garbage in expensive garbage

Our analysis of the situation (as core facilities) in different European institutions

The international position

Tremendous Economic Impact

Freedman LP, Cockburn IM, Simcoe TS (2015) The Economics of Reproducibility in Preclinical Research. PLoS Biol 13(6): e1002165.(using 2012 data)

(56.4109)

Improve experimental design/data analysis!

Improve QC and characterization of the‘protein’ samples

Educative and basic question: how much can I trust my protein? What should I know?

Check that we are working with:

the correct protein

that it is pure/homogeneous

have the right fold/activity

Report the results of each control in publications

Encourage more extensive use of supplementary data

Are we wasting 50% of pre-clinical researchbudget? What can we do?

Did you check your sample? Researcher opinion!

• “I have no time…”

• “SEC? DLS? MS?... My boss thinks it is a waste of time…”

• “It is the way we have prepared samples in the lab for the last ten years…”

• “But some experiments have worked with this sample…”

• “I don’t know how to do it…”

• “I’ll do the experiment anyway, it may work…”

The reality:Quoting people’s experience with not fully validated samples

• “We expressed a human protein in insect cells. By using SDS-PAGE followed byCoomasie blue staining, we observed a single protein band with a “correct” MM. Wesucceeded to crystallize it, and only then we “discovered” that we worked with adifferent target. It was an ubiquitous insect cell protein.”

• “We expressed a yeast protein. SDS-PAGE showed a single bandwith the expected MM. We performed interaction studies andcould not reproduce our previous data. Followoing an analysis bymass spectrometry, we discovered that we were missing 4 aminoacids at one end that were essential for the interaction.”

• “I have done many unsuccessful tests of crystallization with myprotein. I finally decided to do a simple quality test. I noticed thatthe protein was not homogenous in the buffer I used forpurification. After buffer optimization, I obtained an homogenousprotein that did crystallize.”

• “Many people came to our Facility with the aim to crystallize theirprotein after unsuccessful and costly trials with other groups. After ashort talk, we discover that they never check mass homogeneity(aggregation state) nor charge homogeneity.”

Our proposal (P4EU/ARBRE MOBIEU consensus)

- Reproducibility

- Sample optimization

- Initial sample assessment

• Purity• Integrity• Homogeneity/dispersity

• Homogeneity• Solubility• Time stability• Storage

- Activity assessment

- Minimal information to be known

• Protein name, origin, sequence (i.e. protein identity)

• Storage conditions (Buffer, T°….)• Concentration (how it was measured!!)

Our proposal

Protein identity and production parameters protein name and complete sequence, by providing a

NCBI or UniProt accession number, cloning strategy and the source of the DNA (species)

expression vector and host strain, including the tags and cleavage sites used, with the complete sequence of the final protein (or sufficient details to derive it)

Expression and purification conditions, described in full details for possible reproduction

Protein concentrationSpecifying the method used for quantification and the molar extinction coefficient at 280 nm, if applicable

Storage conditionsi.e. final buffer composition (pH, buffer, salts and additives), storage temperature or lyophilization conditions

The minimal information that needs to be provided or known

The minimal quality control parameters

Homogeneity/dispersitychecked preferably by Size Exclusion Chromatography (SEC) and/or dynamic Light Scattering (DLS) or by SEC with Multi Angle Light Scattering (SEC-MALS), Flow Field-Flow Fractionation (F4), F4-MALS or analytical ultracentrifugation (AUC)

Identity and integritychecked preferably by intact protein mass (MS), peptide mass fingerprint (MS), or Edman sequencing

PurityChecked by SDS-PAGE, Capillary Electrophoresis (e.g. LabChip GXII), Reverse Phase LC or Mass Spectrometry

Our proposal

10 20 30

100

200

300

400

500

600

Ab

so

rba

nc

e (

Au

)

Time (min)

UV spectrum between 240 nm and 340 nm Mandatory if protein binds nucleic acidsNucleic acid content and general protein fitness/quality

Batch-to-batch consistency Mandatory if more than one batch is to be usedUse some of the methods listed for minimal QC

Conformational stability/folding stateChecked by Circular Dichroism(CD), Differential Scanning Calorimetry (DSC),NMR, Fourier Transform InfraRed (FTIR)

Homogeneity (e.g. isoforms)Checked by analytical Ion Exchange Chromatography (IEX), analytical Hydrophobic Interaction Chromatography (HIC), Isoelectric Focusing (IEF)

Protein competent fraction, i.e. the relative amount of active protein Measured as specific activity, by active-site titration or other suitable methods (to measure biological activity)

Optimization of storage conditionsLong term stability, activity assay, DLS or thermal shift assay

Extended quality control parameters

Our proposal

The minimum quality control relies on five methods only:

• SDS-PAGE

• UV-Visible spectrophotometry

• Mass spectrometry

• Size exclusion chromatography

• Dynamic light scattering

Minimum quality control guideline

Collection of experimental data in order to provide a statistical basis

31 institutions

47 laboratories

186 samples

0 10 20 30 40 50 60 70

Archae

Artificial

Bacterial

Bovine

Horseshoe crab

Human

Insect

Jellyfish

Lama

Mouse

Phages

Other

Plant

Plasmodium

Platyhelminthes

Protozoa

Viral

Yeast

Type of sample

0 20 40 60 80 100 120 140

Secreted

Cytoplasmic

Membrane protein without any trans-membrane domain (Ecto domain or

intracellular domain)

Membrane protein including part or all ofthe trans-membrane domain

Other

Samples are mainly of human and bacterialorigin. The majority of them corresponds to cytoplasmic proteins

The guideline: minimal information that needs to be provided or known

0,00% 20,00% 40,00% 60,00% 80,00% 100,00%

Full peptidic sequence

NCBI or Uniprot ascession nuber

Cloning strategy

Theoretical extinction coefficient

Measured extinction coefficient

0% 10% 20% 30% 40% 50% 60% 70% 80% 90%

Bacterial

Cell Free

Insect

Mammalian

Natural source

Synthetic

Yeast

Other:

Known informations on the sample Knowledge of the system of production

< 100%

~80%

Knowledge of the storing conditions

0% 20% 40% 60%

280nm with cuvette spectrophotometer

280nm with Nanodrop

Amino Acid Analysis

Fourier Transform InfraRed (Direct detect...)

Bradford assay

BCA assay

Lowry assay

Refractometry

Not performed

other

0,00% 20,00% 40,00% 60,00% 80,00% 100,00% 120,00%

Final buffer composition

Storage Temperature

Lyophilisation condition

Not known

Knowledge of the concentration, and method used for [P] determination

The guideline: minimal information that needs to be provided or known

Method used for control of purity

The guideline: minimal quality control parameters

3.3%


Thus, only 67% of the samples were tested for identityand/or integrity and, out of these, 21% (ca. 1 out of 5) gave masses or sequences that did not correspond to the anticipated values

31%

69%

Failed at least one of the tests

failed

passed

Out of those samples that have been tested for the three criteria (i.e. purity, homogenity and identity), 1/3 did not pass the three controls (i.e. failed at least one of the minimal tests)


0% 5% 10% 15% 20% 25% 30% 35% 40%

Antibody production

Biochemical studies

Molecular Biophysics

Structural determination

In vivo studies

Cellular biology

HTS

Downstream application The outcome of downstream applications was known for 130 out of the 186 samples

Among these 130 samples, we consideredthose that were tested for the 3 criteria (i.e. putity, homogeneity, integrity) and that failedat least one of them:26% failed in the downstream application;53% suceeded only partly;21% fully succeeded in the downstreamapplication.

In conclusion, only 1/5 did fully succeed in downstream applications


Failed one of the minimal test

Failed Succeed partly Succeed

53%

21% 26%

Correlation between favourable QC results and "satisfactory" results obtained in downstreamapplications.

Out of these 130 samples, among those that passed the three criteria (i.e. putity, homogeneity, integrity), 74% yielded satisfactory results, 20% suceeded only patially and 6% failed in the downstream application.In contrast, most (ca. 80%) of the samples that failed one or more of the minimal QC tests didnot give satisfactory results in downstream applications.


6%20%

26% 21%

53%

74%

69%

31%

So what is our proposal?

Guidelines with minimal information that needs to be known before using a protein

Our guidelines are the result from the collective expertise of ca. 150 laboratories

lay the groundwork for the standardization and reproducibility of data in any laboratory that produces recombinant proteins

encourage the whole scientific community to implement this QC methodology

raise awareness amongst colleagues and collaborators

reverse the trend for less and less information available in mainsteam journals on theproduction and purification of laboratory reagents

lobby editors to encourage/oblige more extensive use of supplementary data, and report the results of each check-point in research papers

improve reproducibility between labs + increase confidence in published data

Identity HomogeneityPurity Integrity

“When papers are written and data are presented in public it

looks like everything is just perfect, and that is not what

science is.

Science is an imperfect human activity that we try to do as

best we can.”

Bjorn Olsen, Harvard cell biologist.

Slide from Dr. Mario Lebendiker , WOLFSON CENTRE FOR APPLIED STRUCTURAL BIOLOGY, The ProteinPurification Facility, HEBREW UNIVERSITY OF JERUSALEM - PEGS Europe in Lisbon, November 3rd 2016

References on the subject

The QC team of ARBRE/MOBIEU and P4EU

Nick Berrow - Institute for research in Medicine, Barcelona, Spain (P4EU)

Maria Garcia-Alai - EMBL Hamburg, Germany (ARBRE-MOBIEU)

Stefan Knauer - Bayreuth University, Germany (ARBRE-MOBIEU)

Mario Lebendiker - Hebrew University of Jerusalem, Israel (P4EU)

Blanca Lopez-Mendez - University of Copenhagen, Denmark (ARBRE-MOBIEU)

Ario de Marco - University of Nova Gorica, Slovenia (P4EU and ARBRE-MOBIEU)

André Matagne - University of Liège, Belgium (ARBRE-MOBIEU)

Annabel Parret - EMBL Hamburg, Germany (P4EU)

Bertrand Raynal - Pasteur Institute, Paris, France (ARBRE-MOBIEU)

Kim Remans - EMBL Heibelberg (P4EU)

Stephan Uebel - Max-Planck Institute for Biochemistry, Munich (ARBRE-MOBIEU)

In representation of:

Acknowledgement (merci!!!)

improving the quality of protein samples for better data...

Documents