scientific papers as open discovery tools
DESCRIPTION
Presentation given at the Advanced Lecture Course on Systems Biology 2014, in Innsbruck.TRANSCRIPT
EMBO Scientific Publishing
Thomas Lemberger
Chief Editor, Molecular Systems Biology
Deputy Head of Publications, EMBO
1. Editorial Process
2. Scientific integrity
3. Integrating data in papers
Scientific publishing
“The publication of scientific information is intended to move science forward. More specifically, the act of publishing is a quid pro quo in which authors receive credit and acknowledgment in exchange for disclosure of their scientific findings.”
disclose findings
credit
move science forward
critical evaluation
critical evaluation
editorial process
editorial process @EMBO
EMBO Scientific Publications
• All areas of molecular & cell biology
• First journal launched by EMBO (1982)
•All areas of molecular & cell biology
•Short-format papers
•Science & Society section
•At the interface between basic and clinical life sciences
•Open Access
•Systems biology, synthetic biology, systems medicine
•Open Access
quality
quality
community
EMBO Editors:Thomas LembergerMaria Polychronidou
R Aebersold
GM Church
L Hood
E Liu
P Bork
Julie AhringerCharles AuffrayEwan BirneyTom BlundellThomas S. DeisboeckJan EllenbergMichael ElowitzAlan FershtStan FieldsMark GersteinFrank HolstegeSung Hou KimHiroaki KitanoDoron LancetAndrew J. LinkStephen OliverJeremy NicholsonBernhard PalssonRama RanganathanUwe SauerLuis SerranoLucy ShapiroPamela SilverMichael SnyderJanet ThorntonMasaru TomitaMarc VidalHans V. WesterhoffLothar WillmitzerJohn Yates
Senior Editors Advisory Board
Scope & general policies
OPENACCESS
The editorial process
editorialrejection
review
reject revise
reject accept
time
First editorial decision
editorialrejection
review
reject revise
reject accept
time
First editorial decision
editorialrejection
review
reject revise
reject accept
time
First editorial decision
EMBO editors read the entire manuscript (yes!)
Decision on a balance of multiple factors:
• Scope• Novelty & conceptual advance• Mechanistic, functional, biological insights• Utility of methods, dataset, resource• Completeness and conclusiveness of the analysis
In case of doubt...R Aebersold
GM Church
L Hood
E Liu
P Bork
Julie AhringerCharles AuffrayEwan BirneyTom BlundellThomas S. DeisboeckJan EllenbergMichael ElowitzAlan FershtStan FieldsMark GersteinFrank HolstegeSung Hou KimHiroaki KitanoDoron LancetAndrew J. LinkStephen OliverJeremy NicholsonBernhard PalssonRama RanganathanUwe SauerLuis SerranoLucy ShapiroPamela SilverMichael SnyderJanet ThorntonMasaru TomitaMarc VidalHans V. WesterhoffLothar WillmitzerJohn Yates
Senior Editors Advisory Board
EMBO Editors:Thomas LembergerMaria Polychronidou
Initial editorial decision
To review or not to review...
editorialrejection
review
reject revise
reject accept
time
Peer-review
Referees are invited based on:
• Balance of expertise• Reputation as researcher• Reputation as reviewer• No conflict of interest (positive or negative)
3 (4) reviewers / manuscript
Transparent peer review
1. Summary• Describe your understanding of the story• What are the key conclusions: findings and concepts• What are the methodology and model system
2. General remarks• Are you convinced of the key conclusions?• Place the work in its context.• What is the nature of the advance (conceptual, technical, clinical)?• How important is the advance as compared to previous knowledge?• What audience will be interested in this?
3. Major points• Specific criticisms related to key conclusions• Specify experiments or analyses required to demonstrate the conclusions• Motivate your critique with relevant citations and argumentation
4. Minor points• Easily addressable points• Presentation and style• Trivial mistakes
Referee report
Referee report: example
Editorial decision letter:
Cross-commenting
reviewersreviewersreportsreports
authors
“…[it] settles a controversy in the field which has been going on for more than ten years. In summary, this is a landmark paper. I cannot support publication in your journal strongly enough!”
“As written, the paper is focused on the methods, which is fine given that's where it makes its most substantial contribution. But the writing is quite technical and could benefit from more explanation of the high-level logic of their approach.”
“After reading through this nicely-executed technical work, one is left with an impression that after all we really have not gained much new mechanistic insights.…in addition to the XXX data that should be generated under their current framework…”
…Each reviewer has numerous suggestions about how to do this. It will likely be impossible to incorporate them all while retaining a coherent narrative. […]
Reviewer 3 also calls for an additional experiment - including XXX stains in the current dataset. To incorporate this into their current analytical framework, the authors would have to find parameters and reagents to allow simultaneous imaging of 5 genes (not just the 4 presented here). Moreover, they would then have to reacquire all images using the 5-stain protocol.
While I agree that it would be useful to have XXX data included, I also believe that this is beyond the scope of this paper.
Ref
#1
Ref
#2
Ref
#3
Ref #2 cross-comments
Cross-commenting
Post-revision review
editorialrejection
review
reject revise
reject accept
time
Transparent Process• Transparent Anonymous Peer Review:
Anonymous referee reports and editorial correspondence are published alongside papers
• Single Round of Major Revision:More than 90% of invited revisions are published at Molecular Systems Biology
• Referee Discussion before Decision:Referees are invited to comment on each other's reports before the editor makes a decision
• Scooping Protection:Findings that are published by others during review or revision are not a criterion for rejection
• Source Data for Figures:Authors can archive and make available the data underlying their published figures
• Flexible Formatting:No journal-specific formatting required at submission
Scientific publishing
“The publication of scientific information is intended to move science forward. More specifically, the act of publishing is a quid pro quo in which authors receive credit and acknowledgment in exchange for disclosure of their scientific findings.”
Scientific publishing
The publication of scientific information is intended to move science forward. More specifically, the act of publishing is a quid pro quo in which authors receive credit and acknowledgment in exchange for disclosure of their scientific findings.
Implies:•Originality•Integrity•Authenticity
Respect of:•Laws and ethics•Safety and security•Societal context
Beautification Fabrication
• Clarification • Aesthetics
• Deliberate manufacturin of data
Data integrity
Selective reporting
• Misrepresentation of the data
• Biasing data to fit a particular hypothesis
38
39
New submission:
Seoul National University's report on Dr. Hwang Woo Suk:
The data in the 2005 article including test results from •DNA fingerprinting, •photographs of teratoma,•embryoid bodies, •MHC-HLA isotype matches and•karyotyping have all been fabricated.
http://www.useoul.edu/snunews?bm=v&bbsidx=71494&page=63
Figure 2F Figure 6D
(D) Human NT-ESCs expressed standard pluripotency markers detected by immunocytochemistry for antibodies against OCT4, NANOG, SOX2, SSEA-4, TRA-1–60, and TRA-1–81. Original magnification, ×200; Ph, phase contrast.
(F) NT-ESC colony with typical morphology derived from a caffeine-treated SCNT human blastocyst.
Figure 2F Figure 6D
(D) Human NT-ESCs expressed standard pluripotency markers detected by immunocytochemistry for antibodies against OCT4, NANOG, SOX2, SSEA-4, TRA-1–60, and TRA-1–81. Original magnification, ×200; Ph, phase contrast. Note that the upper-left image for hESO-NT1 is the same shown in Figure 2F.
(F) NT-ESC colony with typical morphology derived from a caffeine-treated SCNT human blastocyst.
In Figures 2F and S5 (upper-right), we presented two phase-contrast photos of fields of cells, correctly labeled as SCNT-derived hESO-NT1 and IVF-derived hESO-7, respectively. These images are the same fields of cells shown in the top two images of Figure 6D; however, in Figure 6D, we inadvertently switched the labels on the images. This re-use of the images was intentional, but we should have indicated this in the original legend for Figure 6. We have corrected the labeling error in Figure 6D.
We would also like to note that the Scientific Integrity Committee at Oregon Health & Science University has carefully assessed the paper and the errors and has concluded that there is no evidence of fabrication, falsification, or plagiarism that would warrant further inquiry or investigation into research misconduct.
In Figures 2F and S5 (upper-right), we presented two phase-contrast photos of fields of cells, correctly labeled as SCNT-derived hESO-NT1 and IVF-derived hESO-7, respectively. These images are the same fields of cells shown in the top two images of Figure 6D; however, in Figure 6D, we inadvertently switched the labels on the images. This re-use of the images was intentional, but we should have indicated this in the original legend for Figure 6. We have corrected the labeling error in Figure 6D.
We would also like to note that the Scientific Integrity Committee at Oregon Health & Science University has carefully assessed the paper and the errors and has concluded that there is no evidence of fabrication, falsification, or plagiarism that would warrant further inquiry or investigation into research misconduct.
• Provide guidance to students and postdocs• View original data • Organize good data management practice• Maintain an open lab environment• Accept only relevant authorship• Cooperate with editors• Retract/correct as appropriate
What can PIs do?
• Rigorous peer review• Check by editors before publication• Investigation and retraction policy• Data transparency
What can journals do?
Title
Abstract
Synopsis
Main paper
‘Expanded view’
Datasets & code
What is a paper?
A scientific result converted into a collection of pixels…
8/27
What is a figure?
11/27
12/27
•Data archival
service
•Data‘transparency’
•Data reuse
•Data-oriented
search
(Level 0:metadata associated to individual panels.)
Level 1: ‘object-oriented’ representation of experimental variables as a list of chemical and biological components.
Level 2: represent the causality of the experimental design: “Measurement of Y as a function of A, B, C, using assay P in biological system S.”
Level 3: machine-readable representation with standard identifiers.
measured componentmeasured component
perturbed componentperturbed component
experimental system
15/27
assayed property
Structured metadata:‘perturbation-observation-assay’
Tools to publish figures as structured digital objects
that link the human-readable illustrations with
machine-readable metadata and ‘source data’ in
order to
•improve data transparency;
•make published data useable;
•enable data-oriented search.
9/27
SourceData
Resulting hypothesis: test drug Z in disease D.
tissue Ttissue T disease D
disease D
gene xgene x
Pap
er 3
protein X protein X PPkinase Ykinase Y
Pap
er 2
kinase Ykinase Y activityactivitydrug Zdrug Z
Pap
er 1
Data-oriented search
19/27
Smad3
Hey1
TGFbetaVE-cdh
Rad51 foci
AR
Tsc2
1 4
6 2 5
3
1,4
4
5
6
2
…
…
Rad51Nuclear
complexesTGFb, Smad3
Scientific publishing
• Dominant channel for the dissemination of peer-reviewed data.
• Journals function as a proxy for quality in research assessment
• The rate of publishing keeps
increasing.• Papers are human-readable but
poorly machine-readable.
search
The Paper of the Future?
<!DOCTYPE article PUBLIC "-//NLM//DTD Journal Archiving and Interchange DTD v2.3 20070202//EN" "archivearticle.dtd"><article article-type="research-article"><?properties no_embargo?> <front> <journal-meta> <journal-id journal-id-type="nlm-ta">Mol Syst Biol</journal-id> <journal-title>Molecular Systems Biology</journal-title> <issn pub-type="epub">1744-4292</issn> <publisher> <publisher-name>Nature Publishing Group</publisher-name> </publisher> </journal-meta> <article-meta> <article-id pub-id-type="pmc">2238715</article-id>
Search
Future directions in systems biology
• Genome-wide genetics of human diseases • Translational systems biology or systems
medicine• Genome-scale engineering & synthetic biology• Temporal structure of biological processes• ‘In vivo biochemistry’ with single-cell single-
molecule assays• Bridge the gap between ‘omics’ and mechanistic
models
“How do we get from the Jimome & Craigome to systems biology?”George M Church, Senior Editor
Multi-omics data integrationIntegration of clinical data with a genome-scale metabolic model of the human adipocyte.Jens Nielsen and colleagues, Mol Syst Biol. 2013;9:649.
Global analysis of genome, transcriptome and proteome reveals the response to aneuploidy in human cells.Zuzana Storchova, Mol Syst Biol. 2012 8:608.
Comparative omics for functional discoveries
19 0
17 h
uman
gen
es
Phylogenetic profiles of across 86 eukaryotic genomes.
Human disease locus discovery and mapping to molecular pathways through phylogenetic profiling.
Gary Ruvkun and colleagues, Mol Syst Biol. 2013 9:692
??
Beyond the hairball
Costanzo et al, Science 2010 Hoog et al, Dev Cell 2007
Spatial patterns
Di Vetura and Sourjik , 2011 Mol Syst Biol 7:457
Waks et al, 2011 Mol Syst Biol 7:506
Temporal patternsPromoter decoding of transcription factor dynamics involves a trade-off between noise and control of gene expression.Hansen AS, O'Shea EK. Mol Syst Biol. 2013 9:704
Cell population & dynamicsDigital cell quantification identifies global immune cell dynamics during influenza infection.Ido Amit and colleagues, 2014 Mol Syst Biol. 10:720
Microbiome & metagenomicsA top-down systems biology view of microbiome-mammalian metabolic interactions in a mouse model.J. Nicholson and colleagues, Mol Syst Biol. 2007 3:112.
Toward molecular trait-based ecology through integration of biogeochemical, geographical and metagenomic data.Peer Bork and colleagues, Mol Syst Biol. 2011 7:473
Synthetic biologySynthesizing a novel genetic sequential logic circuit: a push-on push-off switch.Qi Ouyang (Beijing University), Mol Syst Biol. 2010;6:350.
Genome-scale engineeringGenome-scale engineering for systems and synthetic biology.Esvelt KM, Wang HH, Mol Syst Biol. 2013 9:641.