peer review and science2.0
Post on 06-May-2015
1.762 Views
Preview:
DESCRIPTION
TRANSCRIPT
Peer Review and Science2.0: blogs, wikis and social
networking sites
Jean-Claude Bradley
March 15, 2010
Guest Lecture for “Peer Review Culture in Scholarly Publication and Grantmaking” course at
Drexel University
Associate Professor of ChemistryDrexel University
What Peer Review Does
1. Provides authors with a publication format that counts in academia.
2. Assists editors to determine if the findings of an article are consistent with the data provided and commonly accepted scientific models.
3. Assists editors in attempting to provide a consistent style and focus for a particular journal. (not fundamental – see PLoS ONE)
What Peer Review does not do
Verify the analysis of all the raw data supporting an article.
Even if peer-reviewing tried to take on the responsibility of verifying all the raw data, there are not enough
resources to cope.
TRUST
PROOF
A solution that scales: Open Notebook Science
How bad is our current system? Try to find the solubility EGCG?
=2.3 g/L
WTF?!
The End of the Chain of Provenance
The Tragedy of the Trusted Source Cascade
The NaH oxidation controversy
Information spreads quickly through the blogosphere
15% NMR yield
Khalid Mirza and Marshall Moritz
Top results on a Google search
The Scandal of Bell’s Lab Notebook
Motivation: Faster Science, Better Science
Open Notebook Science Logos (Andy Lang, Shirley Wu)
Sharing: how much and when
There are NO FACTS, only measurements embedded
within assumptions
Open Notebook Science maintains the integrity of data
provenance by making assumptions explicit
The solubility of 4-chlorobenzaldehyde
The Log makes Assumptions Explicit
The Rationale of Findings Explicit
Raw Data Made Public
Splatter?
Some liquid
YouTube for demonstrating experimental YouTube for demonstrating experimental set-upset-up
Calculations Made Public on Google Spreadsheets
Revision History on Google Spreadsheets
Wiki Page History
Comparing Wiki Page Versions
Proof of Purity with interactive NMR spectrum using JSpecView and
JCAMP-DX
Linking to Molecules in Chemistry Databases
Experimental Spectra and User-Deposited Data on ChemSpider
(Andy Lang, Tony Williams)
Open Data JCAMP spectra for education
(Andy Lang, Tony Williams, Robert Lancashire)
Database Curation via Game Playing
Over 100,000 spectrum views so far - worldwide
Link Spectral Game to Open Educational Content
The Ugi reaction: can we predict precipitation?
Can we predict solubility in organic solvents?
Crowdsourcing Solubility Data
ONS Submeta Award Winners
ONS Challenge Judges
Teaching Lab: Brent Friesen (Dominican University)
Solubility Experiment List
Solubilities collected in a Google Spreadsheet
Rajarshi Guha’s Live Web Query using Google Viz API
WE ARE HEREWE ARE HERE
How can the scientific process become more automated?
Semi-Automated Semi-Automated Measurement of solubility via Measurement of solubility via
web service analysis of web service analysis of JCAMP-DX files JCAMP-DX files
(Andy Lang)(Andy Lang)
Solubility Measurement Requests: DoSol sheet
•Outlier Bot: flags measurements with high standard deviation to mean ratios•Google Analytics queries – new solvent/solute searches•Solubility request form – researcher in Israel requesting pyrene in acetonitrile solubility for environmental soil contamination study•Application based models – high priority Ugi reactants
Solubility Prediction (Andy Lang’s Model)
Understanding in addition to empirical modeling
Missed in a prior publication on
solubility for this compound
Data provenance: From Wikipedia to…
…the lab notebook and raw data
Including links to the literature
•Concentration (0.4, 0.2, 0.07 M)•Solvent (methanol, ethanol, acetonitrile, THF)•Excess of some reagents (1.2 eq.)
How does Open Notebook Science fit with traditional publication?
Paper written on Wiki
References to papers, blog posts, lab notebook pages, raw
data
Paper on Journal of Visualized Experiments (JoVE)
Pre-print on Nature Precedings
ChemSpider Automated Mark-up of Chemical Names
BUT…
Open Access: the Choice that Keeps Giving.. and Giving…
Beware of your addiction to metrics: redundancy will reduce
them
Cameron Neylon’s NotebooksCameron Neylon’s Notebooks
Other Open NotebooksOther Open Notebooks
Anthony Salvagno’s Notebook Anthony Salvagno’s Notebook (Steve Koch group)(Steve Koch group)
Archiving Open Notebook Science Projects
What is the role of the librarians, researchers and other parties?
What are options for citing?
Librarians and Science 2.0"The Internet Archive is a 501(c)(3) non-profit that was founded to build an Internet library, with the purpose of offering permanent access for researchers, historians, and scholars to historical collections that exist in digital format."
The internet Archive is not practical for practitioners of
Open Notebook Science or
Science 2.0
Good concept but.....
Most pages look like this....
Where We Began: The ONS backup spreadsheet and ONSPreserver
Publishing Google Spreadsheets as XLS
Where We Are Now
ONSArchive: Semi-Automated Snapshot of the Entire Scientific Record
Snapshot is Self-Contained and Live on the Internet
Lulu.com Data Disks
DSpace – Handle (hdl)
Lulu.com - ISBN
Google Spreadsheet
s
Google Documents
Web Services
ChemSpider & Indiana
Real Time Linear Regression, Unit
Conversions, Style Sheet, etc
Data Book
Bradley, Jean-Claude; Lang Andrew. Solubilities Summary Sheet. Open Notebook Science Challenge. 2009-12-11. URL:http://spreadsheets.google.com/pub?key=plwwufp30hfq0udnEmRD1aQ&output=xls. Accessed: 2009-12-11. (Archived by WebCite® at http://www.webcitation.org/5lx5ry3BV)
Book Editions on Nature Precedings
More about the ONSarchive project:
1) Accept that reporting science in real time is not always pretty. Do your best to avoid and correct mistakes as soon as possible but mistakes and ambiguous results will happen on the way to completing any scientific project. Just be honest about your level of certainty when discussing preliminary results.
2) Provide as much raw data as is reasonable and frame it in such a way that other researchers can understand what you have done and follow your conclusions based on your data without having to ask you questions.
3) Don't wait for the perfect technological solutions before starting to share. General purpose wikis can serve as an excellent starting point for an Open Lab Notebook.
4) Don't wait for the perfect data structuring scheme before starting to share. First share for human readability - you can always restructure the data later for machine readability.
Open Notebook Science Tips - I
5) Periodically write summaries of your research progress in the form of milestones or significant challenges in a format that non-specialists can understand. A blog is a good platform for this. If you link to specific lab notebook pages from your summaries, experts can always click through to dig deeper.
6) Create snapshot archives of your notebooks and supporting raw data files. You can use these as backups and as a convenient way to cite a particular version of your entire research project.
7) Cite specific lab notebook pages and archives when publishing in peer-reviewed journals.
Open Notebook Science Tips - II
top related