tools for reproducible research in an increasingly digital world
TRANSCRIPT
brian m. bot | sage bionetworks | @BrianMBot
mayo clinic - 2015 jan 28
tools for reproducible research
in an increasingly digital world
sage bionetworks~40 FTEs
1/2 research - 1/3 platform - 1/6 leadership/support
sage bionetworks
focused on a world where biomedical research will fundamentally change to be more open and collaborative
production
distribution
aggregation
6%
21%
8%
11%
54%cannot reproduce
can reproduce in principle
can reproduce w/discrepancies
can reproduce from processed data w/discrepancies
can reproduce partially
the status quo tolerates poor communication of findings
Ioannidis A. et al. Nature Genetics 2009
208,294,724 datapoints
124 pages supplemental material
?? lines unobtainable source code
?? version or architecture of statistical analysis program (R)
enumerable R packages and package dependencies
key R package “ClaNC” no longer available
1231 citations
often what is in principle reproducible, is not practically reproducible
unidentified publication‣ from journal with 5 year impact factor of 27‣ article freely available for download‣ data freely available for download
“Scientists often study the past as obsessively as historians because few
other professions depend so acutely on it. Every experiment is a conversation with
a prior experiment, every new theory a refutation of the old”
-Siddhartha Mukherjee, The Emperor of All Maladies
scientific method1. define a question
2. gather information and resources (background research)
3. form a hypothesis
8. retest (frequently done by other scientists)
4. test hypothesis experimentally
5. analyze experimental data
7. publish results
6. draw conclusions based on data
7. publish results
finitein
∞...
conducting research for others to consume
(even if the ‘other’ is future you)
reproducible research
tools for reproducible research
code
data
analysis
tools for reproducible research
code
version control
tools for reproducible research
code
version control
client-server
distributed
tools for reproducible research
code
version control
client-server(e.g svn, cvs)
tools for reproducible research
code
version control
client-server
distributed
tools for reproducible research
code
version control
(e.g git, mercurial)distributed
tools for reproducible research
code
version control
distributed
tools for reproducible research
code
data
analysis
tools for reproducible research
data
generic
domain repositories
results
tools for reproducible research
data
digital object identifier (doi)
a unique identifier which remains fixed over the lifetime of a web-accessible object
metadata, including the object’s location, is stored in association with the doi and may change over time
referring to an online document by its doi provides more stable linking than simply referring to a url
tools for reproducible research
code
data
analysis
tools for reproducible research
analysis
R Sweave knitr
great if you know LaTeX
tools for reproducible research
analysis
R Sweave knitr
great if you are lazy
(like me)
tools for reproducible research
analysis
knitr
# Hello World Title ### Author: Brian M. Bot
This is a narrative with inline code execution to tell me that pi is equal to `r pi`. And a plot to show a simple function.
```{r} x <- 1:100 y <- log(x)/x plot(x,y) ```
tools for reproducible research
analysis
knitr
# Hello World Title ### Author: Brian M. Bot
This is a narrative with inline code execution to tell me that pi is equal to `r pi`. And a plot to show a simple function.
```{r} x <- 1:100 y <- log(x)/x plot(x,y) ```
tools for reproducible research
analysis
ipython notebook
tools for reproducible research
other tools
galaxy
docker
packrat
shiny
tools for reproducible research
other tools
enables sharing of all resources (data, code, results) and their relationships to one another
tools for reproducible research
tools for reproducible research
tools for reproducible research
Go Hawks!
mayo clinic - 2015 jan 28
in an increasingly digital world
brian m. bot ——————
[email protected] @BrianMBot
sage bionetworks
tools for reproducible research