reproducible research sergey fomel the university of texas at austin
Post on 19-Dec-2015
213 views
TRANSCRIPT
![Page 1: Reproducible Research Sergey Fomel The University of Texas at Austin](https://reader031.vdocuments.us/reader031/viewer/2022032800/56649d375503460f94a1021c/html5/thumbnails/1.jpg)
Reproducible Research
Sergey Fomel
The University of Texas at Austin
![Page 2: Reproducible Research Sergey Fomel The University of Texas at Austin](https://reader031.vdocuments.us/reader031/viewer/2022032800/56649d375503460f94a1021c/html5/thumbnails/2.jpg)
Outline• Personal experience
• Why do reproducible research?
• How to do reproducible research?
* Reproducible computational experiments
![Page 3: Reproducible Research Sergey Fomel The University of Texas at Austin](https://reader031.vdocuments.us/reader031/viewer/2022032800/56649d375503460f94a1021c/html5/thumbnails/3.jpg)
Personal Experience• Jon Claerbout and RR at Stanford
•Madagascar open-source software
• CiSE special issue (Jan-Feb 2009)– David Donoho et al– Randall LeVeque – Roger Ping and Sandrah Eckel– Victoria Stodden
![Page 4: Reproducible Research Sergey Fomel The University of Texas at Austin](https://reader031.vdocuments.us/reader031/viewer/2022032800/56649d375503460f94a1021c/html5/thumbnails/4.jpg)
Jon Claerbout’s Story
• 1987: Sunview experience– Interactive programs are slavery
• 1992: LaTeX + cake– Rebuilding books by a single command
![Page 5: Reproducible Research Sergey Fomel The University of Texas at Austin](https://reader031.vdocuments.us/reader031/viewer/2022032800/56649d375503460f94a1021c/html5/thumbnails/5.jpg)
Reproducible Research at SEP• Stanford Exploration Project
– Founded in 1973– 2 Ph.D. students per year
• Reproducible research– From CD-ROMs to WWW– From cake to GNU make– 2001 CiSE paper
• The principal beneficiary is the author
![Page 6: Reproducible Research Sergey Fomel The University of Texas at Austin](https://reader031.vdocuments.us/reader031/viewer/2022032800/56649d375503460f94a1021c/html5/thumbnails/6.jpg)
The Madagascar Project• Multidimensional data analysis• Started in 2006• Open community• Open source (GPL)• Three levels
– Building blocks in C– Recipes in Python/SCons– Papers in LaTeX + SCons
http://ahay.org
![Page 7: Reproducible Research Sergey Fomel The University of Texas at Austin](https://reader031.vdocuments.us/reader031/viewer/2022032800/56649d375503460f94a1021c/html5/thumbnails/7.jpg)
Personal Experience• Jon Claerbout and RR at Stanford
•Madagascar open-source software
• CiSE special issue (Jan-Feb 2009)– David Donoho et al– Randall LeVeque – Roger Ping and Sandrah Eckel– Victoria Stodden
![Page 8: Reproducible Research Sergey Fomel The University of Texas at Austin](https://reader031.vdocuments.us/reader031/viewer/2022032800/56649d375503460f94a1021c/html5/thumbnails/8.jpg)
CiSE Reproducible Research• David Donoho, Arian Maleki, Inam Rahman,
Morteza Shahram, Victoria Stodden
• 15 years of reproducible research in computational harmonic analysis– MATLAB– WaveLab: 690 citations– “Striving for reproducibility imposes a
discipline that leads to better work.”
![Page 9: Reproducible Research Sergey Fomel The University of Texas at Austin](https://reader031.vdocuments.us/reader031/viewer/2022032800/56649d375503460f94a1021c/html5/thumbnails/9.jpg)
CiSE Reproducible Research• Randall J. LeVeque
• Python tools for reproducible research on hyperbolic problems– Fortran + Python– Clawpack: 7,000 registered users– “Scientific and mathematical journals
are filled with pretty pictures of computational experiments that the reader has no hope of repeating.”
![Page 10: Reproducible Research Sergey Fomel The University of Texas at Austin](https://reader031.vdocuments.us/reader031/viewer/2022032800/56649d375503460f94a1021c/html5/thumbnails/10.jpg)
CiSE Reproducible Research• Roger D. Peng and Sandrah P. Eckel
• Distributed reproducible research using cached computations– R language– Cacher package– “We propose that a modular research
approach lends itself more naturally to reproducible results.”
![Page 11: Reproducible Research Sergey Fomel The University of Texas at Austin](https://reader031.vdocuments.us/reader031/viewer/2022032800/56649d375503460f94a1021c/html5/thumbnails/11.jpg)
CiSE Reproducible Research• Victoria Stodden
• The legal framework for reproducible research in the sciences– Licensing and copyright– ORL (Open Research License)– “We need a license designed with the
needs of computational researchers in mind.”
![Page 12: Reproducible Research Sergey Fomel The University of Texas at Austin](https://reader031.vdocuments.us/reader031/viewer/2022032800/56649d375503460f94a1021c/html5/thumbnails/12.jpg)
Why Reproducible? Science is the systematic enterprise of
gathering knowledge about the universe and organizing and condensing that knowledge into testable laws and theories. The success and credibility of science are anchored in the willingness of scientists to independent testing and replication by other scientists. This requires the complete and
open exchange of data, procedures and materials.
![Page 13: Reproducible Research Sergey Fomel The University of Texas at Austin](https://reader031.vdocuments.us/reader031/viewer/2022032800/56649d375503460f94a1021c/html5/thumbnails/13.jpg)
Open-Source Software
“Abandoning the habit of secrecy in favor of process transparency and peer review was the crucial step by which alchemy became chemistry. In the same way, it is beginning to appear that open-source development may signal
the long-awaited maturation of
software development as a
discipline.” Eric S. Raymond
![Page 14: Reproducible Research Sergey Fomel The University of Texas at Austin](https://reader031.vdocuments.us/reader031/viewer/2022032800/56649d375503460f94a1021c/html5/thumbnails/14.jpg)
How to Do Reproducible Computational Experiments?
• Code attached to published results
• Continuous maintenance
• Previous results used for testing– Test-driven development
• Lessons from open-source – Intellectual property– Community http://ahay.org
![Page 15: Reproducible Research Sergey Fomel The University of Texas at Austin](https://reader031.vdocuments.us/reader031/viewer/2022032800/56649d375503460f94a1021c/html5/thumbnails/15.jpg)