schultheiss bosc2010 persistance-web-services
TRANSCRIPT
BOSC 2010Boston, July 9–10
Sebastian J. Schultheiss<[email protected]>
Persistence and availability of bioinformatics web services
TL-Stiftung
1Saturday, July 10, 2010
‣ Curated data set: Nucleic Acids Research Web Server Issues 2003-2009
‣ 927 web services
‣ 322 institutions
‣ 39 countries
‣ 827 corresponding authors (274 replies)
The StudyS. J. Schultheiss et al. (2010) PLoS Comp Biol (under review)
2Saturday, July 10, 2010
‣ Original web address unreachable for 17% of services
The ProblemVeretnik et al. (2008) PLoS Comp Biol 4:e1000136
3Saturday, July 10, 2010
‣ Research is based on existing services
‣ Reproducibility, comparability
‣ Improving methods made difficult
The Problem
4Saturday, July 10, 2010
‣ More recent services are still reachable, have higher quality standards
‣ NAR publishing policies became stricter
Developments over Time
5Saturday, July 10, 2010
6Saturday, July 10, 2010
Long-Term Availability Score
7Saturday, July 10, 2010
Average CitationsS. J. Schultheiss et al. (2010) PLoS Comp Biol (under review)
2003 2004 2005 2006 2007
14
911
47
43
18
129
7
40
17
12
8 7
13%
20% 19%
16%
9%
Average CitationsReachable: Average CitationsUnreachable: Average Citations% Unreachable
8Saturday, July 10, 2010
‣ 64% of services used by researchers without computational background
‣ 58% of services developed by students only, difficult to maintain after graduation
‣ 24% of services will not be maintained
Survey among NAR AuthorsS. J. Schultheiss et al. (2010) PLoS Comp Biol (under review)
9Saturday, July 10, 2010
Web Service Problems# %
Unreachable, web site down
No example datasets
No help text/documentation
Implausible arrangement of interface elements/not intuitive to use
Too stringent limitations (e.g. on file size, number of sequences, ...)
Processing/waiting time unreasonably long
No response upon personal e-mail or on mailing list, no support
Bad design choices (colors, size of edit fields, ...)
Missing contact information
132 48%
110 40%
109 40%
99 36%
87 32%
77 28%
66 24%
53 19%
24 9%
10Saturday, July 10, 2010
1. Consider a stand-alone version
2. Know your audience
3. Use an existing framework
4. Make it portable
5. Provide documentation and assistance
Ten Simple RulesS. J. Schultheiss (2010) PLoS Comp Biol (under review)
11Saturday, July 10, 2010
6. Assist users and involve the community
7. Be explicit about changes
8. Leave a forwarding address
9. Find someone else to do it
10. Plan the end of the service life cycle
Ten Simple RulesS. J. Schultheiss (2010) PLoS Comp Biol (under review)
12Saturday, July 10, 2010
‣Available services cited 2.2x more often
‣Authors: use persistent URLs (PURL, DOI, own domain, ...) and release source code
‣ Prepare to hand over responsibilities
‣ LT score predicts reliability of services for editors and reviewers
Summary
13Saturday, July 10, 2010
Acknowledgements‣ Marc-Christian Münch and Gergana Andreeva
‣ Gunnar Rätsch and AG Rätsch/MLB at the FML
‣ Oliver Kohlbacher, WSI, University of Tübingen
‣ TL Foundation and its Board at University of Tübingen
‣ ISCB, DOE and NSF for a Travel Fellowship
TL-Stiftung
PosterT5
14Saturday, July 10, 2010