implementing web-scale discovery in an academic research ... · • participants were asked 4...
TRANSCRIPT
Implementing Web-Scale Discovery
in an Academic Research Library
Usability Testing
• November 2008
– Tammy Allgood, Web Services Librarian
– Evaluation of library web site home page
• July 2009• July 2009
– Tammy Allgood, Web Services Librarian and Cindi
Farmer, Provost Web and Publication Design
Group
– Evaluation of MyASU student portal search box
Interfaces Tested
Methodology
• Participants were asked 4 questions about library research and then asked to complete 12 tasks.
• Tasks were written to simulate the types of information needs students have when coming to the ASU Libraries web site.
• The students were observed directly and asked to perform • The students were observed directly and asked to perform a cognitive walkthrough, verbally explaining their decisions as they attempted to complete the tasks.
• Completion of tasks are measured on the following scale:– Unsuccessful.
– Successful with extra effort.
– Successful.
Summary of Observed Behavior
• Users often go to "Online Catalog" first thinking it is searching all of the Library resources (and don't get results for journal articles).
• Users don't know what database to go to for articles.articles.
• The research-savvy users who have past experience with Google Scholar found most things quickly.
• There is general user dissatisfaction with the distributed nature of library systems.
Comparison Criteria
Summon (http://www.serialssolutions.com/summon/)
WorldCat (http://www.worldcat.org/)
Google Scholar (http://scholar.google.com/) Encore
Implementation time (workflow impact low/med/high)
Low implementation time (very little disruption in local workflows)
High implementation time (would be disruptive to BAMS workflows / workflows would have to be revised)
Medium implementation time Medium implementation time
Comprehensive content coverage (high-med-low)
High content coverage Medium content coverage Medium content coverage Medium content coverage
Licensed Content Serials Solutions have our Contracts/content must be N/A SomeLicensed Content Negotiation & Contracts
Serials Solutions have our holdings (easy implementation)
Contracts/content must be licensed through WorldCat to be included in holdings search (high impact).
N/A Some
Catalog integration Would need to be indexed by Summon (part of 6 week startup time)
Most Records Already in Worldcat (no coverage of those that are not)
None Yes
Programmatic access to data (API)
High None None None
Privileged local content in search results (show local content first)
Yes Yes No Yes
Allows local repository content
Yes Records must be in WorldCat catalog
Available through crawling; inclusion through Google Scholar is unknown
Yes
Portal integration Yes No No No
Federated No No No Yes
Implementation Team
• Web Services Librarian
• Metadata Librarian
• Cataloger
• Head of Bibliographic and Metadata Services• Head of Bibliographic and Metadata Services
• ILS Admin
• Media Production Manager
• Marketing Director
Implementation
• Purchased product
• Name/Icon Testing - September
• Library Staff Info Sessions - October
• Provost Committee Info Presentation - October
• Internal Beta Testing - End of October • Internal Beta Testing - End of October
• Beta Release - 11/11/09
• Post Beta User Testing - November/December
• Library Staff Training Sessions - January
• Full Release - 1/5/10
Rebranding Summon
Student Advisory Committee
• Voted for “Library One Search”
• Quotes from students:
– “I find it hilarious that both Library Single Search and Library Multi Search are in the running...”and Library Multi Search are in the running...”
– “Questions: Why won't every single item be indexed? What would be the point of a search box such as this if every single item is not indexed? Don't you think this will create a black hole of items that will never be used or accessed?”
Coverage Information
• Electronic Serial coverage is tricky to calculate... 97% of our 100 top accessed journals
• 91.28% of all electronic items with unique ISSNs
• http://www.serialssolutions.com/assets/publications/Summon-represented-titles.pdfons/Summon-represented-titles.pdf
• “The Library One Search Searches most, but not all of our resources. If you are not finding what you expect or for a more comprehensive search, consult your subject librarian. They can assist you with other resources that are not included in the Library One Search.”
ASU Portal
Blackboard
LibGuides
Toolbar
Marketing
Internal Marketing
Usage
User Feedback
• Web Site Survey
• 190 users
• Incentive – book store gift certificates
User Feedback
User Feedback
User Feedback
User Feedback
User Testing
Results
• Currently collecting
• Obvious differences between students and
faculty
• Future publication• Future publication