This article was downloaded by: [University of York]On: 01 October 2013, At: 03:26Publisher: RoutledgeInforma Ltd Registered in England and Wales Registered Number: 1072954 Registeredoffice: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK

Journal of Electronic Resources inMedical LibrariesPublication details, including instructions for authors andsubscription information:http://www.tandfonline.com/loi/werm20

A Study of Librarians' Perceptions andUse of the Summon Discovery ToolGinny M. Boyer a & Megan Besaw aa East Carolina University, Greenville, North Carolina, USAPublished online: 20 Aug 2012.

To cite this article: Ginny M. Boyer & Megan Besaw (2012) A Study of Librarians' Perceptions and Useof the Summon Discovery Tool, Journal of Electronic Resources in Medical Libraries, 9:3, 173-183, DOI:10.1080/15424065.2012.707056

To link to this article: http://dx.doi.org/10.1080/15424065.2012.707056

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all the information (the“Content”) contained in the publications on our platform. However, Taylor & Francis,our agents, and our licensors make no representations or warranties whatsoever as tothe accuracy, completeness, or suitability for any purpose of the Content. Any opinionsand views expressed in this publication are the opinions and views of the authors,and are not the views of or endorsed by Taylor & Francis. The accuracy of the Contentshould not be relied upon and should be independently verified with primary sourcesof information. Taylor and Francis shall not be liable for any losses, actions, claims,proceedings, demands, costs, expenses, damages, and other liabilities whatsoever orhowsoever caused arising directly or indirectly in connection with, in relation to or arisingout of the use of the Content.

This article may be used for research, teaching, and private study purposes. Anysubstantial or systematic reproduction, redistribution, reselling, loan, sub-licensing,systematic supply, or distribution in any form to anyone is expressly forbidden. Terms &Conditions of access and use can be found at http://www.tandfonline.com/page/terms-and-conditions

A Study of Librarians’ Perceptions andUse of the Summon Discovery Tool

GINNY M. BOYER and MEGAN BESAWEast Carolina University, Greenville, North Carolina, USA

Next-generation discovery tools such as Serials Solutions Summonseek to provide a more robust and unified search experience forlibrary patrons. Previous disappointment with federated searchhas tarnished librarians’ perceptions of new improved tools suchas Summon. This study surveys East Carolina University librarians,assessing their use of the Summon discovery tool and their percep-tions of its functionality and content. The results of the studyindicate that librarians are using Summon and identify its worth,but that focused training initiatives would bolster knowledge of thetool and increase confidence when using it in the reference andinstruction environment.

KEYWORDS discovery tool, federated search, retrieval, search,summon, usability

INTRODUCTION

Discovery of library resources is a primary concern for many libraries. Poorrelevancy of traditional library catalogs, multiple silos of information access-ible primarily through their native interface, and a changing searcher popu-lation with higher expectations for information retrieval have all resulted inthe need for a better, smarter search. Due to the proliferation of online libraryresources presenting such a challenge for users trying to find the right tools,vendors and librarians have been working for years to improve and stream-line access to library resources.1 Acquiring new discovery tools and interfacesseeking to mimic a Google-esque search is an attempt at doing just that.

This article chronicles a usability study of librarians’ perceptions and useof the Summon discovery tool at East Carolina University (ECU). The study

Received April 24, 2012; revised June 11, 2012; accepted June 15, 2012.Address correspondence to Ginny M. Boyer, Collection Management and Metadata, East

Carolina University, 600 Moye Boulevard, Greenville, NC 27834. E-mail: boyerg@ecu.edu

Journal of Electronic Resources in Medical Libraries, 9(3):173–183, 2012

Copyright # Taylor & Francis Group, LLC

ISSN: 1542-4065 print=1542-4073 online

DOI: 10.1080/15424065.2012.707056

173

Dow

nloa

ded

by [

Uni

vers

ity o

f Y

ork]

at 0

3:26

01

Oct

ober

201

3

aimed to determine how librarians were using the tool, their perceptions ofits content and functionality, their success in using the tool to conduct a bat-tery of searches, and how these factors impact use and promotion of the toolas a valuable engine for resource discovery. The libraries’ investment inSummon was an attempt to improve the overall search experience after aless-than-impressive period of experimentation with federated search.

The ECU Libraries acquired Summon after previously trialing variousfederated search options. Certain concerns prevailed which were indicativeof other institution’s experience with federated search. Some of theseincluded fears that students would lack individual database searching skillswhen relying on a Google-like search, and too many results would equateto information overload.2 These concerns however, were mediated by thebenefits realized in the potential to deliver a search that could pull togethervarious sources of information resulting in more unified and comprehensiveresults. Unfortunately, federated search fell quite below the bar manifestingsignificant issues such as slow retrieval times, impaired ranking of searchresults, and the inability to search all library content.3

Next generation discovery products such as Serials Solutions’ Summonsought to improve on the issues evident with federated search while main-taining the underlying principle of providing a single search for access toall library content. Unlike federated search, which broadcast searches outto a number of targets, Summon searches across a centralized index of pre-harvested content, improving speed of the search and allowing for most of alibrary’s resources to be aggregated in one central location.4 The promise ofthis type of discovery is that ‘‘library content of all types can be retrieved in asingle search, and users can seamlessly move from their search to full-textcontent.’’5

This seamless connectivity available within a single search interfaceposes major benefits for library users in terms of discovery and use of thelibrary’s resources. While the majority of library users will experiment withthe tool on their own, use and understanding of the tool’s functionality is alsopredicated on perceptions and use of the tool by librarians, particularly thoseteaching library instruction.

LITERATURE REVIEW

Next-generation discovery products such as Summon are a fairly newinnovation in the library world. Most of the literature on the topic focuseson discussions of the various products, implementations, and experiencespostimplementation. Overall, Summon comes across quite favorably in thesereviews. An interview with Jane Burke, vice president of Serials Solutions,reveals that the primary purpose for Summon is to eliminate ‘‘the mostfundamental barrier between libraries and users: the lack of a compelling

174 G. M. Boyer and M. Besaw

Dow

nloa

ded

by [

Uni

vers

ity o

f Y

ork]

at 0

3:26

01

Oct

ober

201

3

starting point for searching the library.’’6 In other words, the idea behindSummon is to provide a simple and intuitive interface for searching all of alibrary’s holdings. In Rowe’s review of Summon, she describes Summon as‘‘how most Google users expect a discovery tool to work.’’7 She identifiessome shortcomings with the product, including the lack of a controlledvocabulary, some thin metadata, and the inability to make personal custo-mizations, though overall these are minor concerns compared with theperformance of the tool. Similarly, Rapp reports on Dartmouth College’sexperience with Summon, citing user evaluations that characterize Summonas being ‘‘fast, simple and familiar’’ and as providing a search that can‘‘traverse masses of disparate resources and return relevant results in anattractive, simple presentation.’’8

Libraries are implementing tools such as Summon precisely becauseusers expect more from their search experience than traditional library searchinterfaces have to offer. Users want fast performance, relevant results, and anintuitive search interface. Lauridsen and Law, who are both employed bySerials Solutions, report on users’ frustrations with the traditional librarysearch.3 They indicate that users prefer a search that functions like Googlebut that includes relevant library results and full-text content. Summon seeksto achieve these lofty goals, though the question remains if implementationof such a tool results in greater access to a library’s resources. Doug Wayreports on Grand Valley State University’s successful implementation ofSummon.9 His analysis of usage statistics indicates that, while individual data-base use declined, full-text downloads of content increased. This suggeststhat Summon was more successful in linking up users with the full-textcontent they were seeking. Further, Rapp reports that Dartmouth Collegelibrarians spoke of Summon’s ability to ‘‘open that window across disci-plines’’ and ‘‘bring those disciplines into conversation with each other in away that did not happen in the past.’’8 This aggregation of resources exem-plifies the interdisciplinary searching capabilities that Summon provides.

With sparse literature covering the adoption and promotion of thesenext-generation discovery tools, it is helpful to look at the experiences oflibrarians with federated searching tools. Lampert and Dabbour’s study illu-minates librarians’ perceptions and use of these tools.2 Of the populationssurveyed, a majority of respondents answered that they did not offer feder-ated search instruction to their users. Among the reasons cited for not teach-ing users to use federated searching were the tools’ poor precision and recall,lack of indexing for all databases, and absence of specialized features=facetsfor limiting results available in native interfaces. Those that responded thatthey were not confident in teaching federated searching tools cited reasonsincluding a lack of experience with the tool and a distrust of search results.Buck and Mellenger sum up the significance of Lampert and Dabbour’sreview quite nicely when they state that this review ‘‘clearly shows thatlibrarian perceptions, assumptions, and willingness to adjust teaching

A Study of Librarians’ Perceptions and Use 175

Dow

nloa

ded

by [

Uni

vers

ity o

f Y

ork]

at 0

3:26

01

Oct

ober

201

3

practices are important aspects in the adoption of new tools.’’5 A review byLaBelle supports this claim, indicating strong correlations between librarians’willingness to promote use of a tool and its adoption in the search repertoireof the information seeker.10 He concludes his article by stating that ‘‘librariansare at a crossroads and need to determine how best to utilize the potentialthat this technology brings to the information landscape... the stance thatlibrarians take is dependent on a variety of factors, including whetheror not instruction is viewed as being instructor- or learner-centered.’’10

This conclusion should be considered when approaching next generationtools such as Summon to ensure that librarians’ fears and perceptionsdo not impede the benefits of promoting use of an effective discoverysolution.

There are few studies attempting to evaluate the usability of newer,next-generation search tools such as Summon or to assess librarians’ respon-ses to integrating these resources into their repertoire of instruction. Buckand Mellinger evaluate librarian perceptions regarding the impact ofSummon on information literacy instruction at Oregon State UniversityLibraries.5 They found an inherent disconnect between what librarians andstudents expect from a discovery interface. While users are simply lookingfor a fast and efficient search, librarians must feel confident in the qualityand performance of the product before viewing it is an asset to instruction.For the librarians in the study, Summon, despite its vast improvements overfederated searching, was viewed more as a supplement to library instructionthan a replacement.5 Similarly, Howard and Wiebrands found that whilestudents and faculty responded positively to Summon’s implementation atEdith Cowan University, librarians harbored misgivings about the tool.11

The authors identify training as an important element to address these mis-givings. They write that ‘‘better understanding of how such tools workmay have meant that the levels of uncertainty and doubt about the newsystem could have been alleviated somewhat.’’11 Perhaps previous dis-appointment with federated searching tarnishes the improvements and ben-efits evident in tools such as Summon. Therefore greater considerationshould be placed in garnering support for the discovery product duringimplementation.

Proven to be an effective search tool that optimizes access to thelibrary’s resources and functions in a manner desired by users, next-generation tools such as Summon enable libraries to capitalize on a searchexperience that is both Google-like and library-specific. The caveat seemsto be that libraries must invest time and consideration into gaining buy-infor these new discovery solutions to fully manifest value to their librariansand gain institutional promotion of the tool. Certain factors, includingfocused training and greater transparency regarding indexed content, mayresult in greater acceptance of such tools and incorporation of the tools intolibrary information instruction.

176 G. M. Boyer and M. Besaw

Dow

nloa

ded

by [

Uni

vers

ity o

f Y

ork]

at 0

3:26

01

Oct

ober

201

3

METHODS

After nearly a year following implementation, the authors felt it was anappropriate time to examine librarians’ perceptions and searching knowl-edge of Summon. The study was designed to be conducted in three parts:a preassessment survey, the actual assessment, and a postassessment survey.The three components were deemed necessary to ascertain participants’initial views of the tool, measure their searching ability, and determine if theirviews of the tool had changed following completion of the assessment.

The anonymous preassessment survey was sent via email to everylibrarian at East Carolina University. The electronic presurvey containedseven questions addressing their current use of Summon as well as generalviewpoints regarding the product. Questions consisted of multiple formatsincluding two short answer, two Likert Scales, and three multiple choice.All nonshort answer questions contained a component allowing participantsto elaborate on their responses. Participants were given 18 days to completethe survey.

Sixteen days following completion of the preassessment survey, an11-question assessment was created and distributed using the education soft-ware Quia. The questions on the assessment were designed to measurelibrarians’ knowledge of Summon’s components and capabilities. Specificquestions addressed features located within the advanced search option,locating specific content types, and general feedback on the discovery tool.The assessment was sent to all 50 librarians at ECU. Respondents had 18 daysto complete the assessment. Certain questions provided participants withfeedback if an incorrect answer was provided. The feedback within theassessment was designed to be instructive and help participants withquestions that might have seemed confusing.

The postassessment survey was sent to all librarians four days followingthe closure of the assessment portion of the study. This particular survey con-tained six questions and was designed to determine if and how participants’views of Summon had altered following the assessment. The survey includedfour multiple choice questions and two short answer questions. Similar to theassessment and preassessment survey, participants were allotted 18 days tocomplete the survey.

RESULTS

There were 19 responses for the preassessment survey. As Figure 1 illustrates,when asked about the frequency with which participants used Summon,36.8% reported using the tool a couple of times during the semester. A littleover 31% used Summon once a week and 26.3% used the product daily. Inaddition, 5% stated that they never used Summon.

A Study of Librarians’ Perceptions and Use 177

Dow

nloa

ded

by [

Uni

vers

ity o

f Y

ork]

at 0

3:26

01

Oct

ober

201

3

Seventy-three percent of users reported using the Advanced Searchfeatures that are available. Figure 2 shows that on a scale of 1–5, with fivebeing the highest, 47.4% of individuals felt very comfortable using limitsand facets.

FIGURE 1 Pre-assessment Survey Results.

FIGURE 2 Familiarity With Using Limits and Facets.

178 G. M. Boyer and M. Besaw

Dow

nloa

ded

by [

Uni

vers

ity o

f Y

ork]

at 0

3:26

01

Oct

ober

201

3

Multiple participants mentioned that they used Summon as an exploratorytool when just beginning their research and as a resource to locate specificitems. Only 31.6% of respondents taught Summon in their library classes.When asked why Summon was not shown, the majority of respondents statedthat it was because they did not teach library instruction in any capacity.

A large number of respondents indicated that a vast improvement toSummon would be greater success and accuracy of linking to full text. Othersfelt that the large quantity of results was overwhelming. The majority of over-all feelings toward Summon were neutral, registering a 3 on a scale from 1 to5 (with 5 being the most positive).

There were 14 respondents to the Summon assessment. In total, the cor-rect answer percentages for each individual question ranged from 0% to100%. The overall average of the group was 67% for all seven questions.All 14 respondents answered one question incorrectly as well as answeringone question correctly. The question that all participants were unable toanswer concerned the addition of Hathi Trust full text indexing within Sum-mon. The majority of participants were able to locate specific items and limitsearches when requested to do so.

When provided with the option to offer their thoughts on the assess-ment in general, participants commented:

‘‘I learned several new things from this assignment. I like that it searcheswithin the text of an item. Also, I didn’t know there was a citationmatcher; I don’t know if I’d use it though since I have other preferences,but still good to know it’s there.’’

‘‘I think One Search (Summon) is fabulous, though it can be buggy attimes. It doesn’t take that long to learn how to use it. Also, it seems like itcan be very precise once you know how to limit and refine searches,specifically after learning the ‘include’ and ‘exclude’ functions.’’

‘‘Very helpful for interdisciplinary situations, and for situations whereyou are not quite sure where to begin (there are many of theseinstances) . . . ’’

There were 12 responses to the request to complete the postassessmentsurvey. Figure 3 illustrates that after completing the assessment 50% of part-icipants felt they had a better understanding of Summon and thought thattheir use of Summon would increase.

Following completion of the assessment, individuals commented onchanges that they would to like to see including confusion with the limitationoptions, clarification on what is included within Summon, and full-text link-ing. Participants were least aware of the advanced search features and allavailable facets. As Figure 4 illustrates, all of the survey respondents thoughtthat Summon was beneficial to our users and that the discovery tool makesthe library resources more discoverable and accessible.

A Study of Librarians’ Perceptions and Use 179

Dow

nloa

ded

by [

Uni

vers

ity o

f Y

ork]

at 0

3:26

01

Oct

ober

201

3

DISCUSSION

The surveys and assessment indicate that ECU’s librarians use the tool quitefrequently and, overall, feel comfortable with its functionality. Summonwas primarily utilized as a place to start research or as a last resort whenthe databases were not producing desired results. Although the majorityreported not using Summon in library instruction, this could possibly be

FIGURE 4 Respondents Felt Summon is Beneficial to Users.

FIGURE 3 One Search Post-Assessment.

180 G. M. Boyer and M. Besaw

Dow

nloa

ded

by [

Uni

vers

ity o

f Y

ork]

at 0

3:26

01

Oct

ober

201

3

because many of the respondents do not serve in teaching positions. A betterassessment of this number could be obtained by targeting future studiesspecifically to Reference and Information Services librarians.

One glaring issue that arose from the study regarded the lack of under-standing of full-text indexing of content provided by the Hathi Trust Foun-dation. This confusion could be remedied by basic instruction on the HathiTrust collection, its inclusion in the Summon index, and how to identifywhen Hathi Trust content is queried in results. One of the biggest improve-ments identified was regarding accuracy for full-text linking. With the releaseof Summon’s new Indexed-Enhanced Direct Linking in September 2011,which provides direct linking to the full text without the intermediary linkresolver, this issue may resolve itself over time.4

Although the majority of librarians reported an overall feeling ofefficiency with the tool, the postassessment revealed that nearly half of part-icipants benefited from the assessment exercises and feedback provided.Those completing the assessment felt that the questions that providedfeedback helped them to learn new aspects of the tool. One participant com-mented, ‘‘After the ‘‘feedback’’ from #7, I was able to do the search only inOne Search (Summon). I got better at One Search as I did this assessment.’’Participants completing the assessment also indicated that their use ofSummon will increase since they feel they have gained new knowledgeabout the resource.

There are limitations with this study. One limitation is that participationwithin the three components of the study declined by small increments witheach assessment or survey that was introduced. Since the assessment wasanonymous, it was not possible to track and see if the same individuals com-pleted all three portions or which portions were taken by certain participants.An additional incentive for identifying study participants would be to gaugethe differences regarding use of the tool by the two separate librarian popula-tions—those serving the main, undergraduate library and those serving thehealth sciences library. Finally, though this study targeted usability by librar-ians to determine efficiency of serving searcher populations, future studiesregarding usability and preferences of searchers would help to guide trainingand instruction initiatives.

CONCLUSION

It is clear that a tool like Summon is necessary and beneficial to library users,but libraries must take care to position such tools to be a successful additionto the resource discovery dynamic. Buck and Mellanger say it quite plainlywith their concluding statement that ‘‘librarians want to teach tools that work,which they understand, and that will help students gain valuable informationliteracy skills. Users want tools that are easy and fast.’’5 The results of this

A Study of Librarians’ Perceptions and Use 181

Dow

nloa

ded

by [

Uni

vers

ity o

f Y

ork]

at 0

3:26

01

Oct

ober

201

3

study indicate that Summon may meet that happy medium if measures aretaken to fully communicate its scope and functionality to library profes-sionals. With Serials Solutions practice of rolling out new releases of Summonevery three weeks detailing interface and content enhancements, it could bebeneficial to hold regular information sessions for librarians to fully report onchanges in functionality and content.

It was encouraging to see that all respondents identified the value of adiscovery tool such as Summon and felt that it was an asset to student’s infor-mation literacy. Though further studies are necessary to obtain a more granu-lar view of usability on behalf of certain searcher populations, both librarianand patron, it is safe to say that this study identified the inherent value ofSummon and its effective use as an information retrieval engine for theECU Libraries.

ABOUT THE AUTHORS

Ginny Boyer, MILS (boyerg@ecu.edu) is Head of Collection Management andMetadata, and Megan E. Besaw, MLIS (besawm@ecu.edu) is Liaison to theCollege of Allied Health Sciences at Laupus Health Sciences Library, EastCarolina University, 600 Moye Boulevard, Greenville, NC 27834.

REFERENCES

1. Eliasen, K.; McKinstry, J.; Fraser, B.M.; and Babbitt, E.P. ‘‘Navigating OnlineMenus: A Quantitative Experiment.’’ College & Research Libraries 58, no. 6(1997): 509–16.

2. Lampert, L.D., and Dabbour, K.S. ‘‘Librarian Perspectives on TeachingMetasearch and Federated Search Technologies.’’ Internet Reference ServicesQuarterly 12, no. 3 (2007): 253–78.

3. Lauridsen, H., and Law, J. ‘‘How Do You Follow Google? Providing a High Qual-ity Library Search Experience.’’ Conference Proceedings of the InternationalAssociation of Scientific and Technological University Libraries (2009). Retrievedfrom <http://iatul.org/doclibrary/public/Conf_Proceedings/2009/Lauridsen-text.pdf>. Accessed: March 15, 2011.

4. Serials Solutions. ‘‘The Summon Service.’’<http://www.serialssolutions.com/en/services/summon/>. Accessed: March 15, 2011.

5. Buck, S., and Mellinger, M. ‘‘The Impact of Serials Solutions’ SummonTM onInformation Literacy Instruction: Librarian Perceptions.’’ Internet ReferenceServices Quarterly 16, no. 4 (2011): 159–81.

6. Brunning, D., and Machovec, G. ‘‘Interview About Summon with Jane Burke,Vice President of Serials Solutions.’’ The Charleston Advisor 11, no. 4 (2010):60–62.

7. Rowe, R. ‘‘Web-Scale Discovery: A Review of Summon, EBSCO DiscoveryService, and WorldCat Local.’’ The Charleston Advisor 12, no. 1 (2010): 5–10.

182 G. M. Boyer and M. Besaw

Dow

nloa

ded

by [

Uni

vers

ity o

f Y

ork]

at 0

3:26

01

Oct

ober

201

3

8. Rapp, D. ‘‘DISCOVERY AT DARTMOUTH.’’ Library Journal 137, no. 3 (2012):36–39.

9. Way, D. ‘‘The Impact of Web-Scale Discovery on the Use of a LibraryCollection.’’ Serials Review 36, no. 4 (2010): 214–20.

10. Labelle, P.R. ‘‘Initiating the Learning Process.’’ Internet Reference ServicesQuarterly 12, no. 3–4 (2007): 237–52.

11. Howard, D., andWiebrands, C. ‘‘Culture Shock: Librarians’ Response toWeb ScaleSearch.’’ Conference Proceedings of the ALIA Information Online Conference(2011). Retrieved from <http://ro.ecu.edu.au/cgi/viewcontent.cgi?article=7208&context=ecuworks>. Accessed: March 15, 2011.

A Study of Librarians’ Perceptions and Use 183

Dow

nloa

ded

by [

Uni

vers

ity o

f Y

ork]

at 0

3:26

01

Oct

ober

201

3