discovery impact erl2014
Post on 21-Oct-2014
512 views
DESCRIPTION
Michael Levine-Clark, John McDonald, Jason PriceTRANSCRIPT
![Page 1: Discovery impact erl2014](https://reader033.vdocuments.us/reader033/viewer/2022061106/54463486b1af9fd33a8b459e/html5/thumbnails/1.jpg)
Not always discovered: Phase two of a study of the effect of discovery systems on online (journal) usage
ER&LMarch 18, 2014
Michael Levine-Clark, University of DenverJohn McDonald, University of Southern California
Jason Price, SCELC Consortium
http://bit.ly/discovery-impact-erl2014
![Page 2: Discovery impact erl2014](https://reader033.vdocuments.us/reader033/viewer/2022061106/54463486b1af9fd33a8b459e/html5/thumbnails/2.jpg)
“…a steep increase in full text downloads and link resolver click‐throughs suggests Summon had a dramatic impact on user behavior and the use of library collections during this time period.”
The Impact of Web-scale Discovery on the Use of a Library CollectionDoug Way (2010) http://scholarworks.gvsu.edu/library_sp/9/
![Page 3: Discovery impact erl2014](https://reader033.vdocuments.us/reader033/viewer/2022061106/54463486b1af9fd33a8b459e/html5/thumbnails/3.jpg)
http://www.oclc.org/partnerships/econtent/solutions.en.html
Vendor marketing
![Page 4: Discovery impact erl2014](https://reader033.vdocuments.us/reader033/viewer/2022061106/54463486b1af9fd33a8b459e/html5/thumbnails/4.jpg)
Does implementation of a discovery service impact usage of publisher-
hosted journal content?
![Page 5: Discovery impact erl2014](https://reader033.vdocuments.us/reader033/viewer/2022061106/54463486b1af9fd33a8b459e/html5/thumbnails/5.jpg)
Publisher-hosted journals are only part of the picture
eBooks, pBooks, aggregator journal content, etc.publisher journal contentThe six publishers in this study
![Page 6: Discovery impact erl2014](https://reader033.vdocuments.us/reader033/viewer/2022061106/54463486b1af9fd33a8b459e/html5/thumbnails/6.jpg)
What did we measure?
• Whether there is an effect
• NOT why that effect exists (that’s a future study!)
![Page 7: Discovery impact erl2014](https://reader033.vdocuments.us/reader033/viewer/2022061106/54463486b1af9fd33a8b459e/html5/thumbnails/7.jpg)
• “Society will need to shed some of its obsession for causality in exchange for simple correlations: not knowing why, but only what”
• (Cukier & Mayer-Schonberger. 2013. Big data: A revolution that will
transform how we live, work, and think.)
![Page 8: Discovery impact erl2014](https://reader033.vdocuments.us/reader033/viewer/2022061106/54463486b1af9fd33a8b459e/html5/thumbnails/8.jpg)
![Page 9: Discovery impact erl2014](https://reader033.vdocuments.us/reader033/viewer/2022061106/54463486b1af9fd33a8b459e/html5/thumbnails/9.jpg)
![Page 10: Discovery impact erl2014](https://reader033.vdocuments.us/reader033/viewer/2022061106/54463486b1af9fd33a8b459e/html5/thumbnails/10.jpg)
Data collection• List of libraries with discovery services
>Searched on lib-web-cats
• Surveyed Libraries>Discovery service Implemented>Implementation Date (month/year)>Search box location>Marketing effort
• 149 Libraries Gave Approval>33 libraries selected for this phase>6 for each of the 4 major discovery services and a
group of 9 libraries with no service
![Page 11: Discovery impact erl2014](https://reader033.vdocuments.us/reader033/viewer/2022061106/54463486b1af9fd33a8b459e/html5/thumbnails/11.jpg)
Dataset• 33 Libraries
– 28 US, 2 CA, 1 each from UK, AUS, NZ
–WorldCat book holdings>Average: 1,114,193 ; Range: ~300k to ~2.6mil
• Implementation dates (Discovery Libraries): >2010 (3), 2011 (19), 2012 (2)
• 6 Publishers
• 9,206 Journals
• 163,545 Usable Observations
![Page 12: Discovery impact erl2014](https://reader033.vdocuments.us/reader033/viewer/2022061106/54463486b1af9fd33a8b459e/html5/thumbnails/12.jpg)
Methodology
Compared COUNTER JR1 total full text article views for the
12 months before vs 12 months after implementation date
June
201
0St
art
Impl
emen
tatio
nM
ay 2
011
May
201
2En
d
Year 1 Year 2
Included implementation month in Year 1 to ensure that both periods included an entire academic year
![Page 13: Discovery impact erl2014](https://reader033.vdocuments.us/reader033/viewer/2022061106/54463486b1af9fd33a8b459e/html5/thumbnails/13.jpg)
Examine Data for Outliers
![Page 14: Discovery impact erl2014](https://reader033.vdocuments.us/reader033/viewer/2022061106/54463486b1af9fd33a8b459e/html5/thumbnails/14.jpg)
Observations by Library & Service
![Page 15: Discovery impact erl2014](https://reader033.vdocuments.us/reader033/viewer/2022061106/54463486b1af9fd33a8b459e/html5/thumbnails/15.jpg)
Observations by Publisher
![Page 16: Discovery impact erl2014](https://reader033.vdocuments.us/reader033/viewer/2022061106/54463486b1af9fd33a8b459e/html5/thumbnails/16.jpg)
![Page 17: Discovery impact erl2014](https://reader033.vdocuments.us/reader033/viewer/2022061106/54463486b1af9fd33a8b459e/html5/thumbnails/17.jpg)
Average Usage Change By Discovery & Publisher
![Page 18: Discovery impact erl2014](https://reader033.vdocuments.us/reader033/viewer/2022061106/54463486b1af9fd33a8b459e/html5/thumbnails/18.jpg)
Analyzing Usage Change: % vs Total
Use 12 months before
Use 12 months
after% Change Total
Change
Journal A 500 600 20% 100
Journal B 5 15 200% 10
Which is the better measure?
Is it the same for publisher- & journal-level data?
![Page 19: Discovery impact erl2014](https://reader033.vdocuments.us/reader033/viewer/2022061106/54463486b1af9fd33a8b459e/html5/thumbnails/19.jpg)
Reducing variation due to institution sizeCurrently converting to change per FTE
Values are shown as x 1,000 to bring the change metric back per journal-library combination to a minimum of 0.1
2013 JISC Discovery study took a similar approach
![Page 20: Discovery impact erl2014](https://reader033.vdocuments.us/reader033/viewer/2022061106/54463486b1af9fd33a8b459e/html5/thumbnails/20.jpg)
Average Usage Change By Discovery & Publisher
Per Journal & Per 10,000 FTE
![Page 21: Discovery impact erl2014](https://reader033.vdocuments.us/reader033/viewer/2022061106/54463486b1af9fd33a8b459e/html5/thumbnails/21.jpg)
![Page 22: Discovery impact erl2014](https://reader033.vdocuments.us/reader033/viewer/2022061106/54463486b1af9fd33a8b459e/html5/thumbnails/22.jpg)
Full Model
Including Discovery Service, Publisher, and Library
Including Discovery Service, Publisher, and Library
![Page 23: Discovery impact erl2014](https://reader033.vdocuments.us/reader033/viewer/2022061106/54463486b1af9fd33a8b459e/html5/thumbnails/23.jpg)
Nested ANOVA Model
[all three factors – preliminary results]
![Page 24: Discovery impact erl2014](https://reader033.vdocuments.us/reader033/viewer/2022061106/54463486b1af9fd33a8b459e/html5/thumbnails/24.jpg)
Does usage change vary across libraries?
Institution (sorted by Mean Change)
![Page 25: Discovery impact erl2014](https://reader033.vdocuments.us/reader033/viewer/2022061106/54463486b1af9fd33a8b459e/html5/thumbnails/25.jpg)
Does usage change vary across libraries using the same service?
Library 10-15 Library 16-21 Library 22-27 Library 28-33Library 1-9
![Page 26: Discovery impact erl2014](https://reader033.vdocuments.us/reader033/viewer/2022061106/54463486b1af9fd33a8b459e/html5/thumbnails/26.jpg)
Does usage change vary across publishers?
Publisher (sorted by Mean Change)
B
AAA A A
![Page 27: Discovery impact erl2014](https://reader033.vdocuments.us/reader033/viewer/2022061106/54463486b1af9fd33a8b459e/html5/thumbnails/27.jpg)
Does usage change vary across discovery services?
A
BB
C
D
![Page 28: Discovery impact erl2014](https://reader033.vdocuments.us/reader033/viewer/2022061106/54463486b1af9fd33a8b459e/html5/thumbnails/28.jpg)
Publisher
Does the effect of discovery service differ across publishers?
![Page 29: Discovery impact erl2014](https://reader033.vdocuments.us/reader033/viewer/2022061106/54463486b1af9fd33a8b459e/html5/thumbnails/29.jpg)
ResultsCan we detect differences between Discovery Services, Publishers, and/or Libraries and/or their interactions? • Library – Yes• Publisher – No• Discovery Service – Yes
• Differential discovery service effect by publisher – Yes
![Page 30: Discovery impact erl2014](https://reader033.vdocuments.us/reader033/viewer/2022061106/54463486b1af9fd33a8b459e/html5/thumbnails/30.jpg)
Next Steps• Design & test for effects of:
–Aggregator full text availability–Publisher Size–Journal Subject–Overall usage trends (Requires Disc Srvc ‘control’)–Configuration options in Discovery services
• Expand pool of libraries• Perhaps explore WHY
![Page 31: Discovery impact erl2014](https://reader033.vdocuments.us/reader033/viewer/2022061106/54463486b1af9fd33a8b459e/html5/thumbnails/31.jpg)
Sharing Data• With participating libraries
–Customized reports for each library• With participating publishers
–Customized reports for each publisher–Presentations as requested
• With discovery vendors–Presentations as requested
• In publications and presentations–Maintaining anonymity of data
![Page 32: Discovery impact erl2014](https://reader033.vdocuments.us/reader033/viewer/2022061106/54463486b1af9fd33a8b459e/html5/thumbnails/32.jpg)
Past/Future Presentations• Ithaka Sustainable Scholarship Conference (October 2013)• Charleston Conference (November 2013)• ER&L/Library Journal Webinar (December 2013)• Shangai Jiao Tong Univ / Beijing Univ Forum (Jan 2014)• SCELC Colloquium (March 2014)• ER&L (March 2014) http://bit.ly/discovery-impact-erl2014 • UKSG (April 2014)• Presentations posted on slideshare :
–http://visualcv.com/lpq4t1s