ramp: repository analytics and metrics portal fileagenda 1. a new reporting model 2. accuracy of...
TRANSCRIPT
![Page 1: RAMP: Repository Analytics and Metrics Portal fileAgenda 1. A New Reporting Model 2. Accuracy of Analytics Reporting Tools Page Tagging Log file 3. RAMP New Prototype Web Service for](https://reader030.vdocuments.us/reader030/viewer/2022040417/5d5672fd88c993df7b8bb022/html5/thumbnails/1.jpg)
Kenning Arlitsch, Dean @kenning_msu
Patrick OBrien, Semantic Web Research Director @sempob
RAMP:
Repository Analytics and Metrics Portal
Accurately measuring use of institutional repositories
ARL Assessment Forum
Atlanta, GA, January 20, 2017
![Page 2: RAMP: Repository Analytics and Metrics Portal fileAgenda 1. A New Reporting Model 2. Accuracy of Analytics Reporting Tools Page Tagging Log file 3. RAMP New Prototype Web Service for](https://reader030.vdocuments.us/reader030/viewer/2022040417/5d5672fd88c993df7b8bb022/html5/thumbnails/2.jpg)
Agenda
1. A New Reporting Model
2. Accuracy of Analytics Reporting Tools
Page Tagging
Log file
3. RAMP
New Prototype Web Service for IR reporting
![Page 3: RAMP: Repository Analytics and Metrics Portal fileAgenda 1. A New Reporting Model 2. Accuracy of Analytics Reporting Tools Page Tagging Log file 3. RAMP New Prototype Web Service for](https://reader030.vdocuments.us/reader030/viewer/2022040417/5d5672fd88c993df7b8bb022/html5/thumbnails/3.jpg)
A New Reporting Model
Page Type Definition Examples
Citable Content
Downloads
Non-HTML scholarly content
that may be formally cited in
the research process
● Publication (.pdf)
● Presentation (.ppt)
● Data Sets (.csv)
Item Summary HTML pages to help user
decide to download the full
publication
● Title & Abstract
● Item Metadata
Ancillary HTML pages that provide
general information or
navigation
● Search Results
● Browse by Author
● Statistics
![Page 4: RAMP: Repository Analytics and Metrics Portal fileAgenda 1. A New Reporting Model 2. Accuracy of Analytics Reporting Tools Page Tagging Log file 3. RAMP New Prototype Web Service for](https://reader030.vdocuments.us/reader030/viewer/2022040417/5d5672fd88c993df7b8bb022/html5/thumbnails/4.jpg)
![Page 5: RAMP: Repository Analytics and Metrics Portal fileAgenda 1. A New Reporting Model 2. Accuracy of Analytics Reporting Tools Page Tagging Log file 3. RAMP New Prototype Web Service for](https://reader030.vdocuments.us/reader030/viewer/2022040417/5d5672fd88c993df7b8bb022/html5/thumbnails/5.jpg)
![Page 6: RAMP: Repository Analytics and Metrics Portal fileAgenda 1. A New Reporting Model 2. Accuracy of Analytics Reporting Tools Page Tagging Log file 3. RAMP New Prototype Web Service for](https://reader030.vdocuments.us/reader030/viewer/2022040417/5d5672fd88c993df7b8bb022/html5/thumbnails/6.jpg)
![Page 7: RAMP: Repository Analytics and Metrics Portal fileAgenda 1. A New Reporting Model 2. Accuracy of Analytics Reporting Tools Page Tagging Log file 3. RAMP New Prototype Web Service for](https://reader030.vdocuments.us/reader030/viewer/2022040417/5d5672fd88c993df7b8bb022/html5/thumbnails/7.jpg)
![Page 8: RAMP: Repository Analytics and Metrics Portal fileAgenda 1. A New Reporting Model 2. Accuracy of Analytics Reporting Tools Page Tagging Log file 3. RAMP New Prototype Web Service for](https://reader030.vdocuments.us/reader030/viewer/2022040417/5d5672fd88c993df7b8bb022/html5/thumbnails/8.jpg)
Accuracy of Analytics Reporting Tools
![Page 9: RAMP: Repository Analytics and Metrics Portal fileAgenda 1. A New Reporting Model 2. Accuracy of Analytics Reporting Tools Page Tagging Log file 3. RAMP New Prototype Web Service for](https://reader030.vdocuments.us/reader030/viewer/2022040417/5d5672fd88c993df7b8bb022/html5/thumbnails/9.jpg)
Two Classes of Web Analytics
HTML
Analytics Service (SaaS) 1
Log Files 2 Page Tagging
{JavaScript}
![Page 10: RAMP: Repository Analytics and Metrics Portal fileAgenda 1. A New Reporting Model 2. Accuracy of Analytics Reporting Tools Page Tagging Log file 3. RAMP New Prototype Web Service for](https://reader030.vdocuments.us/reader030/viewer/2022040417/5d5672fd88c993df7b8bb022/html5/thumbnails/10.jpg)
Page Tagging
Page Tagging does not track non-HTML CCD
HTML
Non-HTML
![Page 11: RAMP: Repository Analytics and Metrics Portal fileAgenda 1. A New Reporting Model 2. Accuracy of Analytics Reporting Tools Page Tagging Log file 3. RAMP New Prototype Web Service for](https://reader030.vdocuments.us/reader030/viewer/2022040417/5d5672fd88c993df7b8bb022/html5/thumbnails/11.jpg)
Show Google Scholar direct link
graphic
![Page 12: RAMP: Repository Analytics and Metrics Portal fileAgenda 1. A New Reporting Model 2. Accuracy of Analytics Reporting Tools Page Tagging Log file 3. RAMP New Prototype Web Service for](https://reader030.vdocuments.us/reader030/viewer/2022040417/5d5672fd88c993df7b8bb022/html5/thumbnails/12.jpg)
Google Analytics use in academic libraries
Tested 279 academic library websites
ARL
DLF
OCLC-RLP
90% US libraries contained Google tracking code
![Page 13: RAMP: Repository Analytics and Metrics Portal fileAgenda 1. A New Reporting Model 2. Accuracy of Analytics Reporting Tools Page Tagging Log file 3. RAMP New Prototype Web Service for](https://reader030.vdocuments.us/reader030/viewer/2022040417/5d5672fd88c993df7b8bb022/html5/thumbnails/13.jpg)
51%
54%
52%
63%
35%
27%
20%
20%
14%
19%
28%
17%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
~51%to63%ofCitableIRActivityisUnreportedbyGoogleAnalytics01/05/16- 05/17/16(days=134)
CitableContentDownloads ItemSumaryPV AncillaryPV
Most IR activity is Citable Content Downloads
![Page 14: RAMP: Repository Analytics and Metrics Portal fileAgenda 1. A New Reporting Model 2. Accuracy of Analytics Reporting Tools Page Tagging Log file 3. RAMP New Prototype Web Service for](https://reader030.vdocuments.us/reader030/viewer/2022040417/5d5672fd88c993df7b8bb022/html5/thumbnails/14.jpg)
54.3%
46.2%
47.9%
42.5%
45.7%
53.8%
52.1%
57.5%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
MostIRActivityisUnreportedbyGoogleAnalytics01/05/16- 05/17/16(days=134)
Reported Invisible
Most IR Activity Unreported by Google Analytics
![Page 15: RAMP: Repository Analytics and Metrics Portal fileAgenda 1. A New Reporting Model 2. Accuracy of Analytics Reporting Tools Page Tagging Log file 3. RAMP New Prototype Web Service for](https://reader030.vdocuments.us/reader030/viewer/2022040417/5d5672fd88c993df7b8bb022/html5/thumbnails/15.jpg)
Web Analytics Accuracy Risks
Risks Analytics Method
Area Types Page Tagging Log Files
OverCount
Visits Low High
Downloads Low High
Page Views Low Low
UnderCount
Visits Medium Medium
Downloads High Low
Page Views Low Low
![Page 16: RAMP: Repository Analytics and Metrics Portal fileAgenda 1. A New Reporting Model 2. Accuracy of Analytics Reporting Tools Page Tagging Log file 3. RAMP New Prototype Web Service for](https://reader030.vdocuments.us/reader030/viewer/2022040417/5d5672fd88c993df7b8bb022/html5/thumbnails/16.jpg)
Data set: Jan 5 - May 17, 2016 (n = 134 days)
Study Participant IR Platform URL
Montana State University ScholarWorks DSpace scholarworks.montana.edu
McMaster University MacSphere DSpace macsphere.mcmaster.ca
University of New Mexico LoboVault DSpace repository.unm.edu
University of Utah USpace CONTENTdm uspace.utah.edu
![Page 17: RAMP: Repository Analytics and Metrics Portal fileAgenda 1. A New Reporting Model 2. Accuracy of Analytics Reporting Tools Page Tagging Log file 3. RAMP New Prototype Web Service for](https://reader030.vdocuments.us/reader030/viewer/2022040417/5d5672fd88c993df7b8bb022/html5/thumbnails/17.jpg)
Page Tagging does not track non-HTML
Citable Content Downloads
Non-HTML
Search Analytics
Does!
![Page 18: RAMP: Repository Analytics and Metrics Portal fileAgenda 1. A New Reporting Model 2. Accuracy of Analytics Reporting Tools Page Tagging Log file 3. RAMP New Prototype Web Service for](https://reader030.vdocuments.us/reader030/viewer/2022040417/5d5672fd88c993df7b8bb022/html5/thumbnails/18.jpg)
Typical Page Tagging (GA) Data Types
Page Type Analytics
Pages Events
Citable Content
Downloads - ✅
Item Summary ✅ -
Ancillary ✅ -
![Page 19: RAMP: Repository Analytics and Metrics Portal fileAgenda 1. A New Reporting Model 2. Accuracy of Analytics Reporting Tools Page Tagging Log file 3. RAMP New Prototype Web Service for](https://reader030.vdocuments.us/reader030/viewer/2022040417/5d5672fd88c993df7b8bb022/html5/thumbnails/19.jpg)
Montana Method Includes Google Search Console
Page Type Analytics
Search
Console
Pages Events Search Analytics
Citable Content
Downloads - ✅ ✅
Item Summary ✅ - -
Ancillary ✅ - -
![Page 20: RAMP: Repository Analytics and Metrics Portal fileAgenda 1. A New Reporting Model 2. Accuracy of Analytics Reporting Tools Page Tagging Log file 3. RAMP New Prototype Web Service for](https://reader030.vdocuments.us/reader030/viewer/2022040417/5d5672fd88c993df7b8bb022/html5/thumbnails/20.jpg)
GA Ancillary Page Views
and Item Summary Page Views vs CCD
![Page 21: RAMP: Repository Analytics and Metrics Portal fileAgenda 1. A New Reporting Model 2. Accuracy of Analytics Reporting Tools Page Tagging Log file 3. RAMP New Prototype Web Service for](https://reader030.vdocuments.us/reader030/viewer/2022040417/5d5672fd88c993df7b8bb022/html5/thumbnails/21.jpg)
All four IR using Google Analytics
Page Type Analytics
Search
Console
Pages Events Search Analytics
Citable Content
Downloads - 26,355 562,933
Item Summary 284,303 - -
Ancillary 201,793 - -
CCD Tracking Improvement
2,000%
![Page 22: RAMP: Repository Analytics and Metrics Portal fileAgenda 1. A New Reporting Model 2. Accuracy of Analytics Reporting Tools Page Tagging Log file 3. RAMP New Prototype Web Service for](https://reader030.vdocuments.us/reader030/viewer/2022040417/5d5672fd88c993df7b8bb022/html5/thumbnails/22.jpg)
Montana Method Challenges
❖Missing non-Google direct link CCD events ● Yahoo
● Bing
● FB
❖ GSC limits time and access ● Moving 90-day window
● Granular data = programing skills to access API
![Page 23: RAMP: Repository Analytics and Metrics Portal fileAgenda 1. A New Reporting Model 2. Accuracy of Analytics Reporting Tools Page Tagging Log file 3. RAMP New Prototype Web Service for](https://reader030.vdocuments.us/reader030/viewer/2022040417/5d5672fd88c993df7b8bb022/html5/thumbnails/23.jpg)
RAMP: Repository Analytics & Metrics Portal
![Page 24: RAMP: Repository Analytics and Metrics Portal fileAgenda 1. A New Reporting Model 2. Accuracy of Analytics Reporting Tools Page Tagging Log file 3. RAMP New Prototype Web Service for](https://reader030.vdocuments.us/reader030/viewer/2022040417/5d5672fd88c993df7b8bb022/html5/thumbnails/24.jpg)
RAMP - Repository Analytics and Metrics Portal
Cloud Web service
Currently accessible and free
During grant period (through November 2017)
No training or configuration required
Consistent method and terminology
Benchmarking across time and organization
Request access - [email protected]
![Page 25: RAMP: Repository Analytics and Metrics Portal fileAgenda 1. A New Reporting Model 2. Accuracy of Analytics Reporting Tools Page Tagging Log file 3. RAMP New Prototype Web Service for](https://reader030.vdocuments.us/reader030/viewer/2022040417/5d5672fd88c993df7b8bb022/html5/thumbnails/25.jpg)
![Page 26: RAMP: Repository Analytics and Metrics Portal fileAgenda 1. A New Reporting Model 2. Accuracy of Analytics Reporting Tools Page Tagging Log file 3. RAMP New Prototype Web Service for](https://reader030.vdocuments.us/reader030/viewer/2022040417/5d5672fd88c993df7b8bb022/html5/thumbnails/26.jpg)
![Page 27: RAMP: Repository Analytics and Metrics Portal fileAgenda 1. A New Reporting Model 2. Accuracy of Analytics Reporting Tools Page Tagging Log file 3. RAMP New Prototype Web Service for](https://reader030.vdocuments.us/reader030/viewer/2022040417/5d5672fd88c993df7b8bb022/html5/thumbnails/27.jpg)
Publications
Published:
Patrick OBrien, Kenning Arlitsch, Leila Sterman, Jeff Mixter, Jonathan Wheeler,
and Susan Borda. “Undercounting File Downloads from Institutional Repositories,”
Journal of Library Administration, vol. 56, no. 7, 2016
Forthcoming:
Patrick OBrien, Kenning Arlitsch, Leila Sterman, Jeff Mixter, Jonathan Wheeler,
and Susan Borda. “RAMP: Repository Analytics and Metrics Portal: A Prototype
Web Service that Accurately Counts Item Downloads from Institutional
Repositories,” Accepted by Library Hi Tech, expected early 2017
Proposal funded by IMLS:
”Measuring Up: Assessing Accuracy of Reported Use and Impact of Digital
Repositories” - scholarworks.montana.edu/xmlui/handle/1/8924
![Page 28: RAMP: Repository Analytics and Metrics Portal fileAgenda 1. A New Reporting Model 2. Accuracy of Analytics Reporting Tools Page Tagging Log file 3. RAMP New Prototype Web Service for](https://reader030.vdocuments.us/reader030/viewer/2022040417/5d5672fd88c993df7b8bb022/html5/thumbnails/28.jpg)
Undercounting Research Team
❖Montana State University
●Kenning Arlitsch, Dean @kenning_msu
●Patrick OBrien, Semantic Web Research Director @sempob
●Leila Sterman, Scholarly Communication Librarian @calamityleila
●Susan Borda, Digital Technologies Librarian @mutanthumb
❖OCLC Research
●Jeff Mixter, Software Engineer @jeffmixter
❖University of New Mexico
●Jonathan Wheeler, Data Curation Librarian