the secret life of articles: from download metrics to downstream … · 2016-02-17 · the secret...
TRANSCRIPT
Center for Information and Communication Studies
The secret life of articles: From download metrics to downstream impact
Carol Tenopir University of Tennessee
Lorraine Estelle Project COUNTER
Wouter Haak Elsevier
CIBER Research Ltd.
Center for Information and Communication Studies
Beyond Project COUNTER
• What are download counts missing?
• How much do scholars share and what do they share?
• How might sharing be measured?
• What is a more complete value of articles?
Center for Information and Communication Studies
14 focus group participants 15 interviews
Interviews / Focus Groups Demographics
Center for Information and Communication Studies
International Survey with 1000 responses
Center for Information and Communication Studies
Main takeaways
1. A means to an end 2. Function drives form 3. Email is #1 4. Version matters 5. The library is key
Center for Information and Communication Studies
1. A MEANS TO AN END
Center for Information and Communication Studies
Center for Information and Communication Studies
“If I have got it, I will just share it. It is easier than having them go track it down, even if I have got the citation.”
“It’s not like I’m giving away something that they could not get anyhow. But it never dawned on me that in effect, you know in theory, I am violating copyright.”
Center for Information and Communication Studies
When working in a research group …
N=771
Per
cent
share more 67%
sharing habits would not change
30%
share less 3%
Center for Information and Communication Studies
2. FUNCTION DRIVES FORM
Center for Information and Communication Studies
Interviews / Focus Groups
Two types of saving and sharing 1. Metadata (citation) When other researchers could easily access articles. 2. Saving document (pdf). When access was an issue.
Full-text vs. reference/link sharing
Center for Information and Communication Studies
Their most recent link or reference is shared …
Email Learning Management Software an average of 10.5
(8.2 minus outliers) times per download.
an average of 13.9 (5.4 minus outliers) times per download.
Center for Information and Communication Studies
Their most recent full-text is shared …
Email Research social networks an average of 10.5 (8.4 minus outliers) times per download.
an average of 10.8 (5.3 minus outliers) times per download.
Center for Information and Communication Studies
3. EMAIL IS #1
Center for Information and Communication Studies
Top preferred methods of sharing for research
Email ranked 1st with 73.8% respondents listing as most preferred method of sharing.
Cloud services 2nd with 11.6% respondents listing as most preferred method of
sharing.
Internal networks ranked 3rd with 10.7% respondents listing as most preferred method of sharing.
Center for Information and Communication Studies
Top preferred methods of sharing for teaching
Email ranked 1st with 82.8% respondents listing as most preferred method of sharing.
Internal networks ranked 3rd with 8% respondents listing as most preferred method of sharing
Learning management systems ranked 2nd with 9.6% respondents listing as most preferred method
of sharing
Center for Information and Communication Studies
However, researchers feel more people are reached through …
General social networks: 96-614 people potentially reached
Internal networks: 40-189 people potentially reached
Research social networks: 559-4066 people potentially reached
Center for Information and Communication Studies
4. VERSION MATTERS
Center for Information and Communication Studies
9.7%
77.3%
9.3% 3.8% 5.5%
83.9%
7.3% 3.3%
Preprint Published Accepted MSS Other
When sharing scholarly articles, are you more likely to share Own Work Other's Work
Center for Information and Communication Studies
My feeling is that I’ve written this, this is my work, they’re publishing it. I’m not getting paid for this, they’re benefiting from this. I’m a little fish . . . Let them come for me.
The main point should be that I share a version that is evidentially the peer-reviewed version (with the publisher and the journal stamp on it).
Center for Information and Communication Studies
5. THE LIBRARY IS KEY
Center for Information and Communication Studies
Average number of downloads for last research project:
65
Average number of downloads for last teaching term:
26
Center for Information and Communication Studies
54.1%
9.4% 8.8%
18.9%
7.0% 6.1%
35.4%
5.8% 5.7% 8.5%
3.4% 6.1%
0%
10%
20%
30%
40%
50%
60%
Library Otherdatabases
Institutionalrepositories
Publisherwebsites
Researchsocial
networks
Otherwebsites
What percentage of your downloads were from the following sources?
Research Teaching
Center for Information and Communication Studies
But, library download counts may underestimate use, because when returning to an article …
61.3%
11.7%
57.3%
9.4%
Saved copy Download again
Research Teaching
Center for Information and Communication Studies
Sharing is a natural behavior
• Sharing supports research & disseminating results
• Convenience is a key concern
• Sharing channels chosen for fit to
work style • New policies and measures must
fit preferences & likely behavior
Center for Information and Communication Studies
Calculating sharing metrics: Possible approaches
Lorraine Estelle
Center for Information and Communication Studies
Range of Systems: Key findings include: • Lack of global usage data standards makes quantifying sharing
problematic.
• Difficult to obtain data on authors’ sharing via email and cloud services.
• Data obtained is likely to be out of date rather quickly.
Center for Information and Communication Studies
Two suggested approaches
1. Data-based approach employing usage data from publishers and citation data – where such data is comprehensive and reliable.
2. Survey-based approach using an online community as a basis for monitoring sharing behavior over time.
Center for Information and Communication Studies
A data-based approach to measuring the extent of scholarly materials sharing
1. Data on article sharing by formal and informal methods will be difficult to collect reliably owing to the large number of channels used and the lack of global standards
2. The mix of sharing methods used varies from discipline to discipline and from year to year
3. It would make sense to supplement a data-based approach to measuring post-download usage with regular surveys of the scholarly community
Center for Information and Communication Studies
Full-Text Sharing Calculator • Can a calculator be derived from the findings? • If not, can a suggested range of “sharing” be derived from the
findings?
Center for Information and Communication Studies
Problems with a calculator…
• Ranges and SE are high • Outliers skew results • Demographics matter a lot
Center for Information and Communication Studies
A range of sharing, in lieu of a calculator, may be the answer. This range can come in the form of an overall range (derived from all respondents) or categorized ranges (derived from selected identifying categories).
Center for Information and Communication Studies
Demographics sometimes matter…
• Researchers age 39 and younger share less through formal methods
• Researchers age 40-59 rely more on formal sharing methods and post slightly less, but to outlets that reach more people
Center for Information and Communication Studies
Demographics sometimes matter…
Social scientists share more via social media.
Center for Information and Communication Studies
Alternatives • Any estimate on number of times an article is shared per
download depends on many things.
• An alternative to an exact calculator would be to apply a Confidence Interval to the findings and provide an estimated “range of sharing”, with an lower bound and upper at a 95% level of confidence.
• This range could in-turn be combined with download
numbers to estimate a level of actual, post-download usage.
Center for Information and Communication Studies
Collaborative initiatives with
• DOI Event Tracking – Open, common infrastructure to track activities
around DOIs recording user content interactions
• Distributed Usage Logging – Peer-to-peer system for the exchange of
usage data on user content interactions
From download metrics to downstream impact => implications for Elsevier
Oct 2015
Wouter Haak [email protected]
• The graph shows how many users have joined Mendeley groups year on year between 2009-2015 • The results were calculated based on groups with more than 1 member • The 2015 extrapolation assumes that the last 10 months’ growth will continue towards end of the year
Source: Mendeley analytics team
49%
17% -15%
17%
165%
697%
At Elsevier / Mendeley we also see that online sharing and collaborating is increasingly important (1)
• The graph shows how many documents were added to the groups by users year on year between 2009-2015 • The results were calculated based on groups with more than 1 member • The 2015 extrapolation assumes that the last 10 months’ growth will continue towards end of the year Source: Mendeley analytics team
148%
546%
46%
3% 24%
14%
At Elsevier / Mendeley we also see that online sharing and collaborating is increasingly important (2)
1) For the researcher: sharing in
2) For the institution: collaborations in
3) With publishers and platforms: principles for sharing on SCN’s
Actions undertaken and next steps – based on this research
40
1) For the researcher: sharing in
41
42
43
44
2) For the institution: collaborations in
45
46
47
3) With publishers and platforms: principles for sharing on SCN’s
48
• SCNs are platforms that allow researchers to develop and maintain professional relationships
• Typically they enable
- Discussion or research and information exchange - Sharing of research results, articles and data
• SCNs have grown substantially* in recent years, particularly in the “big”
three – Academia.edu, ResearchGate, and Mendeley
• STM working group identified 40+ such networks
*CCC/CLA Report on Scholarly Collaboration Platforms, M. Ware, November 27,2014 (CCC/Outsell)
Social Collaboration Networks (SCNs)
49
STM assembled a working group in September 2014 to examine the current landscape of article sharing via SCNs and publishers’ role
Ten publishers, both for profit and not-for-profit, and some of whom owned or interacted with SCNs, took part
They produced the draft set of ‘voluntary principles for article sharing on scholarly collaboration networks’ and posted it for a two month STM Open Consultation in early 2015, and invited wide participation among all stakeholder groups
STM Consultation on Article Sharing
50
The principles are meant to: • clarify how, where and what content should be shared using
these networks and sites,
• improve the experience for all stakeholders
• encourage publishers and SCN’s to work together to facilitate sharing, which benefits researchers, institutions, and society as a whole
• All comments and a summary posted on the STM website • Key recommendation: Principles should be: clear, simple, uniform
Voluntary Principles for Article Sharing on SCNs
51
• Based on the community’s constructive feedback, the working group posted a revised set of principles August 28.
http://www.stm-assoc.org/stm-consultations/scn-consultation-2015/
• Posting includes an explanation of the changes made to the principles as a result
Response to STM’s Open Consultation
52
• Publishers commit to facilitate the dissemination and discovery of their authors’ scholarly articles.
• Sharing should be allowed within a research collaboration group (to be defined by the group) - Allow article sharing between subscribers and non-subscribers
within the group - Include members of the wider public participating with the group - Include commercial researchers, subject to publisher policy or
appropriate licensing • Publishers and libraries should extend their collective use of standards
such as COUNTER to quantify article use on networks.
• Publishers and standards organizations should continue to work together on tools that facilitate sharing (article versioning and access rights metadata).
• Publisher policies on research collaboration group sharing and public posting of articles should be clear and easily discoverable.
Voluntary principles: key points
53
Publisher posting policy - Several publishers (e.g. NPG, Wiley, Elsevier) have issued policies that address posting via SCNs.
Article tagging project - To support implementation a sub-working group is tasked with demonstrating a technical solution based on tagging xmp metadata in scholarly articles
The SCN working group continues to reach out for wider involvement, including non-publisher owned networks
Comments/updates on progress, email [email protected] Updates will continually be posted on:
www.stm-assoc.org/stm-consultations/scn-consultation-2015/
Next steps
54
Seeking your involvement and/or endorsement http://www.stm-assoc.org/stm-consultations/scn-consultation-2015/
Alternatively: Google for “STM principles” or “STM consultation”
Next steps
55
1) For the researcher: sharing in
2) For the institution: collaborations in
3) With publishers and platforms: principles for sharing on SCN’s
From download metrics to downstream impact => implications for Elsevier
56