beethoven vs. bieber on the meaningfulness of (alt)metrics
DESCRIPTION
Beethoven vs. Bieber On the meaningfulness of (alt)metrics. Blaise Cronin, PhD, DSSc , DLitt ( h.c .) Rudy Professor of Information Science, Indiana University, USA. Canonicity vs. Iconicity. Twit( ter ). Facebook. Biebermetrics. Facebook. Beethoven vs. Bieber. Apples & Oranges. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Beethoven vs. Bieber On the meaningfulness of (alt)metrics](https://reader035.vdocuments.us/reader035/viewer/2022062400/568168aa550346895ddf4a85/html5/thumbnails/1.jpg)
Beethoven vs. BieberOn the meaningfulness of (alt)metrics
Blaise Cronin, PhD, DSSc, DLitt (h.c.)Rudy Professor of Information Science,
Indiana University, USA
![Page 2: Beethoven vs. Bieber On the meaningfulness of (alt)metrics](https://reader035.vdocuments.us/reader035/viewer/2022062400/568168aa550346895ddf4a85/html5/thumbnails/2.jpg)
Canonicity vs. Iconicity
![Page 3: Beethoven vs. Bieber On the meaningfulness of (alt)metrics](https://reader035.vdocuments.us/reader035/viewer/2022062400/568168aa550346895ddf4a85/html5/thumbnails/3.jpg)
Twit(ter)
![Page 4: Beethoven vs. Bieber On the meaningfulness of (alt)metrics](https://reader035.vdocuments.us/reader035/viewer/2022062400/568168aa550346895ddf4a85/html5/thumbnails/4.jpg)
![Page 5: Beethoven vs. Bieber On the meaningfulness of (alt)metrics](https://reader035.vdocuments.us/reader035/viewer/2022062400/568168aa550346895ddf4a85/html5/thumbnails/5.jpg)
Biebermetrics
![Page 6: Beethoven vs. Bieber On the meaningfulness of (alt)metrics](https://reader035.vdocuments.us/reader035/viewer/2022062400/568168aa550346895ddf4a85/html5/thumbnails/6.jpg)
![Page 7: Beethoven vs. Bieber On the meaningfulness of (alt)metrics](https://reader035.vdocuments.us/reader035/viewer/2022062400/568168aa550346895ddf4a85/html5/thumbnails/7.jpg)
Beethoven vs. Bieber
![Page 8: Beethoven vs. Bieber On the meaningfulness of (alt)metrics](https://reader035.vdocuments.us/reader035/viewer/2022062400/568168aa550346895ddf4a85/html5/thumbnails/8.jpg)
Apples & Oranges
![Page 9: Beethoven vs. Bieber On the meaningfulness of (alt)metrics](https://reader035.vdocuments.us/reader035/viewer/2022062400/568168aa550346895ddf4a85/html5/thumbnails/9.jpg)
Apples … and apples
![Page 10: Beethoven vs. Bieber On the meaningfulness of (alt)metrics](https://reader035.vdocuments.us/reader035/viewer/2022062400/568168aa550346895ddf4a85/html5/thumbnails/10.jpg)
The numbers game
‘Not everything that can be counted counts and not everything that counts can be counted’
![Page 11: Beethoven vs. Bieber On the meaningfulness of (alt)metrics](https://reader035.vdocuments.us/reader035/viewer/2022062400/568168aa550346895ddf4a85/html5/thumbnails/11.jpg)
Defining excellence
‘But I know it when I see it ….’U.S. Supreme Court Justice Potter Stewart, 1964
‘It is easier to measure activities than it is to measure quality’NISO, 2014
![Page 12: Beethoven vs. Bieber On the meaningfulness of (alt)metrics](https://reader035.vdocuments.us/reader035/viewer/2022062400/568168aa550346895ddf4a85/html5/thumbnails/12.jpg)
Assessing excellence(Peer review … on a good day)
• Domain knowledge• Expertise• Discrimination• Connoisseurship• Wisdom
![Page 13: Beethoven vs. Bieber On the meaningfulness of (alt)metrics](https://reader035.vdocuments.us/reader035/viewer/2022062400/568168aa550346895ddf4a85/html5/thumbnails/13.jpg)
A culture of metrics
Yesterday (c. 1955): ISI’s Citation indexes (SCI, SSCI A&HCI)
Today: WoS, Scopus, Google Scholar, et al.
Tomorrow: Social media monitoring & analytics (e.g., altmetric.com)
![Page 14: Beethoven vs. Bieber On the meaningfulness of (alt)metrics](https://reader035.vdocuments.us/reader035/viewer/2022062400/568168aa550346895ddf4a85/html5/thumbnails/14.jpg)
Measuring article impact
• Reputation of journal• Journal Impact factor• No. of citations• Quality of citations• Persistence of citations
• Times accessed• Times downloaded• Included in syllabi• Media mentions etc.
![Page 15: Beethoven vs. Bieber On the meaningfulness of (alt)metrics](https://reader035.vdocuments.us/reader035/viewer/2022062400/568168aa550346895ddf4a85/html5/thumbnails/15.jpg)
Beyond bibliometrics
• Citations miss important traces/impacts & are lagged
• Online reference managers, slide-sharing services and social media capture impacts in real-time (but only partially)
![Page 16: Beethoven vs. Bieber On the meaningfulness of (alt)metrics](https://reader035.vdocuments.us/reader035/viewer/2022062400/568168aa550346895ddf4a85/html5/thumbnails/16.jpg)
Twitter reality
a) All scholars b) % with Twitter accounts c) % who tweet d) % of tweets about scholarly topics e) % that are “Twitter citations”/ “tweetations”
![Page 17: Beethoven vs. Bieber On the meaningfulness of (alt)metrics](https://reader035.vdocuments.us/reader035/viewer/2022062400/568168aa550346895ddf4a85/html5/thumbnails/17.jpg)
Scholarly buzzometer
![Page 18: Beethoven vs. Bieber On the meaningfulness of (alt)metrics](https://reader035.vdocuments.us/reader035/viewer/2022062400/568168aa550346895ddf4a85/html5/thumbnails/18.jpg)
‘Researchers must ask if altmetrics really reflect impact, or just empty
buzz.’ – altmetics manifesto
![Page 19: Beethoven vs. Bieber On the meaningfulness of (alt)metrics](https://reader035.vdocuments.us/reader035/viewer/2022062400/568168aa550346895ddf4a85/html5/thumbnails/19.jpg)
Citations ≠ Altmetrics
Cites authorsAlts readers
Cites fewAlts many
Cites laggedAlts real-time
Cites mandatoryAlts optional
![Page 20: Beethoven vs. Bieber On the meaningfulness of (alt)metrics](https://reader035.vdocuments.us/reader035/viewer/2022062400/568168aa550346895ddf4a85/html5/thumbnails/20.jpg)
Effects of researchImmediate vs. delayed impacts
Scholarly vs. professional vs. social impacts
Cited vs. read vs. used
Substance vs. buzz
![Page 21: Beethoven vs. Bieber On the meaningfulness of (alt)metrics](https://reader035.vdocuments.us/reader035/viewer/2022062400/568168aa550346895ddf4a85/html5/thumbnails/21.jpg)
Immediate effects: PLOS article-level metrics
![Page 22: Beethoven vs. Bieber On the meaningfulness of (alt)metrics](https://reader035.vdocuments.us/reader035/viewer/2022062400/568168aa550346895ddf4a85/html5/thumbnails/22.jpg)
![Page 23: Beethoven vs. Bieber On the meaningfulness of (alt)metrics](https://reader035.vdocuments.us/reader035/viewer/2022062400/568168aa550346895ddf4a85/html5/thumbnails/23.jpg)
Article-level metrics
• Transparency• Real-time• Multi-dimensional• Countable
• Ego-boosting• Behavior-modifying• Culturally corrosive?
![Page 24: Beethoven vs. Bieber On the meaningfulness of (alt)metrics](https://reader035.vdocuments.us/reader035/viewer/2022062400/568168aa550346895ddf4a85/html5/thumbnails/24.jpg)
![Page 25: Beethoven vs. Bieber On the meaningfulness of (alt)metrics](https://reader035.vdocuments.us/reader035/viewer/2022062400/568168aa550346895ddf4a85/html5/thumbnails/25.jpg)
Academic social capital• Highly ‘liked’
• Much tweeted/followed
• Heavily blogged about
• Frequently recommended
• Often mentioned/quoted in the media
![Page 26: Beethoven vs. Bieber On the meaningfulness of (alt)metrics](https://reader035.vdocuments.us/reader035/viewer/2022062400/568168aa550346895ddf4a85/html5/thumbnails/26.jpg)
Not to be confused! Social capital Symbolic capital
![Page 27: Beethoven vs. Bieber On the meaningfulness of (alt)metrics](https://reader035.vdocuments.us/reader035/viewer/2022062400/568168aa550346895ddf4a85/html5/thumbnails/27.jpg)
Complementary metrics
• Acknowledgments• Data citation counts• Micro-attributions for data curation• Social media mentions• Recommendations• Downloads• Mentions in extra- scientific texts• Press coverageetc., etc…
![Page 28: Beethoven vs. Bieber On the meaningfulness of (alt)metrics](https://reader035.vdocuments.us/reader035/viewer/2022062400/568168aa550346895ddf4a85/html5/thumbnails/28.jpg)
Anticipating altmetrics: ‘Invoked on the Web’
(Cronin et al., 1998)
‘polymorphous mentioning’
‘presence density’
‘diverse ways in which academic influence is exercised and acknowledged’
![Page 29: Beethoven vs. Bieber On the meaningfulness of (alt)metrics](https://reader035.vdocuments.us/reader035/viewer/2022062400/568168aa550346895ddf4a85/html5/thumbnails/29.jpg)
Jason Priem, 2011
![Page 30: Beethoven vs. Bieber On the meaningfulness of (alt)metrics](https://reader035.vdocuments.us/reader035/viewer/2022062400/568168aa550346895ddf4a85/html5/thumbnails/30.jpg)
Genres of altmetricsTaylor & Plume (2014)
Social activity(tweets, ‘likes’)
Mass media(news coverage)
Scholarly commentary(scientific blogs)
Scholarly activity(reference managers)
Altmetrics
![Page 31: Beethoven vs. Bieber On the meaningfulness of (alt)metrics](https://reader035.vdocuments.us/reader035/viewer/2022062400/568168aa550346895ddf4a85/html5/thumbnails/31.jpg)
Unequal attentionTaylor & Plume (2014)
![Page 32: Beethoven vs. Bieber On the meaningfulness of (alt)metrics](https://reader035.vdocuments.us/reader035/viewer/2022062400/568168aa550346895ddf4a85/html5/thumbnails/32.jpg)
The hunt for correlations…
• Citations in Wikipedia and JCR data (Nielsen, 2007)• Article tweets and citations (Eysenback, 2012)• F1000 score and JIF (Nature Neuroscience, 2005)• Inclusion in reference managers and citations (Bar-Ilan, 2012)• Downloads and subsequent citations (Brody et al., 2006; Nieder, Dalhaug,
Aandahl, 2013)• Citations in blogs and subsequent citations (Shema, Bar-Ilan, Thelwall, 2013)• Altmetrics and citations (Thelwall, Haustein, Larivière & Sugimoto, 2013;
Costas, Zahedi & Wouters, 2014)Etc., etc., ….
![Page 33: Beethoven vs. Bieber On the meaningfulness of (alt)metrics](https://reader035.vdocuments.us/reader035/viewer/2022062400/568168aa550346895ddf4a85/html5/thumbnails/33.jpg)
Downloads vs. citations ScienceDirect (Moed, 2012)
![Page 34: Beethoven vs. Bieber On the meaningfulness of (alt)metrics](https://reader035.vdocuments.us/reader035/viewer/2022062400/568168aa550346895ddf4a85/html5/thumbnails/34.jpg)
Downloads & citationsNieder, Dalhaug & Aandah (2013)
![Page 35: Beethoven vs. Bieber On the meaningfulness of (alt)metrics](https://reader035.vdocuments.us/reader035/viewer/2022062400/568168aa550346895ddf4a85/html5/thumbnails/35.jpg)
Twitter mentions & arXiv downloadsShuai, Pep, Bollen (2012)
![Page 36: Beethoven vs. Bieber On the meaningfulness of (alt)metrics](https://reader035.vdocuments.us/reader035/viewer/2022062400/568168aa550346895ddf4a85/html5/thumbnails/36.jpg)
Downloading, reading, citingSchlögl et al. (2014)
![Page 37: Beethoven vs. Bieber On the meaningfulness of (alt)metrics](https://reader035.vdocuments.us/reader035/viewer/2022062400/568168aa550346895ddf4a85/html5/thumbnails/37.jpg)
1 citation = ? tweets
• Citations• Acknowledgments• Downloads• Tweets• ‘Likes’ etc.
![Page 38: Beethoven vs. Bieber On the meaningfulness of (alt)metrics](https://reader035.vdocuments.us/reader035/viewer/2022062400/568168aa550346895ddf4a85/html5/thumbnails/38.jpg)
(Alt)metrics issuesMetrics
• Validity
• Reliability
• Utility
• Ethicality
Platforms
• Transparency• Usability• Persistence• Cost/benefit ratio
![Page 39: Beethoven vs. Bieber On the meaningfulness of (alt)metrics](https://reader035.vdocuments.us/reader035/viewer/2022062400/568168aa550346895ddf4a85/html5/thumbnails/39.jpg)
Mirror, mirror on the wall, who is the fairest of them all?
Users, narcissism and control – tracking the impact of scholarly publications in the 21st centuryWouters & Costas (2012)
![Page 40: Beethoven vs. Bieber On the meaningfulness of (alt)metrics](https://reader035.vdocuments.us/reader035/viewer/2022062400/568168aa550346895ddf4a85/html5/thumbnails/40.jpg)
Google Scholar: Narcissism?
![Page 41: Beethoven vs. Bieber On the meaningfulness of (alt)metrics](https://reader035.vdocuments.us/reader035/viewer/2022062400/568168aa550346895ddf4a85/html5/thumbnails/41.jpg)
![Page 42: Beethoven vs. Bieber On the meaningfulness of (alt)metrics](https://reader035.vdocuments.us/reader035/viewer/2022062400/568168aa550346895ddf4a85/html5/thumbnails/42.jpg)
Scholarly Panopticon?
‘an Orwellian surveillance net’
‘cybernating the academy’
Sosteric, 1999
![Page 43: Beethoven vs. Bieber On the meaningfulness of (alt)metrics](https://reader035.vdocuments.us/reader035/viewer/2022062400/568168aa550346895ddf4a85/html5/thumbnails/43.jpg)
The Holy Grail of holism
A matrix of established & alternative metrics?
A unified measure/composite score (a super h-index)?
![Page 44: Beethoven vs. Bieber On the meaningfulness of (alt)metrics](https://reader035.vdocuments.us/reader035/viewer/2022062400/568168aa550346895ddf4a85/html5/thumbnails/44.jpg)
![Page 45: Beethoven vs. Bieber On the meaningfulness of (alt)metrics](https://reader035.vdocuments.us/reader035/viewer/2022062400/568168aa550346895ddf4a85/html5/thumbnails/45.jpg)
![Page 46: Beethoven vs. Bieber On the meaningfulness of (alt)metrics](https://reader035.vdocuments.us/reader035/viewer/2022062400/568168aa550346895ddf4a85/html5/thumbnails/46.jpg)
New Age numerology?• Atomization of inputs, outputs and impacts
• Fetishization of metrics
• Transparency vs. triviality
• Immediacy vs. canonicity
• Goal displacement?
![Page 47: Beethoven vs. Bieber On the meaningfulness of (alt)metrics](https://reader035.vdocuments.us/reader035/viewer/2022062400/568168aa550346895ddf4a85/html5/thumbnails/47.jpg)
Suggested readingsScholarly Metrics Under the
Microscope
edited by
Blaise Cronin and Cassidy R. Sugimoto
2014
ITI / ASIST
![Page 48: Beethoven vs. Bieber On the meaningfulness of (alt)metrics](https://reader035.vdocuments.us/reader035/viewer/2022062400/568168aa550346895ddf4a85/html5/thumbnails/48.jpg)
And the winner is ….