From Reputation to Citation: Varying Roles

for Scholarly MetricsElaine M. Lasda, MLS CAS

Associate Librarian, University at AlbanySpecial Libraries Association

July 18, 2017

What I’m Going to Say:

• Types of metrics/approaches for assessing research impact

• Reputation

• Bibliometric

• Altmetric

• Appropriate Usage and Best Practices

• Grassroots Movements

• Some common sense objectives

https://www.amazon.com/Make-

Your-Bed-Little-

Things/dp/1455570249

Why “Impact”?

https://flic.kr/p/SHujQt

Maximizing Research Impact Workshop

• Author Identifiers (ORCID)•Bibliometrics (oldies, goodies and freebies)

• Open Access Scholarship

• Altmetrics

• Three Quick Hits

•ORCID

•Scholars Archive (IR)

•Author Profile in WoS/Scopus/GS

“Reputation” approach

https://flic.kr/p/JTndV5

“Squishy” Reputation Approaches

• Colleagues – word of mouth

• Colleagues – published surveys

• Ulrich’s – content indexed in what dbs

• Editorial Board composition

• Peer researchers have published there

• Published scholarly surveys of disciplinary experts

Caveats of Reputation Approach

• Subjective

• Overlook new journals/output platforms

• Rely heavily on/misuse JIF metric

• Survey methodologies vary & don’t always exist

• Info can be outdated

Traditional citation metrics

https://flic.kr/p/9XtEKC

Bibliometrics: Context and Background

Pritchard, A. (1969). Statistical bibliography or bibliometrics. Journal of Documentation, 25, 348-9.

Traditional-ish

• Citation count

• JIF

• SNIP/SJR

• h-index/e-index, g-index, etc.

• i-10

New(er) Citation-Based Tools

• Cross disciplinary metrics

• InCites

• CiteScore

• Viz!

• Manuscript Matcher

• PoP

Caveats of bibliometrics

• Self citations

• “Cartels” of scholars, journals

• Significant variation in source datasets

• Quality control vs. Inclusivity

• Static or rigid disciplinary categories

•emerging fields

•inter- and multidisciplinary fields

Altmetrics (with a small “a”)

What are altmetrics? • ImpactStory

https://impactstory.org/u/0000-0002-9498-7074

• Altmetric

• Bookmarklet: https://www.altmetric.com/products/free-tools/bookmarklet

• Scholars Archive “donut”

http://scholarsarchive.library.albany.edu/ulib_fac_scholar/11/

http://scholarsarchive.library.albany.edu/ulib_fac_scholar/18/

• PlumX

In Ebscohost databases

http://libproxy.albany.edu/login?url=http://search.ebscohost.com/login.aspx?direct=true

&bquery=(altmetrics+AND+are+AND+awesome)&type=0&site=eds-live&scope=site

• Bought by Elseiver

Altmetrics: More of a Reputation or Citation Approach?

https://flic.kr/p/5xgQBY

If a Tweet mentions your paper,

does it make an impact?

https://flic.kr/p/5xgQBY

Role of Altmetrics

More of a quantifiable “reputation” indicator than empirical evidence of use

Can overcome caveats of more “squishy” reputation approaches

• Trendsetting/Immediacy

• Transcends disciplinary “silos”

• Can generate mainstream traction/impact

• Leads (hopefully) to increased impact in scholarly peer reviewed literature (ie, boosts one’s citation count)

Caveats of altmetric approaches

•Not Standardized•Transparency varies

• Which scholars are on what platforms?

•Who else is looking where our scholars look?

•Proprietary academic social media

•ResearchGate, Academia.edu, Mendeley, etc.

Whither “Gaming”?

https://flic.kr/p/6mzCJkhttps://flic.kr/p/6mzCJk

https://flic.kr/p/6mzCJk

Appropriate Usage of Scholarly Metrics

Rubber-Meets-the-Road Impact Schemas

Scholarly Metrics Best Practices

• Use transparent methodologies

• Supplemental to expert qualitative evaluation

• Altmetrics serve as a reputation measure (for now)

• Citation Metrics remain a more empirical measure

• Contextualize the metric to your need

• (institution, discipline, peer groups, country/region, journal, other?)

What I just talked about

• Workshop to address researcher/scholar needs on campus

• Reputation and empirical (citation) impact assessments are complementary

• Gaming happens with all measures, as does dirty data

• New approaches measure tangible outcomes of scholarly activity

• Best practices include transparent methods, contextualized interpretation, and usage that is supplemental to expert qualitative evaluation (no magic numbers!!!)

Heeeere’s

Richard!

Elaine M. Lasda

University at Albany

elasda@albany.edu

@ElaineLibrarian

http://slideshare.net/librarian68