overview of citation metrics

BASIC CITATION METRICS: AN OVERVIEW

WHAT DO BIBLIOMETRICS MEASURE?

Eugene Garfield“Father of Citation

Analysis”

• Scholarly communication: tracing the history and evolution of ideas from one scholar to another

• Measures the scholarly influence of articles, journals, scholars, institutions

Photo: http://www.chemheritage.org/Discover/Collections/Oral-Histories/Details/asset_upload_file66_69008_thumbnail.jpg

SOURCES OF CITATION DATA AT UALBANY

• UAlbany subscription is 1993-present• Pre-1993 data is available in print volumes – Ask A Librarian!• Currently covers over 12,000 journals • Journals undergo quality review process prior to inclusion• Weak in conference proceedings, books, data repositories• Master Journal List of Web of Science

• Many subject specific databases also include cited reference information.

• There will be overlap in coverage of different databases on a cover, take care to identify/remove duplicate citations.

• See Subject Specific Databases on this guide for more information on these resources.

SOURCES OF CITATION DATA

• Journal coverage unknown, sporadic– Whatever the web crawler can “read”

• No quality control process• Better coverage of

– conference proceedings– foreign language articles– Nontraditional publication sources (blogs, reports, white papers)– Theses and dissertations

• Coverage roughly from 1996-present; some older but inconsistent

SOURCES OF CITATION DATA

• Citation data overlaps in all sources, but not completely

• Unique citing references in all databases

• Unique metrics developed using each major database

WHAT’S THE DIFFERENCE?

• Citation Count• Journal Impact

Factor• Eigenfactor• H-index• SNIP• SJR

METRICS

• Number of times cited in scholarly journals indexed in a particular resources

• Citation count metric does not take into account:– Materials not included in

citation database– Sometimes self citations

are eliminated from count

CITATION COUNT

• Found in: Web of Science – Journal Citation Reports

• Essential concept: “how fast are ideas spreading from this journal to other publications?”

• Formula:

JOURNAL IMPACT FACTOR

Number of citations to a journal in a given year from articles occurring in the past 2

years, divided by the number of scholarly

articles published in the journal in the past 2 years

Journal of Hypothetical Examples

JOURNAL IMPACT FACTOR

Citing references appearing in 2010, to articles published in Journal in 2009 and 2008

100200

Total number of articles in Journal published in 2009 and 2008

0.50 JIF

• Cannot be used to compare across disciplines

• Two year time frame not adequate for social sciences, humanities

• Coverage of some disciplines not sufficient in Web of Science

• Is a measure of “impact” a measure of “quality”?

CONCERNS WITH JIF

• Found in: – eigenfactor.org – Web of Science:

Journal Citation Reports

• Essential concept: Takes into account the total amount of “citation traffic” appearing in JCR

• Formula:

EIGENFACTOR

Influence of the citing journal, Divided by the total number of citations appearing in that journal.

• Journal Impact Factor: – All citing references weighted

equally

• Eigenfactor: – SOME CITING REFERENCES ARE

MORE IMPORTANT THAN OTHERS• The citing articles from

journals that are heavily cited themselves demonstrate greater influence

ESSENTIAL DIFFERENCES: JIF & EIGENFACTOR

• Eigenfactor will always be bigger if a journal is larger, i.e., publishes more articles

• Article Influence Score: corrects for journal size– takes the journal’s

Eigenfactor score and further divides it by the number of articles in the journal.

– Correlated to the JIF

CONSIDERTHE FOLLOWING

H-INDEX

http://proj.ncku.edu.tw/research/commentary/e/20080815/images/0808110317430yfDpK.jpg

• Found in:• Many citation analytics

sources • Web of Science• Google Scholar Citations• Etc…

• Essential Concept: Identifies a sustained record of multiple publications that have been cited frequently

• Formula:The maximum number of articles h, such that each has received at least h citations.

Jorge E. Hirsch

1 2 3 4 5 6 70

5

10

15

20

25

30

H-indexScholar AScholar B

Article Number

Num

ber

of

Cit

ati

ons

H-INDEX EXAMPLE

Scholar A Scholar B

10 2710 129 58 47 46 26 2

56 citations 56 citations6 h-index 4 h-index

• Found in: – www.journalmetrics.c

om– Scopus

• Essential concept: Citation Potential: total number of citing references in all journals which have cited this journal

• Formula:

SNIP: SOURCE NORMALIZED IMPACT PER PAPER

The ratio of the journal’s average citation count per paper to the citation potential in its subject field

• Can compare SNIP scores across disciplines

• Aggregate of a journal, so larger journals automatically have higher scores than smaller journals

PROS AND CONS OF SNIP

• Found in: – www.scimagojr.com– Scopus data

• Essential Concept: measures “current average prestige per paper”

• Formula:

SJR: SCIMAGO JOURNAL RANKING

Prestige factors include: # of journals in the Scopus database, # of articles in Scopus from this journal, citation count, eigenvector analysis of important citing references, corrections for self-citations, and normalization by the number of significant works published in the journal.

• Corrects for self citations• Correlated to JIF• Scores can be compared

across disciplines• Web version provides

data on countries• Three year window not

good for social sciences

PROS AND CONS OF SJR

• Journal quality ≠ article quality

• Citing a work ≠ agreement with that work

• Self citations: including them is controversial

• Variance in citation patterns from domain to domain

KEEP IN MIND

• Is a trend analysis of scholarly influence over time needed, or simply a count of total citations?

• Does the database adequately cover the key journals in my subject domain?

• Are there significant works in my domain that are not peer reviewed, scholarly journals (such as conference proceedings, government reports, etc.)

KEEP IN MIND