Introduction to Citation Metrics Library Tutorial for PC5198

Geok Kee slbtgk@nus.edu.sg 6 March 2014

1

Searching in databases

Introduction to citation metrics • Journal metrics

• Author impact metrics

Managing your publication history

Outline

2

Searching in Databases

List search terms 1 Form search

statement 2 Choose a database 3

Keywords, synonyms, alternative words

Research topic

3

Operators Function

AND Results must contain all search terms (To narrow searches) waves and interference

OR Result contains either of the search terms (To broaden searches) handphone or mobile phone

Truncation *, ?, $

enzym* enzyme, enzymes, enzymatic, enzymology

(zero to many characters) synthesi?e synthesise, synthesize (one character) colo$r color, colour (zero or one character)

Phrase searching “ ”

Search for exact phrase “energy conservation”

Proximity Operators NEAR, SAME

Used to specify search terms appearing in the same sentence. fiber NEAR composites

Parenthesis, ( ) Use for grouping of words/phrases & to specify order of search

4

Topic

Study of electronic structure of graphene using

spectroscopic ellipsometry and ultrafast pump-

probe techniques

5

Search statements

electronic structure graphene

“electronic structure” graphene

graphene ellipsometry “pump probe”

AND

AND OR ( )

6

Web of Science Core Collection

One of the largest citation index 52 million papers, 800 million citations

Multidisciplinary Organised into 250 categories covering Science, Social Sciences, Arts & Humanities

Our subscription – Science Citation Index Expanded (1900-present) – Social Sciences Citation Index (1900-present) – Arts & Humanities Citation Index (1975-present) – Conference Proceedings Citation Index- Science (1990-present) – Conference Proceedings Citation Index- Social Science &

Humanities (1990-present)

Updated weekly (Thomson Reuters, 2013)

7

Search history

8

Which journal articles to read?

Review Process

Edited Peer Reviewed

Self- Published

Editors Subject experts

Author

9

What is peer review?

Definition of peer review

“Peer review is the critical assessment of manuscripts submitted to journals by experts who are not part of the editorial staff.” International Committee of Medical Journal Editors (ICMJE). Uniform Requirements for Manuscripts Submitted to Biomedical Journals Source: Hames, Irene (2007). Peer Review and Manuscript Management in Scientific Journals: Guidelines for Good Practice. Blackwell Publishing.

10

Nature Physics

11

12

13

Which journal articles to read?

• Read peer reviewed articles

• Read review articles for historical development of a subject – Primary vs secondary sources

• Retrieve relevant papers in your subject area

• Identify high impact papers

• Trace research progress using citations

14

Citation network in Web of Science

2007 Times

Cited Cited

References

1996

2004

2006

2013

2008

2010

2004

2008

1998

2011

Related Records

Paper A

Citing

articles

15

Related Records

16

17

Introduction to Citation Metrics

Cited Reference Searching Guide: http://libguides.nus.edu.sg/citedref

18

Journal metrics

Journal Citation Reports (Thomson Reuters) Impact Factor, 5-Year Impact Factor, Immediacy Index

Eigenfactor® Score http://www.eigenfactor.org/

Article Influence® Score

SCImago Journal Rank (SJR) & Source Normalised Impact per Paper (SNIP) – Can be accessed via Scopus http://www.scimagojr.com/

19

Journal Citation Reports

• Quantitative tool or resource for ranking, evaluating, categorising and comparing journals

• Published annually, compiled using citation data drawn from over 11,000 scholarly and technical journals from more than 3,300 publishers in over 80 countries

20

Average number of times articles from the journal published in the past two years have been cited in the JCR year

Used in evaluating or comparing a journal's relative

importance to others in the same field/subject category

Underlying assumption No. of citations is an indication of the importance & influence of a journal

Journal Impact Factor

Impact Factor = Cites in 2012 to 2010 & 2011 articles

No. of articles published in 2010 & 2011 (2012)

21

Reviews of Modern Physics

22

Journals in Physics ranked by impact factor

23

Journal impact factor: Trend graph

24

Exercise: Journal rankings in a subject category

Launch Journal Citation Reports

Select View a group of journals by: Subject category

“Physics, Multidisciplinary”

View Journal Data by “impact factor”

Repeat the steps for

Mathematics Computer Science Chemistry Engineering Physiology, …

What do you observe?

25

Exercise

Find the impact factor of the journal, Acta Crystallographica Section A

Look for the trend graph

What can you deduce?

26

Example of a journal where its impact factor is inflated

Year Total Citations

Impact Factor

2007 260 2.385

2008 201 2.051

2009 6,091 49.926

2010 6,520 54.333

2011 245 2.076

URL: http://libguides.nus.edu.sg/citedref

27

5 Year Journal Impact Factor

Based on cites in the current year received by articles published in the prior 5 years

For journals – Where body of citations is not large enough to make

reasonable comparisons

– With consistently late publication schedules

– Of subject fields which require longer citation period, e.g. Mathematics, Geology or Management or Sociology, etc

28

Journal immediacy index

Average number of times an article is cited in the year it is published

Indicates how quickly articles in a journal are cited

Immediacy Index no. of citations (in current year)

no. of articles (in current year) (in current year) =

29

Issues to note when using impact factor

Different disciplines have different publication cultures & citation practices – Cannot compare journals across disciplines – Impact factor varies considerably among disciplines

Short citation window favours rapidly moving fields – In some disciplines, many citations occur outside of the

2-year window

Not all journals are indexed, analyses are limited to the journals listed in JCR, few publications from languages other than English are included

30

May be subject to manipulation ‒ Eg Self citations; publish a larger percentage of review

articles which receive more citations

Review journals tend to have higher impact factors than those publishing original research in the same field

Citation profile of journals often skewed

‒ Distribution of citation counts for individual papers in a journal is highly skewed

Issues to note when using impact factor

31

Some misconceptions

Articles published in a high impact journal will receive more citations

– No correlation between publishing in a high impact journal and the citations an article receives

Individual articles published in a high impact journal must also be of excellent quality

It is inappropriate to use impact factors to assess quality of individual articles or evaluate/compare researchers

32

Other Journal Metrics Eigenfactor Score

Rates journals according to the number of incoming citations, similar to page-rank algorithm used by Google & citations from highly-ranked journals are weighted higher

Article Influence Score

Measures the average influence, per article, of the papers in a journal

SCImago Journal Rank (SJR)

Also uses algorithms similar to page-rank in Google. Considers the prestige of the journal based on the ranking of the sources of the incoming citations

Source Normalised Impact per Paper (SNIP)

Accounts for subject differences by weighting citations based on the total number of citations in a subject field (Citations in subject field less likely to receive citations are given more weight)

URL: http://libguides.nus.edu.sg/citedref

33

Researcher impact metrics

Citation based metrics Number of publications Citation Count h-index

Other measures: Peer review (Gold standard)

- Assessment by fellow researchers

Measures of esteem such as invitations, membership on editorial boards, funding grants & awards

Underlying assumption No. of times a publication is cited by other researchers is an indication of the influence or impact the research has within the field

34

Cited Reference Search

35

Professor Wee Thye Shen, Andrew Professor Andrew Wee T S

36

Cited Reference Search

37

Issues & limitations of citation metrics

Citations take time to accrue

Citation comparisons are only meaningful if comparing like with like, i.e. researchers in the same field of research and at similar career stages

Raw count of citations & analyses depend on coverage of the data sources/citation databases

38

Citation Data Sources/Databases

Web of Science Scopus Google Scholar

> 12,000 Journals

> 148,000 Proceedings

~ 19,500 Journals

~ 250 Proceedings

No. not stated - Journals, Papers, Theses, Acad. Books, Conf. Proceedings, Abstracts & Tech. Reports

From 1900 - From 1996 - Not stated

Worldwide

N. American bias

Worldwide

50% from Europe, Latin America and Asia-Pacific

Not stated

English Language bias

(Chinese Science Citation DB)

English Language bias Multiple Languages

Multidisciplinary Multidisciplinary Multidisciplinary

“Publish or Perish” Cited Reference Searching Guide: http://libguides.nus.edu.sg/citedref

39

Issues & limitations of citation metrics

Different disciplines have different citation behaviours

Science: Dominated by international, peer reviewed journals published in English; sometimes involve multiple co-authors

Social Sciences & humanities: Publish more in books & conference papers which are less likely to have citation counts; usually single author

No. of references/paper published varies with field e.g. Mathematics – low, Biomedical - high

40

h-index

A scientist has index h if h of [his/her] Np papers have at least h citations each, and the other (Np - h) papers have at most h citations each

Hirsch, J.E. An index to quantify an individual's scientific research output, PNAS , vol. 102 no. 46 (2005)

41

Consider this …

Case 1:

Researcher A publishes exactly 10 papers of 10 citations each.

Researcher B publishes 10 papers with 10 citations and an additional 90 papers with 9 citations each.

Case 2:

Researcher C publishes exactly 10 papers of 10 citations each.

Researcher B publishes exactly 10 papers of 100 citations each.

Adler, R., Ewing, J. & Taylor, P. (2008). Citation Statistics: A report from the International Mathematical Union (IMU) in cooperation with the International Council of Industrial and Applied Mathematics (ICIAM) and the Institute of Mathematical Statistics (IMS).

42

From Hirsch himself ….

43

“Obviously a single number can never give more than a rough approximation to an individual’s multifaceted profile, and many other factors should be considered in combination in evaluating an individual.”

“This and the fact that there can always be exceptions to rules should be kept in mind especially in life-changing decision such as the granting or denying of tenure.”

Jorge Hirsch (Creator of the h-index)

Hirsch, J.E. An index to quantify an individual's scientific research output, PNAS , vol. 102 no. 46 (2005)

Putting Citation Counts in Perspective

http://www.jbc.org/content/280/28/e25.full

as at 06 Mar 2014

44

Putting Citation Counts in Perspective

“…it is flattering to be the ‘most cited author’ but I am afraid it does not signify great scientific accomplishment…

Although method development is usually a pretty pedestrian affair, others doing more creative work have to use methods and feel constrained to give credit for the same…”

Oliver Lowry (Devised the Lowry Protein Assay)

Lowry OH, Rosebrough NJ, Farr AL & Randall RJ. Protein measurement with the Folin phenol reagent. J. Biol. Chem. 193: 265-75, 1951.- Classic Paper. Citation Classic Commentary http://garfield.library.upenn.edu/classics1977/A1977DM02300001.pdf

45

Google

46

Citation based metrics

Impact factor

Times cited

h-index

Journal

Paper

Author/ Institution

47

Manage your publication history

Be consistent in the use of your name syntax in all your publications, e.g. James Tiberius Kirk

Distinguish yourself from other authors by choosing a name syntax that is complex enough

List your institutional affiliations over the course of your career correctly, including

Kirk, James T. Kirk, JT Kirk, J

48

Name of Institution National University of Singapore

Faculty Faculty of Science instead of Science Faculty, Sci Fac, etc

Department Department of Physics instead of Phy Dept

For Research Group Include the Faculty (if applicable) and Department

Manage your publication history

49

Manage your publication history

Check that your publications are correctly listed in databases such as Web of Science & Scopus

Promote your publications by participating in researcher identification registries

• ORCID (Open Researcher and Contributor ID), http://orcid.org/

• ResearcherID, http://www.researcherid.com/

• Google Scholar researcher profile

50

ORCID (Open Researcher and Contributor ID), http://orcid.org/

51

ResearcherID

52

Google Scholar researcher profile

53

Thank you

Let us have your feedback at

http://tinyurl.com/l6yjehe

54