Measuring Journal and Research Prestige

Stephen Troth26th March 2011

s.troth@elsevier.comMedicReS International Conference on

Good Medical Research

2

Opening Questions What is the best scientific journal that you know of?

Why is that one the best?

Why can’t another journal be just as good as that one?

3

Objectives Why is there a need to discern quality among different

journals? What metrics are used to compare journals? Which journal is appropriate for me to submit my

research to?

Why is there a need to discern quality among different journals?

5

Growth of peer-reviewed journals

“This is truly the decade of the journal and one should seek to limit their number rather than

to increase them, since there can be too many periodicals.”

(1789)

Neues medicinisches Wochenblatt fur Aerzte

6

Growth of scholarly journals

0

5000

10000

15000

20000

25000

<1900 1900s 1910s 1920s 1930s 1940s 1950s 1960s 1970s 1980s 1990s >2000Decade

Nu

mb

er

of

Ac

tiv

e,

Pe

er-

Re

vie

we

d J

ou

rna

ls

Source:

~3% per annum• Number of journals and disciplines is increasing

• The need to discern journal quality becomes that much more important

7

Journal Competition

• Journals must vigorously compete with each other for the best papers and the best authors

• The concept of journal prestige originates from this competition

For example: In the category of “Analytical Chemistry”

Analytical Chemistry

Analytica Chimica Acta

Analytical Biochemistry

Analytical and Bioanalytical Chemistry

Journal of Electroanalytical Chemistry

Analyst

Electroanalysis

Analytical Sciences

Journal of Analytical Chemistry

Current Analytical Chemistry

Reviews in Analytical Chemistry

And >50 others!

How can you tell which of these are high quality journals?

What metrics are used to compare journals?

9

Overview of Journal Metrics

Impact Factor H-index SCImago Journal Rank Usage Others

• Journal citation data and bibliometrics can be used to measure the impact or influence of articles, authors, and journals

10

Impact Factor

• Citation index of Science, Social Science, Arts & Humanities journals

• Impact Factors of Science and Social Science journals

Impact Factor is the most well-known citation metric

11

Impact Factor Definition & Calculation

IF is published 6 months after the end of the year it relates to (i.e. 2006 data published in mid-2007)

Definition: A ratio between citations and recent citable items published in a journal

(i.e. the average number of citations received per published article)

12

The Impact Factor

13

Citations to non-source items (editorials, letters, newsitems, book reviews, abstracts, etc) may inflate the IF

The Impact Factor anomaly

To all items (regardless of type)

Of source items (“articles” and “reviews”)

14

Influences on the IF: Article TypeC

itatio

ns

Articles

Reviews

Years after publication

Notes

4 8 12 160 2 6 10 14 18 20

Impact Factor

15

0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5

Mathematics & Computer Sciences

Social Sciences

Materials Science & Engineering

Biological Sciences

Environmental Sciences

Earth Sciences

Chemistry & Chemical Engineering

Physics

Pharmacology & Toxicology

Clinical Medicine

Neuroscience

Fundamental Life Sciences

Mean Impact Factor (1998)

Influences on the IF: Subject Area

CHARACTERIZATION & TESTING

0.000 0.500 1.000 1.500 2.000 2.500 3.000

PAPER & WOOD

TEXTILES

CERAMICS

COMPOSITES

COATINGS & FILMS

MULTIDISCIPLINARY

POLYMER SCIENCE

NANOSCI & NANOTECH

BIOMATERIALS

Aggregate 2006 IF

Materials Science disciplines

Impact Factors carry little meaning unless they are compared

within the same subject area and

discipline

16

Influences on the IF: Subject Area

17

2008 IF

18

Impact Factor Use and Abuse

• Used for library collection development

• Open to manipulation by authors, reviewers,editors and publishers

• Used to compare journals of different types • Used to compare journals in different fields • Used to derive a ‘personal IF’

• Used as a lone proxy for journal ‘quality’

19

Impact Factor doubts

June 5, 2006

October 14, 2005

20

Elsevier’s philosophy on the IF

“Elsevier uses the Impact Factor as one of a number of performance indicators for journals. It acknowledges the many caveats associated with its use and strives to share best practice with its authors, editors, readers and other stakeholders in scholarly communication. Elsevier seeks clarity and openness in all communications relating to the IF and does not condone the practice of manipulation of the IF for its own sake.”

21

Other IF-related metrics Cited Half-life

The cited half-life for the journal is the median age of its articles cited in the current JCR year

Immediacy Index The immediacy index is the average number of times an

article is cited in the year that it is published Influence

The influence indicates the share of citations that an individual title or publisher has within a given subject category or subject group

22

h-index

• Proposed by physicist Jorge Hirsch in 2005

• Rates individuals or journals based on career publications

• Incorporates both quantity (no. of publications) and quality (no. of citations)

23

Calculating the h-index

If you list a scientist’s papers in descending order of the number of citations received to date, his/ her h-index is 8 if 8 papers have each received 8 or more citations

24

h-index

25

h-index for journals

26

SCImago Journal Rank

Produced by experts in Spain Data sourced from Scopus and incorporated in it A ratio of citations in current year to articles published in the

previous 3 years Citations are weighted by the SJR of the

citing journal (like Google weights links to webpages)

27

SCImago Journal Rank

Note European decimal notation!

New metrics : Popularity

28

29

Free (eigenfactor.org); also now part of the JCRSimilar to Impact Factor, but considers 5 yearsSelf-citations excludedCitations weighted by EF of the citing journal

Eigenfactor Article Influence

30

Usage Usage is a new concept for measuring journal value and

impact Typically defined as when a full-text article is

downloaded or viewed COUNTER is attempting to standardize usage reporting

and develop a “Usage Factor” metric Libraries already use usage statistics heavily to evaluate

their collections and spending Authors also interested to see how much their works are

used

31

Summary of Different Metrics H-index, SJR, and usage are gaining weight as more

users include them as evaluative tools

Other metrics are also being studied and developed not mentioned here: Eigenfactor

But IF is still the industry-standard and first metric that is used to assess journal importance

Which journal is appropriate for me to submit my research to?

33

Journal Selection

How would you select among the journals of “Analytical Chemistry”?

Analytical Chemistry

Analytica Chimica Acta

Analytical Biochemistry

Analytical and Bioanalytical Chemistry

Journal of Electroanalytical Chemistry

Analyst

Electroanalysis

Analytical Sciences

Journal of Analytical Chemistry

Current Analytical Chemistry

Reviews in Analytical Chemistry

And >50 others!

34

Journal Selection

Selection of a journal will depend on many factors in addition to journal metrics

The aims and scope of the journal The type of manuscript you have written (review, letter, articles) The specific subject area The significance of your work The prestige/quality of the journal The respect of the editors in the field The editorial and production speed of the journal The community and audience associated with the journal The coverage and distribution (regional, international)

35

Summary

Why is there a need to discern quality among different journals? Increasing number of journals and disciplines

What metrics are used compare journals? Impact Factor H-index SCImago Journal Rank SNIP/Eigenfactor Usage

Which journal is appropriate for me to submit my research to? Consider the significance and scope of your work. Ask professors in your field

what journal would be appropriate for the area and level of research you have conducted

Consider the aims, scope, subject area, prestige, editors, editorial and production speed, community/audience, and coverage of a journal