impact factor and citation metrics: what do they really mean? session 1206 room 203a january 30,...
TRANSCRIPT
Impact Factor and Citation Impact Factor and Citation Metrics: What do they Really Metrics: What do they Really Mean?Mean?
Session 1206 Room 203AJanuary 30, 2009
Debbie Chaves, Wilfrid Laurier UniversityBrian Cameron, Ryerson University
OutlineOutline
• Measurement of the Business of Science
• Background on various methods
• Where things can go wrong
• What that means for librarians
Why Measure?Why Measure?
• Security of knowledge
• Predictability
• Comparison analysis
• Subject and domain dependent
• Quantifiability fallacy
Historical Background on Historical Background on Impact FactorImpact Factor
Eugene Garfield, "Citation Indexes for Science: A New Dimension in Documentation through Association of Ideas," Science 122, 3159 (July 15, 1955): 108
What is an Impact FactorWhat is an Impact Factor
ratio of articles published to articles cited during a rolling two-year window
A/BA = citations in 2008 : articles cited in
2006-7B = citable items published in 2006-7
year 2008 citations to 2006 + 2007 articlesarticles published in 2006 + 2007
Your (real) Impact FactorYour (real) Impact Factor
Use of Impact FactorsUse of Impact FactorsCollection developmentChoosing journals for publicationJournal assessment/marketing by
publishersEvaluation of scholarly research
& individual performance, for purposes of tenure and promotion, and funding
Evaluation of departments, institutions, and nations
Problems with IFProblems with IFTwo-year window# of journals published in
disciplineISI coverageLanguagePublication typeNot representativeJournal size
Other IssuesOther Issues
Citation errorsCitation clubsCitation bartering
ManipulationManipulation
Review articlesCase studiesEditorial interference
hh-index-index
Hirsch, J. E. (2005). "An index to quantify an individual's scientific research output". PNAS 102 (46): 16569–16572.
hh-index-indexh-index, developed by Jorge
Hirsch
A scientist has index h if h of [their] Np papers have at least h citations each, and the other (Np - h) papers have at most h citations each.
Criticisms of the Criticisms of the hh-index-indexComparisonsAgeInsensitiveContextLimitations of citation databases# of authors
From From hh to to ggGiven a set of articles ranked in
decreasing order of the number of citations that they received, the g-index is the (unique) largest number such that the top g articles received on average at least g citations.
What do the numbers really mean?What do the numbers really mean?
Author inaccuracies
ISI calculation inaccuracies
What does an H-index of 1 really mean?
Citation Game-Playing – How to Citation Game-Playing – How to make it Work for YOU!make it Work for YOU!• Become famous so everyone gives you
authorship on their paper• HEP in ArXve submission timing• Ride the bias – multiple authors: choose the
middle• Folia Phoniatrica et Logopaedica: 1 articles
cited all the papers published in the last 2 years for IF=1.439 compared to 0.655
• Find and ride the trend – Martin Fleischmann was once the most highly cited chemist in Britain
Nature 451 2008 pg 766-767
Search Web of Science for Wilfrid LaurierSearch Web of Science for Wilfrid Laurier
Wilfrid Laurier as Author in 2007?Wilfrid Laurier as Author in 2007?
Wilfrid Laurier as Author in 2007?Wilfrid Laurier as Author in 2007?
Ann Arbor as Author (MUCH worse in Ann Arbor as Author (MUCH worse in Google Scholar)Google Scholar)
Ann Arbor as Author in 2007?Ann Arbor as Author in 2007?
William H. Gates = Bill GatesWilliam H. Gates = Bill Gates
Researcher ID = A-1930-2009Researcher ID = A-1930-2009
Researcher ID Publication ListResearcher ID Publication List
Anyone can write a letterAnyone can write a letter
NATURE Vol 53 June 2008 pg. 718
Chemistry Department Scopus H-IndexChemistry Department Scopus H-IndexCreated Feb 9, 2008 with the faculty at that time. Dependent on information included in SCOPUS.
09-Feb-08
Position h-index patents total number of papers
first-year publication
last-year publication
Publications/year
president 5 0 13 1984 2008 0.54
dean science 15 0 51 1988 2008 2.55
assistant 5 0 14 1999 2007 1.75
assistant 5 0 6 1998 2007 0.67
assistant 15 2 48 1998 2007 5.33
assistant 6 0 13 1998 2007 1.44
assistant 7 0 12 1998 2007 1.33
assistant 9 0 14 2000 2005 2.80
CAS 3 0 4 1999 2002 1.33
CAS 10 33 25 1979 2007 0.89
professor 8 0 36 1967 2007 0.90
associate 7 0 17 1992 2005 1.31
CAS 0 0 2 2007 2008 2.00
CAS 17 0 24 1984 2001 1.41
Average 8 2.50 19.93 1992.21 2006.14 1.73
ISI Accuracy?ISI Accuracy?
Sciencewatch.com Sciencewatch.com
Institutional Metrics Institutional Metrics http://webometrics.infohttp://webometrics.info
What does this mean for What does this mean for Librarians?Librarians?Must be familiar and knowledgeable
about the current calculations of Science Worthiness
International students are aware
Administrations and granting agencies will ask for these statistics but may not be aware of their limitations
Avoid the citation culture
Take Home MessageTake Home Message
Avoid Disregard Syndrome Practice Citation Vigilance