www.gu.se bo jarneving bibliometrifunktionen gub 2011 evaluative bibliometrics: citation based...
Post on 19-Dec-2015
217 views
TRANSCRIPT
www.gu.se
Bo Jarneving
Bibliometrifunktionen
GUB
2011
Evaluative Bibliometrics: Citation based Research Indicators
www.gu.se
Variables for citation based performance indicators
1. Document type
articles, reviews, letters, notes
2. Observation period
Citation interval
3. Self-citations
With or without
4. Subject classification
Generally based on ISI ”Journal Subject Categories”
Classification of papers published in multidisciplinary journals
5. Counting method decisions
How to count co-authored papers?
How to count when there are multiple author affiliations?
How to count when there are multiple subject categories?
www.gu.se
Observation period
• Observation period– 2004-2009– Variable or fixed citation interval (window)
PY 2004 2005 2006 2007 2008 2009
Number of years
6 5 4 3 2 1
Interval 2009-2004
2009-2005
2009-2006
2009-2007
2009-2008
2009-2009
www.gu.se
What is a self citation?
Schubert A, Glanzel W, Braun T (1989) Scientometric Datafiles - A Comprehensive Set of Indicators 0n 2649 Journals and 96 Countries In All Major Science Fields and Subfields 1981-1985.
Glanzel W, Thijs B, Schubert A, Debackere K (2009) Subfield-specific normalized relative indicators and a new generation of relational charts: Methodological foundations illustrated on the assessment of institutional research performance.
www.gu.se
Journal Subject Category
• A proxy for subject field• Sometimes multiple assignments of journals to categories• Based on citation data and ”manual methods”• A total of 173 categories in the science volume of Journal Citation Report (JCR)• Alphabetical order – no hierarchy
ISI Journal Subject CategoryACOUSTICSAGRICULTURAL ECONOMICS & POLICYAGRICULTURAL ENGINEERINGAGRICULTURE, DAIRY & ANIMAL SCIENCE···VETERINARY SCIENCESVIROLOGYWATER RESOURCESZOOLOGY
www.gu.se
Classification of multidisciplinary papers
• Reversing pathological neural activity using targeted plasticity• NATURE, 2011, V470, N7332, P101-U114
Frequency Abbreviated journal title Journal Subject Category Journal Subject Category1 ARCH OTO-RHINO-LARYN OTORHINOLARYNGOLOGY1 AUDIOLOGY OTORHINOLARYNGOLOGY1 BEHAV NEUROSCI BEHAVIORAL SCIENCES NEUROSCIENCES
8 HEARING RES NEUROSCIENCES OTORHINOLARYNGOLOGY1 J ACOUST SOC AM ACOUSTICS1 J CLIN NEUROPHYSIOL CLINICAL NEUROLOGY NEUROSCIENCES 2 J NEUROPHYSIOL NEUROSCIENCES PHYSIOLOGY1 J NEUROSCI NEUROSCIENCES2 J NEUROSCI RES NEUROSCIENCES PHYSIOLOGY1 J PHARMACOL EXP THER PHARMACOLOGY & PHARMACY 1 JARO OTORHINOLARYNGOLOGY1 LANCET MEDICINE, GENERAL & INTERNAL 1 LARYNGOSCOPE MEDICINE, RESEARCH & EXPERIMENTAL OTORHINOLARYNGOLOGY1 NAT NEUROSCI NEUROSCIENCES2 NATURE MULTIDISCIPLINARY1 NEUROSCIENCE NEUROSCIENCES2 P NATL ACAD SCI USA MULTIDISCIPLINARY3 PROG BRAIN RES NEUROSCIENCES1 SCIENCE MULTIDISCIPLINARY1 TRENDS NEUROSCI NEUROSCIENCES
www.gu.se
Counting method: fractional counts or whole counts
• Whole counts– Every author, institution, department and country is
assigned one full paper• Fractional counts
– A paper is divided up over its authors– If a paper has 3 authors, every author is assigned 1/3
of the paper. – If 2 of these authors are associated with institution A
and 1 with institution B, A recieves 2/3 of the paper and B 1/3.
www.gu.se
Counting methods: fractional counts or whole counts
When more than one institutional affilation of an author:• Whole counts
– One author associated with two institutions – each insitution recieves one full paper
• Fractional counts– One author associated with two institutions – each
insitution recieves one half of a paper
www.gu.se
Counting methods: fractional counting of papersExample 1: 1 paper, 1 category, 2 authors, 3 institutions (A-C)
1c
1/2
1/2
1/4
1/4
14+14+12
A gets
B g C g
1/2
www.gu.se
Counting methods: fractional counting of papersExemple 2: 1 paper, 2 categories (1, 2), 2 authors, 3 institutions (A-C)
1c
1/2
1/2
1/4
1/4
1/4
1/4
1/8
1/8
1/8
1/8
14+18+18+14+18+18
Category 1
A gets
B gets
C gets
Category2
A gets
B gets
C gets
1/4
1/4
www.gu.se
Counting methods: fractional counting of citations.Example 3: 1 paper, 1 category, 2 authors, 3 institutions (A-C).The paper has recieved 10 citations (c).
1c
1/2
½ x 10
¼ x 10
¼ x 10
14∙10+
14∙10+
12∙10
A recieves 2.5 cB recieves 2.5 cC recieves 5 c
1/2
www.gu.se
Counting methods: fractional counting of citationsExample 4: 1 paper, 2 categories (1, 2), 2 authors, 3 institutions (A-C). The paper has recieved 10 citations (c)
1c
½
½
¼ x 10
¼
¼ x 10
¼
1/8 x 10
1/8 x 10
1/8 x 10
1/8 x 10
Category 1
A recieves
B recieves
C recieves
Category 2
A recieves
B recieves
C recieves
¼
¼
www.gu.se
Counting methods: whole counts of citationsExample 5: 1 paper, 2 categories (1, 2), 2 authors, 3 institutions (A-C). The paper has recieved 10 citations
1c
1 x 10
1 x 10
1 x 10
1 x 10
1 x 10
1 x 10
1 x 10
1 x 10
1 x 10
Kategori 1
A r
B rC r
Kategori 2
A r
B rC r
www.gu.se
Normalization of citation counts
• Observed value: institutional (departmental) citation counts• Expected value: approximated world averages
– based on Web of Science citation data
www.gu.se
Normalization of citation counts
The expected value (reference value)• Define groups on basis of the institutional publication pattern. Group
by:– Publication date (year)– Journal Subject Category– Document type
• For each such group, compute the Field Citation Score (FCS)– Based on citations between papers indexed in WoS
» Approximations of world-citation-averages for each group
www.gu.se
Normalization of citation counts: defining groups• Example 1: The department (group) publishes in journals assigned to
one journal subject category only (neurosciences).– Brain Research 4 articles
2004– Brain Research 1 letter
2004– Journal of Neuroscience 3 articles 2002– Journal of Neuroscience 1 review 2003
• All authors are from one department
• Gather all articles published in journals assigned to neurosciences and published 2004
• Gather all letters published in journals assigned to neurosciences and published 2004
• Gather all articles published in journals assigned to neurosciences and published 2002
• Gather all reviews published in journals assigned to neurosciences and published 2003
www.gu.se
Normalization of citation counts: reference values
For each group (1- 4) compute the world-average number of citations (Field Citation Score [FCS]):
• Neurosciences-articles 2004 FCS = 4.2 • Neurosciences-letters 2004 FCS = 1.0 • Neurosciences-articles 2002 FCS = 3.8 • Neurosciences-reviews 2003 FCS = 8.5
www.gu.se
Normalization of citation counts: the department’s counts
Paper DT PY # citations1 article 2004 72 article 2004 43 article 2004 54 article 2004 55 letter 2004 16 review 2003 77 article 2002 78 article 2002 69 article 2002 1
www.gu.se
Normalization of citation counts: comparing
Paper DT PY # citations FCS1 article 2004 7 4.22 article 2004 4 4.23 article 2004 5 4.24 article 2004 5 4.25 letter 2004 1 1.06 review 2003 7 8.87 article 2002 7 3.88 article 2002 6 3.89 article 2002 1 3.8
Pic 20
www.gu.se
Normalization of citation counts: averages
• CPP = Citation Per Publication (department)• FCSm = Mean Field Citation Score• CPP/FCSm = the Crown Indicator
• Compute CPP: (3 x 7 + 1 x 6 + 2 x 5 + 1 x 4 + 2 x 1) / (3+1+2+1+2) = 4.78
• Compute FCSm: (4 x 4.2 + 1 x 1.0 + 1 x 8.8 + 3 x 3.8) / (4 + 1 + 1 + 3) = 4.22
• Compute CPP/FCSm: 4.78 / 4.22 = 1.13
www.gu.se
Normalization of citation counts: defining groups
• Example 2: The department (group) publishes in journals assigned to more than one journal subject category.
– Brain Research 4 articles2004
– Brain Research 1 letter2004
– Journal of Neuroscience 3 articles 2002– Neuroimage 1
article 2003
Four classifications:• Gather all articles published in journals assigned to neurosciences and
published 2003• Gather all articles published in journals assigned to neuroimaging and
published 2003• Gather all articles published in journals assigned to Radiology and
published 2003• Gather all articles published in journals assigned to Nuclear Medicine
& Medical Imaging and published 2003
www.gu.se
Normalization of citation counts: reference values
• For each group (1- 7) compute the world-average number of citations (Field Citation Score [FCS]):
– Neurosciences-articles 2004 FCS = 4.2
– Neurosciences-letters 2004 FCS = 1.0
– Neurosciences-articles 2002 FCS = 3.8
– Neurosciences-articles 2003 FCS = 5.5– Neuroimaging-articles 2003 FCS
= 3.2– Radiology-articles 2003 FCS
= 2.8– Nuclear Medicine &
Medical Imaging articles 2003 FCS = 4.6
www.gu.se
Normalization of citation counts: comparing
Paper DT PY # citations Fields FCS1 article 2004 7 Neurosciences 4.2
2 article 2004 4 Neurosciences 4.2
3 article 2004 5 Neurosciences 4.2
4 article 2004 5 Neurosciences 4.2
5 letter 2004 1 Neurosciences 1.0
6 article 2002 7 Neurosciences 3.8
7 article 2002 6 Neurosciences 3.8
8 article 2002 1 Neurosciences 3.8
9 article 2003 7 / 4 Neurosciences 5.5 / 4
9 article 2003 7 / 4 Neuroimaging 3.2 / 4
9 article 2003 7 / 4 Radiology 2.8 / 4
9 article 2003 7 / 4 Nuclear Medicine & Medical Imaging
4.6 / 4
www.gu.se
Normalization of citation counts: averages
• CPP = Citation Per Publication (department)• FCSm = Mean Field Citation Score• CPP/FCSm = the Crown Indicator
• Compute CPP: (2 x 7 + 1 x 6 + 2 x 5 + 1 x 4 + 2 x 1 + ¼ x 7 + ¼ x 7 + ¼ x 7+ ¼ x 7) / (2+1+2+1+2+¼ + ¼ + ¼ + ¼) = 4.78
• Compute FCSm: (4 x 4.2 + 1 x 1.0 + 3 x 3.8 + ¼ x 5.5 + ¼ x 3.2 + ¼ x 2.8 + ¼ x 4.6) / (4 + 1 + 3 + ¼ + ¼ + ¼ + ¼) = 3.69
• Compute CPP/FCSm: 4.78 / 3.69 = 1.30
www.gu.se
Normalization of citation counts: paper by paper • Normalization on paper level: 10.48 / 9 = 1.16• Ratio of averages vs. average of ratios
Paper DT PY # citations FCS # citations /FCS
1 article 2004 7 4.2 1.6666672 article 2004 4 4.2 0.9523813 article 2004 5 4.2 1.1904764 article 2004 5 4.2 1.1904765 letter 2004 1 1.0 16 review 2003 7 8.8 0.7954557 article 2002 7 3.8 1.8421058 article 2002 6 3.8 1.5789479 article 2002 1 3.8 0.263158
Sum 10.47967
www.gu.se
Total performance indicator
• The average impact times the total number of papers:
• Where ci is the citation score for paper i, ei the expected number of citations for paper i and N the total number of papers for a department
N
i
N
i i
i
e
cN
N
ec
1 1
11
www.gu.se
A real-life example
• TI- Engineering of a novel Saccharomyces cerevisiae wine strain with a respiratory phenotype at high external glucose concentrations
• JN - Applied and Environmental Microbiology• SC- Biotechnology & Applied Microbiology; Microbiology• DT- article• PY- 2005• AU- Cecilia Henricsson (Institutionen för kemi- och bioteknik, Chalmers)
Maria de Jesus-Ferreira (-)
Kristina Hedfalk (Institutionen för kemi)
Karin Elbing (-)
Christer Larsson (Institutionen för kemi- och bioteknik, Molekylär bioteknik, Chalmers)
Joakim Norbeck (Institutionen för kemi- och bioteknik, Molekylär bioteknik, Chalmers)
Roslyn Bill (-)
Stefan Hohmann (Institutionen för cell- och molekylärbiologi)
Lena Gustafsson (Institutionen för kemi- och bioteknik, Molekylär bioteknik, Chalmers)
www.gu.se
A real-life example
Bibliometric data:• FCS or = field citation score (reference value), self-citations excluded• C = observed citations, self-citations excluded• Observation period 2004-2009
– A citation window of 5 years• Fractionalized author counts• Fractionalized subject category counts
– Journal subject categories: » Biotechnology & Applied Microbiology » Microbiology
www.gu.se
A real-life example
Computing the reference value:– Find all papers published in journals assigned to Biotechnology & Applied
Microbiology and/or Microbiology during 2005-2009.– Count all citation to these papers
– Fractionalize in case of multiple journal subject category assignments
– Compute FCS or :– for Biotechnology & Applied Microbiology , and – for Microbiology.– N = 247 (journals)
www.gu.se
A real life example
C FCS C/FCS PYCitation Interval # authors # departments Count
8 10.709 0.747 2005 5 9 1 0.083
www.gu.se
How to affect the average field-normalized citation score?
• Publish in journals covered by Thomsen ISI (SCOPUS)• Publish in high esteem journals• Use a standard departmental name• Use a standard institutional name