![Page 1: RESEARCH EVALUATION - THE METRICS UNITED KINGDOM OCTOBER 2010](https://reader035.vdocuments.us/reader035/viewer/2022062322/5697bf701a28abf838c7d66b/html5/thumbnails/1.jpg)
RESEARCH EVALUATION - THE METRICS
UNITED KINGDOM
OCTOBER 2010
![Page 2: RESEARCH EVALUATION - THE METRICS UNITED KINGDOM OCTOBER 2010](https://reader035.vdocuments.us/reader035/viewer/2022062322/5697bf701a28abf838c7d66b/html5/thumbnails/2.jpg)
DO WE NEED MORE METRICS? 1962
![Page 3: RESEARCH EVALUATION - THE METRICS UNITED KINGDOM OCTOBER 2010](https://reader035.vdocuments.us/reader035/viewer/2022062322/5697bf701a28abf838c7d66b/html5/thumbnails/3.jpg)
METRICS ARE HERE TO STAY! 2010
![Page 4: RESEARCH EVALUATION - THE METRICS UNITED KINGDOM OCTOBER 2010](https://reader035.vdocuments.us/reader035/viewer/2022062322/5697bf701a28abf838c7d66b/html5/thumbnails/4.jpg)
CITATION METRICS
• What Types of Data are best for which Purposes?– There are no all-purpose indicators
– Start by identifying the question the results are supposed to answer, then collect data
– Clearly define
• Purpose of the evaluation
• Types of data required
• How the results will be used
![Page 5: RESEARCH EVALUATION - THE METRICS UNITED KINGDOM OCTOBER 2010](https://reader035.vdocuments.us/reader035/viewer/2022062322/5697bf701a28abf838c7d66b/html5/thumbnails/5.jpg)
WHAT DO INSTITUTIONS WANT TO FIND OUT FROM CITATION METRICS
• What is the university’s research performance?
• Are we competitive compared with our peers?
• How can the university forecast growth?
• Which are our centers of excellence?
• What is our citation ranking?
• What is the influence of our research?
• Which are our most influential papers?
• Which are our top researchers?
![Page 6: RESEARCH EVALUATION - THE METRICS UNITED KINGDOM OCTOBER 2010](https://reader035.vdocuments.us/reader035/viewer/2022062322/5697bf701a28abf838c7d66b/html5/thumbnails/6.jpg)
PRODUCTIVITY
• Publication counts – most basic bibliographic measure– Identifies areas of research intensity by field
![Page 7: RESEARCH EVALUATION - THE METRICS UNITED KINGDOM OCTOBER 2010](https://reader035.vdocuments.us/reader035/viewer/2022062322/5697bf701a28abf838c7d66b/html5/thumbnails/7.jpg)
PRODUCTIVITY – INSTITUTIONS
7
![Page 8: RESEARCH EVALUATION - THE METRICS UNITED KINGDOM OCTOBER 2010](https://reader035.vdocuments.us/reader035/viewer/2022062322/5697bf701a28abf838c7d66b/html5/thumbnails/8.jpg)
INFLUENCE – CITATIONS
8
![Page 9: RESEARCH EVALUATION - THE METRICS UNITED KINGDOM OCTOBER 2010](https://reader035.vdocuments.us/reader035/viewer/2022062322/5697bf701a28abf838c7d66b/html5/thumbnails/9.jpg)
INFLUENCE – CITATIONS
Source: Thomson Reuters U.S. and Canadian University Science Indicators
Number of citations to North American scientific papers
![Page 10: RESEARCH EVALUATION - THE METRICS UNITED KINGDOM OCTOBER 2010](https://reader035.vdocuments.us/reader035/viewer/2022062322/5697bf701a28abf838c7d66b/html5/thumbnails/10.jpg)
INDIRECT INFLUENCESECOND GENERATION CITATION COUNTS
• Sum of citation counts for articles citing target paper– Measures long term impact of a paper
– Similar to Google PageRank but uses citations, not clicks
Thomson Reuters Web of Science
![Page 11: RESEARCH EVALUATION - THE METRICS UNITED KINGDOM OCTOBER 2010](https://reader035.vdocuments.us/reader035/viewer/2022062322/5697bf701a28abf838c7d66b/html5/thumbnails/11.jpg)
EFFICIENCY H-INDEX
• Distribution based indicator– Attempt to reflect productivity and influence
– Does not control for time period and field
Thomson Reuters InCites
![Page 12: RESEARCH EVALUATION - THE METRICS UNITED KINGDOM OCTOBER 2010](https://reader035.vdocuments.us/reader035/viewer/2022062322/5697bf701a28abf838c7d66b/html5/thumbnails/12.jpg)
EFFICIENCY % PAPERS CITED – COMPUTER SCIENCE
![Page 13: RESEARCH EVALUATION - THE METRICS UNITED KINGDOM OCTOBER 2010](https://reader035.vdocuments.us/reader035/viewer/2022062322/5697bf701a28abf838c7d66b/html5/thumbnails/13.jpg)
EFFICIENCYJOURNAL IMPACT FACTOR
2009 Impact Factor
200920082007
Source paper – published in 2009
Cited reference – published in 2007 or 2008
Citations
All Previous Years
2006 2010
![Page 14: RESEARCH EVALUATION - THE METRICS UNITED KINGDOM OCTOBER 2010](https://reader035.vdocuments.us/reader035/viewer/2022062322/5697bf701a28abf838c7d66b/html5/thumbnails/14.jpg)
CALCULATING 2009 JOURNAL IMPACT FACTOR HEALTH ECONOMICS
Citations in 2009To items published in 2008 = 156
To items published in 2007 = 210
Sum = 366
Number of itemsPublished in 2008 = 97
Published in 2007 = 85
Sum = 182
366
182
= 2.011
![Page 15: RESEARCH EVALUATION - THE METRICS UNITED KINGDOM OCTOBER 2010](https://reader035.vdocuments.us/reader035/viewer/2022062322/5697bf701a28abf838c7d66b/html5/thumbnails/15.jpg)
EFFICIENCYJOURNAL IMPACT FACTOR
15
![Page 16: RESEARCH EVALUATION - THE METRICS UNITED KINGDOM OCTOBER 2010](https://reader035.vdocuments.us/reader035/viewer/2022062322/5697bf701a28abf838c7d66b/html5/thumbnails/16.jpg)
INTRODUCTION TO THE IMPACT FACTOR NATURAL FLUCTUATIONS OVER TIME
![Page 17: RESEARCH EVALUATION - THE METRICS UNITED KINGDOM OCTOBER 2010](https://reader035.vdocuments.us/reader035/viewer/2022062322/5697bf701a28abf838c7d66b/html5/thumbnails/17.jpg)
EVALUATING RESEARCHLIMITATIONS OF THE IMPACT FACTOR
![Page 18: RESEARCH EVALUATION - THE METRICS UNITED KINGDOM OCTOBER 2010](https://reader035.vdocuments.us/reader035/viewer/2022062322/5697bf701a28abf838c7d66b/html5/thumbnails/18.jpg)
EFFICIENCY5-YEAR IMPACT FACTOR
Citation rates to journals in the ‘Acoustics’ category are
maintained thus increasing the 5-year Impact Factor relative to
its 2-Year counterpart
![Page 19: RESEARCH EVALUATION - THE METRICS UNITED KINGDOM OCTOBER 2010](https://reader035.vdocuments.us/reader035/viewer/2022062322/5697bf701a28abf838c7d66b/html5/thumbnails/19.jpg)
EFFICIENCY5-YEAR IMPACT FACTOR
The opposite picture in ‘Genetics and Heredity’ demonstrates a
shorter citation lag
![Page 20: RESEARCH EVALUATION - THE METRICS UNITED KINGDOM OCTOBER 2010](https://reader035.vdocuments.us/reader035/viewer/2022062322/5697bf701a28abf838c7d66b/html5/thumbnails/20.jpg)
USING THE IMPACT FACTOREVALUATING JOURNALS
• Appropriate use– To evaluate journals
• Misuse– Evaluation of individual articles
– Evaluation of institution or researcher
![Page 21: RESEARCH EVALUATION - THE METRICS UNITED KINGDOM OCTOBER 2010](https://reader035.vdocuments.us/reader035/viewer/2022062322/5697bf701a28abf838c7d66b/html5/thumbnails/21.jpg)
21
USING THE IMPACT FACTOR MISUSE: EVALUATING INDIVIDUAL PAPERS
30% of articles in Food Policy were not cited at all
Source: Thomson Reuters Journal Citation Reports
Journal Impact Factor = 2,011
![Page 22: RESEARCH EVALUATION - THE METRICS UNITED KINGDOM OCTOBER 2010](https://reader035.vdocuments.us/reader035/viewer/2022062322/5697bf701a28abf838c7d66b/html5/thumbnails/22.jpg)
BENCHMARK YOUR PAPERS AGAINST GLOBAL AVERAGES – IS THIS A HIGHLY CITED PAPER?
22
Hematology articles from this year have been cited 18,83 times
This article is ranked in the 12,92nd percentile in its field by citations
Articles published in ‘Blood’ from 2004 have been cited 34,30 times
This paper has received 40/34,30=1,17 times the expected citations for this journal
This paper has received 40/18,83=2,12 times the expected citations for this subject category
![Page 23: RESEARCH EVALUATION - THE METRICS UNITED KINGDOM OCTOBER 2010](https://reader035.vdocuments.us/reader035/viewer/2022062322/5697bf701a28abf838c7d66b/html5/thumbnails/23.jpg)
WHICH ARE OUR CENTRES OF EXCELLENCE?
23
Dept. Medicine Dept. Mechanical Engineering
![Page 24: RESEARCH EVALUATION - THE METRICS UNITED KINGDOM OCTOBER 2010](https://reader035.vdocuments.us/reader035/viewer/2022062322/5697bf701a28abf838c7d66b/html5/thumbnails/24.jpg)
PERCENTILE INDICATORS
• Paper percentile– Determine percentage of papers at each level of citation