eurocris platform meeting - vienna 2-3 october 1998 cris as a source for tracking science...

EuroCRIS Platform Meeting - Vienna 2-3 October 1998

CRIS as a source for tracking science publication patterns

Fulvio Naldi - Carlo Di Mento

Italian National Research Council

Naldi F., Di Mento C. - CRIS as a source ... - EuroCRIS Meeting - Vienna 2-3 Oct. 1998

CRIS: Databases on

Scientific & Technological Outputs

Papers published on scientific journals Books or chapters of books Contribution to conferences Grey literature Other kinds of scientific/technological outputs


Main purposes

• A way to describe current activity and available skill and knowledge

• Scientific evaluation of people and institutes

• Bibliographic research when traditional bibliographic databases fail


Scientific evaluation

Outputs are usually classified according some rough indicators:

• National/International journals (distribution, editorial board, …)

• Type of paper: scientific article, review, letter, note, …

• National/International Conference (participants, speakers, scientific committee, …)

• Availability of the Proceedings

• Type of contribution: invited, full / short communication, …

• Number of co-authors

• ……


Other "cold" indicators

• Diffusion• Inclusion in SCI • Impact Factor (IF) of the Journals

• Number of Citations, Immediacy Index, Self-Citing, Self-Cited, … ...

Mainly based on the Science Citation Index® (SCI) of the Institute for Scientific Information (ISI) - Philadelphia - US


Inclusion in SCI

50.000 scientific journals

10.000 judged significant

4.779 included in SCI 1996 (631.137items, 13.348.007 citations)


Inclusion in SCI

Reliable but no selective (poor significance)

50.000 scientific journals

10.000 judged significant

4.779 included in SCI 1996 (631.137items, 13.348.007 citations)


SCI includes:

• CELL • NATURE • SCIENCE

Inclusion in SCI


SCI includes:

• CELL • NATURE • SCIENCE

and also:• JOURNAL OF THE AMERICAN HELICOPTER SOCIETY• ORIENTAL INSECTS • USDA FOREST SERVICE PACIFIC NORTHWEST RESEARCH

STATION RESEARCH PAPER • USDA FOREST SERVICE SOUTHERN FOREST EXPERIMENT

STATION RESEARCH PAPER

Inclusion in SCI


Impact Factor (IF)

- Geographical and disciplinary coverage - Technical applicability- Political opportunity

Widely adopted with cautions and several criticisms on:

total number of citations in the last 2 years IF =

number of articles of last year


Impact FactorGeographical Coverage

% of the Journals are published in

40 US

20 UK10 Holland

There is a very poor coverage of the scientific literature in the under-developing countries (cfr. The European Report on Science and Technology Indicators - EC 1994)


Impact FactorDisciplinary Coverage

Discipline Journals Items

Medicine 25.6 % 25.9 %

Biology 17.0 % 16.7 %

Physics 6.5 % 13.6 %

Engineering 13.6 % 12.0 %

Chemistry 6.9 % 11.2 %


Impact FactorTechnical applicability

• IF values depend on the Journals selected by ISI

• IF values of journals in different disciplines are not directly comparable

• Journals with low IF values, mainly if published in US, do not have international relevance

• High relevance, very specialised (low diffusion) journals may have low IF values

• Multidisciplinary journals rank first in less populated disciplines


Impact FactorA possible approach

1. Group the journals of each scientific field (Mathematics, Physics, Chemistry, Biology, Medicine, etc.) and/or of each discipline (175 in ISI schema) in a small number (e.g. 4) of "Relevance Groups" according their IF value

2. Count the number of items in each "Relevance Group" and use these numbers for disciplinary and geographical comparisons


Impact FactorPolitical opportunity

• New fields and borderline researches may be penalised

• Researchers are less motivated to submit their papers to new journals

• The selection of the journals:- influences IF values- has important economical implication


Conclusions - 1

• Publication-based evaluation of individual scientists and of research groups (Labs, Institutes, University, Councils, etc.) requires a way to rank scientific journals according their relevance in the scientific community.


Conclusions - 2

• There are some doubts on the general applicability of Impact Factor for any evaluation purpose.

• Emerging media (Web) are not yet included in SCI


Proposal

Pre-feasibility study: Compare our national CRIS on scientific output with SCI indicators


Proposal


If doubts are confirmed ...


Proposal


If doubts are confirmed ...

Set up a group of experts (CRIS, Scientometry, Bibliometry, Science Policy, etc.) to implement and maintain a "Michelin Guide" of the Scientific Journals according both “cold”, heuristic and empirical indicators.

eurocris platform meeting - vienna 2-3 october 1998 cris as a source for tracking science...

Documents