usage factor vs. impact factor: a case study with

Usage Factor vs. Impact Factor: A case study withBioOne Journals

Jason Price, PhD Head of Collections and Acquisitions, Claremont University Consortium

eJournal Package Analyst, Statewide Calif. Electronic Library Consortium

Researcher opinion on usefulness metric: Downloads as good (or better) than citations

CIBER: New journal publishing models: an international survey of senior researchers; Ian Rowlands and Dave Nicholas, 2005

Researcher opinion on usefulness metric: Downloads better than citations

CIBER: New journal publishing models: an international survey of senior researchers; Ian Rowlands and Dave Nicholas, 2005

Downloads per researcher have increased more rapidly

than articles per researcher

Source: Elsevier Customer Research, Scopus

Usage Factor - Background

• Idea: Leverage COUNTER statistics to create a universal usage-based metric of relative journal quality

• Information gathering process: – Oct 2006: In-depth interviews

• ~10 @ authors, librarians & publishers– March 2007: Web survey

• 150 librarians; 1500 Authors– June 2007: Final Report: Stage 1 (Shepherd)

• http://www.uksg.org/usagefactors/final

Usage Factor Formula

Usage Factor =

Total usage over period ‘x’ of articles published during period ‘y’÷

Total articles published during period ‘y’

Usage Factor - Advantages

• Can be applied to all online journals• Measures practitioner & student use • Easily calculated by publishers• Additional pre-purchase journal eval. metric• Not discipline dependent?• Data availability: more rapid? less

proprietary?

Usage factor - Concerns

• More potential for abuse• Difficult to agree on length of publication

period to assess • Many publishers unwilling to provide usage

data to 3rd parties – will require audit process• Many journals published on multiple

platforms, making download # harder to count

Usage Factor Pilot Study – Bioone Journals

Title-level Data (n = 134)• Impact Factor (n = 94)• Total Times Cited in ISI (2007)• 2008 Global Downloads*

* of abstracts & articles from the BioOne Platform- adjusted for automated indexing hits

• # of articles in Bioone (≈ Publication period)• 2008 Claremont and Oberlin Group

downloads

Questions

1. How do impact and usage factor relate to total citations & downloads?

2. (How) Do impact and usage factor rankings differ?

3. How does usage factor compare:1. between ISI-ranked & unranked? 2. Between .1 and .2 ?

4. Does usage factor differ from local usage?[How could usage factor be used to promote or

select Bioone journals?]

Strong?

Impact Factor Usage Factor

2007ISI

CitationsStrong? Weak?

2008 Global

DownloadsWeak? Strong?

2007

Impact and usage factor ranks are not related

0

10

20

30

40

50

60

70

80

90

100

(low

er)-

->RA

NK-

->(h

ighe

r)

(lower)-->RANK-->(higher)

IF_rankUF_rank_(ISIJs)

num art?

Higher impact journals with lower usage factor

Higher usage journals with lower impact factor

0

10

20

30

40

50

60

70

80

90

100

(low

er)-

->RA

NK-

->(h

ighe

r)

(lower)-->RANK-->(higher)

IF_rankUF_rank_(ISIJs)

num art?

0

20

40

60

80

100

120

140

(low

er)-

->RA

NK-

->(h

ighe

r)

(lower)-->RANK-->(higher)

IF_rankUF_rank(All)num art?

UF_rank(All)--not ISI rated

Bioone Profile

Count of Coll.Coll. ISI Ranked? Total

1Yes 72No 14

1 Total 862Yes 23No 25

2 Total 48Grand Total 134

Largely overlapping usage factor distributions

Local usage unrelated to usage factor

Back to Survey Data: Librarian results: new journals

Ranking without Usage Factor Ranking with Usage Factor

1. Feedback from library users 1. Feedback from library users

2. Price 2. Usage Factor

3. Reputation/status of publisher 3.Price

4. Impact Factor 4. Impact Factor

5. Reputation/status of publisher

Survey Data: Librarian results: existing journals

Ranking without Usage Factor

Ranking with Usage Factor

1. Feedback from library users 1. Feedback from library users

2. Usage 2. Usage

3. Price 3. Usage Factor

4. Cost per Download 4. Price

5. Impact Factor 5. Cost per Download

6. Reputation/status of publisher

6. Impact Factor

7. Reputation/status of Publisher

“I would view Usage Factor as an aid for collection rather than cancellation decisions. Usage per se is a more suitable tool for us when considering cancellation.”

Usage Factor - Stage 2

• Project Steering Group established – 6 publishers – 1 aggregator – 1 hosting service

• Usage logs converted to uniform standard report format for analysis by expert third party

• RFP for third party selection process

Stage 2 – Deliverables A report (early 2009) which will:

Outline the various metrics assessed

Recommend which of them prove consistent and robust enough to be adopted for scaled up onward monitoring

Suggest any ways in which data providers might amend the way they capture, structure, label, and maintain their data which would make the measurement of Usage Factors easier and more reliable.

Propose ways to audit Usage Factor for accuracy

Summary: Usage Factor Implications

• Can break down the ISI ‘divide’• More favorable evaluation of

practitioner/student used journals• Can be used to show value of less well known

journals (particularly Bioone.2)• Could be used to identify candidates for

expansion• What others?

Comments

• Doesn’t address journals w/ narrow coverage / small audience

• Basic vs. Applied Science journals might be differentially affected

• Usage factor might correlate better with Impact factor if a dif’t publishing period was examined

• Need a better estimate of downloads from all platforms to verify results

Sources/FMI

• Usage Factor Website• http://www.uksg.org/usagefactors

• Rowland & Nicholas. 2005 CIBER: New journal publishing models: an international survey of senior researchers• http://www.ucl.ac.uk/ciber/pa_stm_final_report.pdf

• McDonald. 2008. Usage Factor: Final Rpt. & Next Steps• http://www.nfais.org/Usage08McDonald.ppt

----------------------• Peter Shepherd AKA Tom Angus Novel Let Them Eat Cake

A hilarious comedy caricature of the excesses of the publishing industry‘whose characters are 2nd to none in their wild & offbeat excentricity’Read more & Order http://tinyurl.com/tomangus