evidence-based librarianship: utilizing data from all available sources to make judicious print...

& Technical Services 29 (2005) 169–179

Evidence-based librarianship: Utilizing data from all

available sources to make judicious print

cancellation decisions

John GallagherT, Kathleen Bauer, Daniel M. Dollar

Yale University, Cushing/Whitney Medical Library, 333 Cedar Street, P.O. Box 208014,

New Haven, CT 06520-8014, USA

Available online 13 June 2005

Abstract

As the cost of periodicals continues to rise, libraries must consider the value of titles currently

acquired or subscribed to. At Yale University’s Cushing/Whitney Medical Library (CWML), staff

employed an evidence-based librarianship (EBL) approach that combined use data from several

disparate sources to make the best decisions regarding the cancellation of specific journals’ print

format. These best-evidence sources include the following: a 3-month usage study of 1249 current

unbound print journals; statistics about 3465 MEDLINE-indexed electronic journals accessed via

ExLibris’ linking tool SFX; statistics from the Association of Research Libraries (ARL) and the

American Association of Health Sciences Libraries; and various traditional library statistics.

D 2005 Published by Elsevier Inc.

1. Introduction

The Cushing/Whitney Medical Library (CWML) serves the information needs of

members of Yale University and the Yale-New Haven Medical Center. Like other

1464-9055/$ -

doi:10.1016/j.l

T Correspond

E-mail add

Daniel.Dollar@

Library Collections, Acquisitions,

see front matter D 2005 Published by Elsevier Inc.

cats.2005.04.004

ing author.

resses: [email protected] (J. Gallagher)8 [email protected] (K. Bauer)8

yale.edu (D.M. Dollar).

J. Gallagher et al. / Libr. Coll. Acq. & Tech. Serv. 29 (2005) 169–179170

libraries in the science, technology, and medical fields, CWML is in the midst of a

paradigm shift. Because users of these libraries’ resources demand the advantages that

electronic access offers, libraries are subscribing to e-journals. Library users appreciate

the accessibility and convenience that e-journals provide them. E-journal licenses allow

libraries to provide patrons with access to journals’ current content and any available

backfiles as well.

Because the transition to the electronic environment has been so rapid, many libraries have

found themselves unprepared to manage the change effectively. The transition to electronic

access impacts many facets of the library, as different resources are required to manage the

electronic collection than are needed to manage the print collection; for instance, the costs

associated with supporting print are vastly different than those involved in managing

electronic resources. A study of Drexel University’s W.W. Hagerty Library that analyzed all

the operational costs necessary to support each format concluded that electronic journals are

significantly more cost-effective on a per use basis [1]. Libraries’ serial expenditures,

however, continue to spiral upward because of the increased cost of journals and the increased

access to content. Libraries attempting to retain control of their budgets are concerned about

the value of the services they are providing their patrons and about how efficiently they are

utilizing library resources to provide these services.

For service and budgetary reasons, the CWML decided it necessary to analyze how the

journal collection was being used. Because e-journal usage statistics provided by journal

publishers cannot be compared on a one-to-one basis with in-house print journal usage

statistics, the principles of evidence-based librarianship (EBL) were utilized. bEBL employs

the best available evidence based upon library science research to arrive at sound decisions

about solving practical problems in librarianshipQ [2]. CWML staff proposed analyzing

pertinent data from all available sources to determine the most effective and efficient use of

the library’s resources. Sources included the following:

! Unbound print journal usage statistics captured by a 3-month study of the library’s Current

Periodical Room conducted in October, November, and December 2003.

! SFX statistics of online journals accessed during the same time frame as the print study.

! Annual photocopying statistics for public copiers located within the library.

! Annual gate counts of patrons entering the physical library.

! Annual bound-journal shelving statistics.

! Annual association statistics, including the Association of Research Libraries (ARL) and

the American Association of Health Sciences Libraries.

2. Methodology

2.1. Current periodical usage study

At the time of this study, CWML’s Current Periodical Room actively received 1249 journal

titles in print. As each new issue arrives, it is processed by technical services staff and then

J. Gallagher et al. / Libr. Coll. Acq. & Tech. Serv. 29 (2005) 169–179 171

shelved by circulation staff in the Current Periodical Reading Room. For patron convenience,

new issues are shelved alphabetically by title. Issues are bound according to each journal

title’s binding schedule. Once bound, they are shelved alphabetically by title in the journal

stacks of the library.

Several methods to accurately track title use were considered. The selected method was by

far the least technical, yet was extremely efficient. An alphabetical list of the titles was

generated in Excel. Columns for each week of the study were added so that shelvers could

simply place a check mark each time they re-shelved an issue during a specific week.

Shelvers gathered all unbound periodicals in the library that were not shelved and placed

them in the unbound periodical pre-shelving area. Here they were arranged in alphabetical

order and recorded. Each issue of a title received one check mark to indicate one buseQ.Shelvers then immediately re-shelved the recorded issues in their correct place to prevent

them from being recounted.

During the study technical services staff delivered all newly received issues directly to the

circulation desk, instead of the unbound periodical pre-shelving area as is the normal

workflow. This was done to eliminate the risk that these issues would be included in the usage

study, since we were only interested in counting issues re-shelved as a direct result of patron

use. Additionally, signage was posted throughout the room, requesting patrons not to re-

shelve any issues since a study was in progress. This was done to try to limit the amount of

uncounted uses.

The usage data were collected weekly and input into the Excel file. The total use of the

unbound issues was tracked. The strength of this approach was its simplicity. The staff

involved in recording usage clearly understood the process and found it easy to track use

accurately.

2.2. SFX statistics

SFX is a linking tool from ExLibris that makes it convenient for patrons to access an

electronic resource. The Yale SFX Server maintains holdings information for all Yale

Library journal holdings. When patrons search library research databases (such as

MEDLINE, EMBASE, etc.) for specific articles, a link to SFX is provided for each

search result. If the full text of the article is available electronically, SFX will use the

OpenURL standard to construct a dynamic link to the full text. If the article is not available

electronically, SFX will prompt the user to search the Library’s Catalog ORBIS to

determine if the library system subscribes to the journal in print. Since it saves a

considerable amount of time and effort, SFX has grown popular with patrons and its use

has grown 80% each year since its inception.

SFX statistics provide request data about all the electronic journals Yale University Library

System (YULS) subscribes to that are accessed via SFX. In order for the SFX statistics

analyzed to reflect the subject areas covered by CWML’s print journal collection, only those

journals indexed in MEDLINE were extracted. At the time of this study the number of

MEDLINE-indexed titles accessed via SFX was 3465. SFX statistics were gathered for the 3-

month period that print usage statistics were gathered.


3. Limits of the study

The shortcomings of traditional journal usage statistics have been well documented [3].

Re-shelving statistics cannot tell us how the journal was used while off the shelf. Did a patron

read an article in the issue? Did the patron read more than one article? Perhaps the patron

merely browsed the table of contents or used the journal to block an irksome air-conditioning

floor register that was blowing cold air on her. These are just some of the possibilities that we

categorize as duseT when we rely on re-shelving statistics.

Electronic journal use statistics pose similar problems. Publishers and vendors provide

reports that detail how many uses each particular title received, but again these statistics

have both technical and practical shortcomings. Other than an ethical obligation for

example, what motivation do publishers have to provide statistics that may negatively

impact libraries’ perception of the value of certain titles? In 2001 Luther wrote bPublishersare concerned that the data they share with libraries lack context. If, in the absence of such

a context, usage data seem low, the publishers fear that librarians may use such information

as a basis for canceling subscriptionsQ [4]. As for accuracy, what constitutes an electronic

buseQ for one vendor may not be the same as how another vendor defines it. Some vendors

link to the article level, while others link to the title, volume, or issue level, but whether the

user actually read or even browsed the article cannot be established. Publishers and

librarians have taken a positive step towards solving the problems currently associated with

electronic journal usage data through the creation of Project Counter, an initiative formed in

2002 to attempt to standardize the way online journal usage statistics are gathered and

shared [5].

Unfortunately, not all vendors permit nor can facilitate SFX linking within their

databases. As a result, approximately 86% of the Medical Library’s e-journals can be linked

to via SFX. Additionally, SFX is only one means of accessing e-journal articles. It

represents articles accessed as a result of an active search in a database. There are

numerous other ways a patron can arrive at the full text of an article, such as following

links through the e-journal database or accessing them using a bookmark they created to a

specific title. To better determine what portion of e-journal use SFX statistics actually

represents, a random sample of 75 titles was assessed to compare SFX and publisher

statistics. The results of this study were not entirely clear-cut. For the 10% of this sample

representing the most popular journals (as measured by vendor usage data) SFX use

comprised only 3.5% of all electronic use. For the other 90% of the electronic journals in

the sample, SFX use represented 15.0% of all electronic journal usage. In fact, for the least

popular journals SFX usage approached 60% of overall use. But overall, SFX use

represented approximately 8.5% of all electronic journal usage.

Finally, whereas the usage study of the current periodical collection only measured use

of articles contained in issues not yet bound, SFX statistics reflected use of articles from

backfiles, and therefore focused on a broader timeframe. Generally researchers studying

the subjects of science, technology, and medicine are primarily concerned with the most

recent research, but nevertheless it is difficult to quantify exactly what effect this has on

statistics.


4. Results

CWML has in recent years responded to patrons’ requests for convenient access to

resources by aggressively acquiring electronic resources. In FY 1999 CWML subscribed to

528 biomedical e-journals, but within 5 years this figure rose to 3391 to reflect a 642%

increase (Fig. 1). Statistics from the American Association of Health Sciences Libraries

(AAHSL) from 1998 to 2002 indicate that this trend is not unique to the CWML [6]. AAHSL

member libraries have been reducing the quantity of print titles they subscribe to while

purchasing access to an increasing number of online titles (Fig. 2).

Undoubtedly electronic access to journals and resources constitutes a major service

enhancement for patrons. However, providing remote access to journals is costly for libraries.

Data from the Association of Research Libraries (ARL) from FY 1999 to FY 2002 show that

spending on electronic serials has increased by almost 125% [7] (Fig. 3). At CWML between

FY 1999 and FY 2003 serial expenditure rose by 71% (Fig. 4).

CWML gate counts show that there has been a steady decline in the number of patrons

entering the library in recent years. Gate count figures accrue each time the exit gate of the

library is opened. Since it is a fact that oftentimes patrons will hold the gate open for each

other, the figures do not represent the actual amount of people who were in the library during

the year. However, such behavior is consistent and therefore the figures provide us a reliable

gauge to measure the percentage of decrease. Between July 1, 1998, and June 30, 2003 (a 4-

year period), a 32% decline in patrons entering the physical library was observed (Fig. 5).

As a result of the decrease in the number of patrons coming to the library, use of print

journals within the library has been decreasing. There are several methods to track this but the

most accurate and telling gauge of this decrease has been the considerable drop in the number

of photocopies made in the library’s public photocopying room. This room is located on the

same level as the bound print journal collection. It contains 14 digital photocopiers that are

Fig. 1. CWML journal subscriptions by format.

Fig. 2. AASHL average journal subscriptions by format.


operated and maintained by an external vendor. Other than enhancements, no negative

changes that would impact patrons have been made. The cost to photocopy materials has

remained the same during the study period. Nevertheless, statistics indicate that 56% fewer

copies were made over the 5-year time period (Fig. 6). It is highly probable that use of the

print journal collection shows a similar trend.

The most notable finding of the current print journal usage study was that 53% of the

collection (657 titles) received no use during the 3 months that the study was conducted

(Table 1). While it was suspected that print use had been declining, that it was so severe was

Fig. 3. —n— Average ARL member library electronic resources expenditure.

Fig. 4. —x— Annual CWML serial expenditures.


surprising. Another notable statistic is that only 7.1% (89 titles) was used more than once a

month, and only 1.28% was used one or more times a week. Only a small portion of the

current print collection is used significantly. In fact 10% (125 titles) of the collection

represents 60.7% of the total use of the current print collection.

SFX statistics on the usage of 3465 MEDLINE indexed titles show that 14.8% (513 titles)

were not accessed at all during the second quarter of FY 2004. 10% of the journals

represented 56.8% of all SFX usage (Fig. 7).

5. Discussion

The one overriding observation of the study was that print journals are used only a fraction

as often as their electronic counterparts. Even though SFX represents only a small portion of a

title’s online use, electronic use via SFX far exceeds statistics for the title’s print use. For

Fig. 5. —x— CWML physical library annual gate counts.

Fig. 6. —x— CWML Copico annual photocopying statistics.


instance, the most-used print journal during the 3 months of the study was JAMA. The print

format received 69 uses over 3 months, while it was accessed 570 times via SFX. Similarly,

the Lancet received 35 print uses versus 524 SFX hits. When compared to publisher statistics

the difference is magnified. For example, publisher statistics for the Lancet show that it was

accessed 3468 times during the 3 months of the study. It would be foolhardy to conclude that

this confirms that the Lancet’s electronic format is used 100 times more often than the print

version (i.e., 3468 hits versus 35 print uses), but undoubtedly the difference between the two

formats is incontrovertible and staggering.

The study also showed that titles that were used most in print were also the titles most used

electronically. An analysis of the top 10 titles used in each format shows that these titles

tended to be titles that included articles from a broad spectrum of disciplines as opposed to

those that are discipline specific (Table 2).

That such a large portion of the print collection was unused during the study merits

action. Maintaining the print collection is far more labor intensive and costly than

maintaining an electronic collection [8]. The resources expended by the library to support

the 53% of the current print collection that was not used seriously impacts efficiency. While

some of these resources will most likely be used at some point in time, the value of

acquiring low and no-use journals in print format is minimal. Regardless of the actual cost

of these print items, the cost of the staff time involved in acquiring, receiving, and shelving

these resources could be used more productively on other endeavors. Other costs such as

the cost of shelving space, processing space, etc., must also be considered. Most

importantly, spending money to support unused resources results in more valuable

resources not being acquired for users, and this directly impacts how they perceive the

value of the library to them.

Table 1

Use per title of current periodical (print format) for Q2 FY 2004 CWML

Number of uses 0 1–5 6–10 11–15 16+

Number of titles 657 528 38 14 12

Fig. 7. Use of e-journals accessed via SFX for Q2 FY 2003 at CWML.


Identifying the titles that receive little or no use is only the first step towards the efficient use

of library resources. Actually transferring existing subscriptions to online only subscriptions or

canceling titles that do not provide an online equivalent is the next step. In order to cancel print

only titles, the library will need to consider carefully with feedback from relevant departments

and discipline areas whether to drop those titles. However, most of the print titles are received

in conjunction with an e-journal package. Cancellation clauses as specified by licensing terms

with publishers, vendors, and aggregators of these titles, mandate whether the print can be

cancelled or not. Many of these licenses allow for converting subscriptions to e-only for a

cost savings usually representing between 5% and 10% of the print list price. Where such

an option is available the library should avail of the opportunity to unburden itself from

the cost of supporting the print format. To that end, CWML will be transferring 212

existing print subscriptions to online only for 2005 as allowed under the licensing terms

from seven separate e-journal packages. The final transfer decisions were based on criteria

developed in cooperation with the Yale University Libraries that covered such issues as

Table 2

Most frequently used titles by format at CWML

Top 10 current print uses Top 10 current SFX uses

JAMA 69 680 New England Journal of Medicine

New England Journal of Medicine 47 659 Nature

Science 45 639 Science

Lancet 35 570 JAMA

Nature 28 524 Lancet

Consumer Reports 27 460 Journal of Biological Chemistry

American Journal of Psychiatry 24 405 Proceedings of the National

Academy of Science

Annals of Internal Medicine 23 404 Circulation

Cell 22 386 American Journal of Psychiatry

BMJ: British Medical Journal 20 320 Child Development


archival access, reliability of the journal interface, and allowances for scholarly sharing.

Where there is no price advantage to shedding print for e-only access, the library should

choose not to process the print even though they receive it.

Another observation about the use of print and electronic formats is that a minority of the

titles subscribed to constitutes a majority of the use. The CWML collection shows that 20%

of print titles accounted for 77.8% of print use, while 20% of e-journals accounted for 73.8%

of use. This is consistent with other studies of health science libraries’ collections. For

instance, a 1998 study of the Norris Medical Library at the University of Southern California

found that 20% of titles accounted for 60% of use, regardless of format [9].

6. Further study

While the cancellation of print is often permitted, many journal package licenses prohibit

complete journal title cancellation or only allow for modest reductions in overall spending

whether in print with online or e-only. Thus, titles are bbundledQ together as part of a bBigDealQ [10]. Therefore, further analysis of the use of titles in specific packages is warranted to

determine the value of the package. Many institutions are not renewing specific packages for

budgetary reasons without data to determine what impact this will have on the resources’

users. Analyzing e-journal statistics by vendor and package will provide libraries with useful

information to better determine the true value of each package deal.

7. Conclusion

The study provides factual information about use of the CWML journal collection. It

confirms that in general CWML patrons utilize the electronic format of articles instead of the

print version. Statistics considered from various sources determined this finding and

emphasized the need for the library to take steps towards reducing serial expenditure. To

ensure that the library remains valuable to its users in an era when the Internet is most often

the first source checked for information, libraries must continue to make authoritative

resources convenient and accessible for patrons. Undoubtedly e-journals and other electronic

resources will remain the means to accomplish this mission for the foreseeable future. It is

crucial therefore for libraries to determine how best to manage these resources, since failure to

do so will result in the continued inefficient use of resources, and an underserved user-base

whose perception of the value of the library will diminish.

References

[1] Montgomery, C. H., & King, D. W. (2002). Comparing library and user related costs of print and electronic

journal collections. Retrieved October 21, 2002, from http://www.dlib.org/dlib/october02/montgomery/

10montgomery.html

http:\\www.dlib.org\dlib\october02\montgomery\10montgomery.html


[2] Eldredge, J. D. (2000). Evidence-based librarianship: An overview. Bulletin of the Medical Library

Association, 88(4), 289–302.

[3] Chrzastowski, T. E. (2003). Making the transition from print to electronic serial collections: A new model

for academic chemistry libraries? Journal of the American Society for Information Science and Technology,

54(12), 1141–1148.

[4] Luther, J. (2001). White paper on electronic journal usage statistics. Serials Librarian, 41(2), 118–119.

[5] COUNTER. (2002). Project COUNTER. Retrieved October 1, 2004, from http://www.projectcounter.org/

[6] Byrd, G. D., & Shedlock, J. (2003). The association of academic health sciences libraries annual statistics:

An exploratory twenty-five-year trend analysis. Journal of the Medical Library Association, 91(2), 186–202.

[7] Association of Research Libraries (2003). ARL Statistics Interactive Edition. Retrieved October 21, 2004,

from http://fisher.lib.virginia.edu/arl/

[8] Montgomery.

[9] Morse, D. H., & Clintworth, W. A. (2000). Comparing patterns of print and electronic journal use in an

academic health science library. Issues in Science and Technology Librarianship.

[10] Frazier, K. (2001). The Librarians’ Dilemma: Contemplating the Costs of the bBig DealQ . Retrieved

November 1, 2004, from http://www.dlib.org/dlib/march01/frazier/03frazier.html

http:\\www.projectcounter.org\

http:\\fisher.lib.virginia.edu\arl\

http:\\www.dlib.org\dlib\march01\frazier\03frazier.html

evidence-based librarianship: utilizing data from all available sources to make judicious print...

Documents