dream 5: building evidence of the value and impact of library information services

Center for Information and Communication Studies

DREaM Project ConferenceBritish Library, 9th July 2012

Opening keynote presentationCarol Tenopir

University of Tennessee, USA


Building evidence of the value and impact of library and information

services: Methods, metrics, and ROI

Carol TenopirUniversity of Tennessee

[email protected]

DREaM ConferenceJuly 9, 2012


Today’s topics…

• Lib-Value project overview• Defining value in the library and

information context• Techniques and examples:

– Critical incident– ROI and contingent valuation– Qualitative and personas


Lib-Value: Multiple academic institutions using multiple methods to measure multiple values for

multiple stakeholders


Ebooks Special Collections

Information Commons

Journal Collections

Methods for Measuring

ValueTeaching and

Learning

Reading and Scholarship Digitization

Website and Value

Bibliography

Current projects


1. purchase or exchange value: what one is willing to pay for information in money and/or time, and

2. use value: the favorable consequences derived from reading and using the information.

In the information context economist Machlup described 2 types of value:


Lib-Value comprehensive library value study (Bruce Kingma)

• Economic (private)– What is the value to an individual to use the library

resources?• Social (public)

– What is the value to the institution of the library?• Environmental (externality)

– What is the value of the environmental savings of library provision of electronic resources?

– Have libraries gone green without knowing it?


1) Implied value (i.e., usage, downloads)2) Explicit value (i.e., outcomes, critical incident)3) Derived values (i.e., contingent valuation, ROI)

*These methods are useful in any type of library, but most of my examples are from academic libraries

Value of reading can be measured in many ways*


Implied value: DownloadsUTK Article Downloads


Article readings 1977 to present by scientists and social scientists

*

*2011-2012 (UIUC) n=639, (UK),n=1013; 2005,n=932; 2000-03, n=397; 1993, n=70;

1984, n=865; 1977, n=2350

Rea

ding

s pe

r yea

r


Exchange

Use/Outcomes

Going beyond implied value to show…


Tenopir & King scholarly reading studies, 4 types of questions:

1.Demographic

2.Recollection

3.Critical Incident

4.Comments

Therefore, insights intoboth READERS and

READINGS


“The following questions in this section refer to the SCHOLARLY ARTICLE YOU READ MOST RECENTLY, even if you had read the article previously. Note that this last reading may not be typical, but will help us establish the range of patterns in reading.”

Critical incident of last reading


Time spent (exchange value) reading• Article

• 49 min/article X 22 read per month X 12 months = 216 hours

• Book• 106 min/book X 7 per month X 12 months=

148 hours• Other Publication

• 42 min/publication X 10 per month X 12 months=84 hours

U.K. academics spend on average per year nearly three months of their work time reading scholarly

material.


A majority of articles are from the library:

UK, n=1189, June 2011; UIUC, n=256 April 4, 2012

Per

cent


Use of library collections for articles

N=775, 6 UK universities, June 6 2011


Environmental and exchange value

ActivityAverage last 30

days% of

respondents Physical visits 2.9 visits 73%Remote visits 14.2 visits 88%

Average total resources used: in-person visit to the library 7.3 uses 80%

remotely online 14.9 uses 89%


Source of reading by purpose of reading: Faculty in UK

n=1161, 2011

Teaching

Research

Current Awareness


Outcomes of journal article reading

1st Inspire new thinking or ideas 54%2nd Improve results 38%3rd Narrow/broaden/change the focus 28%

4th Resolve technical problems 10%5th Save time or other resources 10%6th Aid in faster completion 5%7th Assist or result in collaboration/joint research

4%

n=2117, 6 UK universities, June 2011


Return on investment in a strict sense…

…is a quantitative measure expressed as a ratio of the value returned to the institution for each

monetary unit invested in the library. For every $/€/£ spent on the library,

the university receives ‘X’ $/€/£ in return.

Demonstrate that library collections contribute to income-generating activities


Library’s value to the grants process 9 institutions in 8 countries

http://libraryconnect.elsevier.com/whitepapers/roi2/lcwp021001.html


• 13.2:1 to 15.5:1Research

STM

• 1.3:1 to 3.4:1Research and

TeachingSTM/Hum/SS

• Under 1:1Research and

Teaching

ROI from access to journal articles cited in grant proposals


an economic method of evaluation of services and goods which looks at the implications of not having the services.

ROI through contingent valuation


National Network of Libraries of Medicine: Retail Value Calculator

• http://nnlm.gov/mcr/evaluation/calculator.html• How much would it cost to replace your

library services on the retail market?• Calculate what it would cost to buy library

services - at a book store, through pay per view for articles, from an information broker - if you and your library weren't there.

http://nnlm.gov/mcr/evaluation/calculator.html


National Network of Libraries of Medicine: Library value calculator

• http://nnlm.gov/mcr/evaluation/roi.html• How much benefit does your institution, your

user, receive for every dollar spent by the library?

• Value of benefits and costs for each service• Total value of your library use

http://nnlm.gov/mcr/evaluation/roi.html


Measuring the ROI of Today’s Libraries: About the study:

• Part of a grant to assess the value and ROI of academic library services (Lib-value)

• Assesses the use, value, and ROI of 77 academic library services

• This data is from Bryant University

• Similar project underway at Drexel University (May 2012).


Readings from the LibraryN

o. o

f rea

ding

s fro

m li

brar

y

Total Number of Readings from Library per year


Purchase Value of Some Services:

• The hourly rate of users:–Faculty: $56.20 per hour–Staff: $36.00 per hour–Students: $34.60 per hour


Contingent Value

• The cost of not having access to library-provided articles.

• Faculty-only (students not asked)• Total cost: $408,600• Cost per Faculty: $1,200• Cost per reading: $27


Return on investment is also…

…values of all types and outcomes that come to stakeholders and the institution from use of the library’s collections, services, and contribution to its communities.


Academics praise the library for its long-term outcomes

Electronic access to the university library system from off-site is crucial for swift access to articles to support my teaching and research activities.

Library resources have been essential to my work for the past 20 years.

The journal collection at my institution is excellent and scholarship is all the richer for the contribution for easy access to journals and print publications.


• Has won an award in the last two years.

• Publishes four or more items per year.

• Reads more of every type of material.

• Spends more time per book and other publication readings.

• Uses the library for articles

• More often buys books and obtains other publications from the Internet.

• Occasionally participates and creates social media content.

What a ‘successful’ academic looks like:


Persona: “Akrum Patel”I would like my library to have subscriptions to more journals and for longer periods of time.

• Key Facts:• Associate professor in physics.• Reads 30 articles, 2 books, and 11 other

publications per month.• What he needs:

• Current issues of articles.• Off-site access to collections. • Access to search engines and online resources

without a distinction between library and non-library resources.

• Factors:• Reads seminal books.• Wants to see trends over time.• Has not visited a physical library for many years.


Persona: “Sally Fitzgerald”• Key Facts:

– Spends majority of time on research and teaching.

– Reads 30 articles and 15 books per month.

• What she needs:– Older articles in addition to new

publications.– Frustrated when can only find abstracts

and not full-text.• Factors:

– Library doesn’t always have the books she needs.

– Needs wide range of material.

My research and teaching cannot exist without [library’s e-collections]…and not finding them right away is heavily disruptive on my work.


Value is demonstrated by time invested, by value to purpose, by

outcomes of use and by ROI.

Multiple methods should be used to measure value.


Some final thoughts on measuring value• Tie what you measure to the

mission• Quantitative data can show ROI

and trends• Qualitative data tell a story• No one method stands alone• Measure outcomes, not inputs• The further downstream the

value, the more challenging to measure, but perhaps the

most important.


More details and searchable Lib-Value bibliographic database available on the

project website:

http://libvalue.cci.utk.edu

http://libvalue.cci.utk.edu/


For further information: [email protected]

mailto:[email protected]

dream 5: building evidence of the value and impact of library information services

Technology

critical incident

6 uk universities

libraries

national network

reads 30 articles

measure

collections

library