Experiences on the Ground: usage stats as a practical tool in the Library

Jill Taylor-RoeNewcastle University LibraryUKSG Usage Statistics Training Seminar 27 June, 2006

There is nothing new under the sun…

Libraries have been collecting and using usage data for many years

Initial core data – based around print based services and physical presence of users in Library

Current data more likely to focus on remote use of e-services

Usage and Value

What do we measure and why? How does it help? Are we measuring the right things? Are we making correct assumptions

from the data?

What do we measure and why?

Anything which tells us about the utilisation of services in our Library

Data which might help to demonstrate whether we are getting a reasonable return on our financial investment

Data which may help to support cases for funding Data which can inform our strategic

planning

Usage data can be…

A practical management tool - helps assess effectiveness

A means of assessing performance A planning tool A political tool A means of helping to demonstrate VFM A means of highlighting user satisfaction Support for funding bids

Traditional Library Usage Data

Academic Year 94/95 99/00 03/04 04/05

Issues 574,618 760,128 505,044 759,091

ILL Requests 35,771 23,335 13,180 12,939

Photocopies 4,431,397 4,498,000 1,746,648 1,875,358

Visits to the Library 1,243,970 1,175,249 1,012,670 1,121,759

Graphs can (sometimes!) be helpful

Tradtional Library Usage Data

0

500,000

1,000,000

1,500,000

2,000,000

2,500,000

3,000,000

3,500,000

4,000,000

4,500,000

5,000,000

94/95 99/00 03/04 04/05

Issues

ILL Requests

Photocopies

Visits to the Library

New Library Performance Indicators

Gross Library Expenditure per staff/student FTE Gross Information Expenditure per staff/student FTE Library Staff Expenditure per staff/student FTE Cost per seat hours per annum Cost per use of electronic journals Expenditure on e-resources as a % of total expenditure Uses per day of e-resources per staff/ student FTE

E-Journal Key Performance Indicators per package @ NUL (1)

METRICS COSTS

Total No. titles Total package Total full text downloads Cost as % of total serials

budget Mean cost per title

Max downloads (month) Mean cost per F/T download Min downloads (month) Cost per FTE user***Mean downloads (year)PDF full text downloads HTML full text downloads PDF/HTML ratioDownloads per FTE user*Total turnaways (titles not included in licence) Medium use: (11-99 hits)

High use: (= >100 hits) Top 20 best performing titles as % of total hits Low-use: (0-10 hits)

E-Journal KPI profile with data

Graphs CAN sometimes be helpful

What are these usage stats telling us?

Average cost per f/t download for large STM packages now around £1.50 - £1.75

Average cost per use of e-journals = £0.93 Usage still appears to be increasing year on year Humanities and Social Sciences packages seem

to be less well used than STM Initially, titles we had as print were always better

used than unsubscribed content, but now many of the latter are beginning to show heavy use

How does this help? (1)

Given the significant sums we are investing in e-resources – cost per use figs appear to show good value for money

Total f/t downloads per annum are still rising – ( may offset decline in traditional ILL) Journals collection has effectively doubled in size

– more choice for users Because usage is increasing year on year – MAY

help persuade paymasters to maintain and preferably increase funding

How does this help? (2)

Dealing with practical queries: “Why are you wasting money on all these

second rate journals – you should only buy the top IF rated titles in each field and cancel all the rest.”

QUESTION: does high IF correlate with high use?

How does this help? (3)

“There is a new journal called Nature Physics. This is essential for my research so please take a subscription out immediately.”

QUESTION: would it be worth trying to find funds for this title?

Building up evidence…

Average cost per use of current Nature Journal Collection: £0.74 in 2005

Overall package usage has increased year on year by over 30%

Would be worth securing trial access to the title to see how it compares with the rest

This MAY provide enough supporting evidence to warrant making a case for funding a subscription.

How does this help? (4)

“ SD is excellent – but it would be even better if you bought the back-files then we can search back to volume 1.”

QUESTION – is it worth looking at the backfiles? How could we make a case to buy them??

Building up evidence…

How many metres do our current print back-runs occupy? If we were to charge an annual rental for this space what would

it be? Have we any evidence of usage of the print runs? Can we trial access to the back runs? Can we separate out usage of the backfiles from the current

files? What level of usage of the digital backfile would support a case

for purchase? Would it need to be significantly cheaper than the cost of

maintaining the print runs on the shelves?

Are we measuring the right things?

Managerial Realities… (1)

Library budgets still fail to keep pace with the demands placed upon them

Most Libraries are competing for funds with other services and academic schools

Demand for new resources, especially (but not exclusively) e-resources is insatiable

Managerial Realities… (2)

In the print environment it was difficult to assess claims that particular resources were “essential” both pre and post purchase

E-usage stats are a powerful new tool to help us assess the value of the resources which are absorbing increasingly large percentages of our budgets

We are constantly having to respond to new initiatives requiring us to demonstrate the cost effectiveness of our services

Summary points

We need usage data in order to… Provide KPIs for our paymasters To inform collection development strategies Evaluate whether the resources we have

selected are providing good VFM Support cases for future funding

Are we making the correct assumptions from the data?

At regular intervals you should always stop and think:

Are we taking a fair and balanced view? Is there critical data that we are missing? Are there any significant flaws or omissions in the

data we are using?

More work is needed

More info re usage of backfiles vs current issues Clarify the position re subscribed and non-subscribed

content in usage reports More info on turnaways – these too could influence

future purchases More comparative data to facilitate benchmarking

between libraries Greater number of publishers to be fully COUNTER

compliant Data gathering process needs further streamlining

Concluding thoughts

Usage metrics have come a long way in a short time They are far from perfect, and need to be interpreted

with care, but they are important to librarians in our role as information managers

They are important to publishers as a means of understanding and refining their market

We need to acquire and share more expertise in collating and analysing usage data

We need more and better usage data – particularly comparative data so that we can benchmark effectively

Openly collaborative work between librarians and publishers will help us to move forward faster