E-Use Measurement: A Detour Around the Publishers

ALCTS | Summer 2005

Joe ZuccaAssessment, Planning and Publications LibrarianUniversity of Pennsylvania Library

0.0

0.5

1.0

1.5

2.0

2.5

3.0

3.5

4.0

Harnwell N=757

Kings Court N=358

Stouffer N=345

Av

r. L

og

in p

er

Re

sid

en

t

Student Use of E-Resources: Three Profiles by Residence logins per capita, by hour. Feb ‘03-May ’05 [49,857 logins total]

Harnwell N=757

Kings Court N=358

Stouffer N=345

Hour of Day

ALCTS Summer 2005

TITLES3 HOUSES as a

GROUPALL DOMAINS

LexisNexis Academic 1 8

Factiva 2 4

Onefile 3 11

MEDLINE 4 1

JSTOR 5 12

ScienceDirect Journals 6 2

Oxford English Dictionary 7 27

PubMed Plus 8 3

MLA International Bibliography 9 25

PsycINFO 10 10

CINAHL/ Nursing 11 17

ABI/Inform - Proquest 12 18

ISI Citation Indexes 13 5

Encyclopaedia Britannica 14 37

Oxford Reference 15 28

The Top 15 Titles by Use, Comparative Ranks: 3 Houses and All Users

ALCTS Summer 2005

Key Attributes of Metrics and Measurement Processes

Metrics

Comprehensive [Licensed Stuff, Local, Public]

Processes-Architecture

Scaleable, Efficient (high output for low effort), Sustainable, Distributed

Processes-Data Granular, Normalized, Integrative, Anonymous

Context-Sensitive [Location, Time, Demographic, Cost, Resource Type & Class, Need]

ALCTS Summer 2005

Consistent [Uniform Across Resources, Replicable]

1. User initiates a session using pages spawned by a database

Schematic View of the Data Logging Process

5. Db logs event 5.1 Proxy server logs the event

2. Db swaps handle for authentic url

3. Proxy scan

3.1 Authentication challenge

4. Pass to vendor site

6. Data Farm

ERMS

ALCTS Summer 2005

Service Events

CustomersStaff

Data Farm Project: Events, Data, Plumbing, Information

People andNetwork Data

Clean | Anonymize | Integrate

Data Streams

Inform

Staff

Borrow a Video

Access an E-journal

Catalog a Book

Visit a Library

Copy or Print

Data Farm Environment

ALCTS Summer 2005

E-Resource Use Reference ContactCirculation

Acquisitiions Funds

Holdings

Copier | Printer Use

Gate Swipes

Resolve Resource

Resolve People

Resolve Places

Web Analytics

Image Collection Use

Data Farm Oracle Space: 14+ gb, in 75 tables, tracking events and their properties

Staff Census

LDAP

Tech Processing Workflow

ALCTS Summer 2005

Digital Library (erms)

Print Library (Voyager)

Building Use

Metadata

Administration

Reference|Instruction (dynamic)

So, what do we do with the data?

Dashboard Reports (static)

Report Builders (dynamic)

Data Bureau (ad hoc reports and fishing expeditions)

Lower barriers between management information and people with management responsibility (or any other staff)

Help to scale and institutionalize assessment

Provide a distributed and multi-tiered environment for interacting with raw data and generating quantitative information

ALCTS Summer 2005

Dashboards: Vital signs at a glance

INTERACTIONS with DATA

ALCTS Summer 2005

Report Builders

INTERACTIONS with DATA

ALCTS Summer 2005

Attempts at ranking for selection

based on use and other

factors

ALCTS Summer 2005

A Hypothetical Scheme for Journal Selection Ranking

Qualitative Scores Quantitative Scores Factor Rank

Librarian rank Impact Penn Artcls. Use Metric Cost Effc

Title 1 1 2.60 1 3 4 -1.54 1

Title 2 1 3.87 6 4 54 -1.72 2

Title 3 3 22.4 9 9 8 -1.89 3

Title n

ALCTS Summer 2005

Ad Hoc | Mediated

INTERACTIONS with DATA

Van Pelt (public areas) 174,584 55.2% 55.2%

Biomedical (public areas) 41,577 13.2% 68.4%

Staff (Van Pelt-Dietrich and Biomedical) 41,356 13.1% 81.5%

Electronic Classroom 14,535 04.6% 86.1%

Fine Arts 13,671 04.3% 90.4%

Lippincott 10,293 03.3% 93.6%

Veterinary 5,708 01.8% 95.4%

Dental 4,304 01.4% 96.8%

Museum 3,598 01.1% 97.9%

Chemistry 1,950 00.6% 98.6%

Math 1,901 00.6% 99.2%

Undergraduate Study Ctr 1,502 00.5% 99.6%

Rare Book and Manuscript 954 00.3% 99.9%

High Density Storage 209 00.1% 100.%

Total Libraries 316,140 100%

Distribution of E-resource Use Across the Libraries FY04

Logins Pct. Cml Pct.

ALCTS Summer 2005

Items Charged and E-resource Log-ins Fall 2004

Ad Hoc Interactions with the Database

ALCTS | Summer 2005

Relationship Between E-Resource Use and Item Circulation, Fall 2004 72 observations

ALCTS | Summer 2005

Practical Applications of Data Farm Resources

Budget defense (slow but steady rates of increase, curtailing the need to make major cuts in materials)

Feeding the data-mavens and spreading best practices

E-resource selection (driving trade-offs more typically than cuts)

Print collection management

Staffing | Facilities Planning

Developing and testing web tools

ALCTS Summer 2005

Key Attributes of Metrics and Measurement Processes [REPRISED]

ALCTS Summer 2005

Metrics

Comprehensive [Licensed Stuff, Local, Public]

Processes-Architecture

Scaleable, Efficient (high output for low effort), Sustainable, Distributed

Processes-Data Granular, Normalized, Integrative, Anonymous

Context-Sensitive [Location, Time, Demographic, Cost, Resource Type & Class, Need]

Consistent [Uniform Across Resources, Replicable]

E-Use Measurement: A Detour Around the Publishers

Joe Zuccazucca@pobox.upenn.edu

University of Pennsylvania Library

ALCTS | Summer 2005

ScienceDirect Journals: Regression of View Counts (Elsevier) on Penn Login Counts. Feb-Nov 2003. [1,270 cases]

ALCTS | Summer 2005

R2 = 0.9814

0

200

400

600

800

1000

1200

1400

1600

1800

0 500 1000 1500 2000

Penn Login Count [count-use]

OC

LC L

ogi

n C

oun

t

OCLC Databases: Regression of Vendor Login Counts on Penn Login Counts, jl2001-je2002. [48 cases]

Three E-Use Profiles by Residence: Top Ten Titles Used Logins per Capita, Feb ‘03-May ‘05

Harnwell N=757

Kings Court N=358

Stouffer N=345

LexisNexis Academic 2.7 1.7 0.6

Factiva 2.2 1.3 0.5

Onefile 1.4 1.4 0.4

JSTOR 1.2 0.7 0.5

MEDLINE (complete file) 1.1 0.8 0.5

Oxford English Dictionary 0.7 0.8 0.4

PubMed Plus 0.8 0.7 0.2

MLA International Bibliography 0.7 0.8 0.3

PsycINFO 1.0 0.3 0.2

CINAHL 0.7 0.4 0.5

ALCTS | Summer 2005

TITLES Harnwell Kings Court StoufferTotal

Logins

LexisNexis Academic 2011 597 201 2809

Factiva 1672 454 164 2290

Onefile 1381 611 188 2180

MEDLINE 916 297 200 1413

JSTOR 887 261 158 1306

ScienceDirect Journals 809 226 190 1225

Oxford English Dictionary 565 286 126 977

PubMed Plus 617 250 82 949

MLA Intrn’l Bibliography 511 301 104 916

PsycINFO 731 102 53 886

CINAHL/ Nursing 563 134 171 868

ABI/Inform 689 121 43 853

ISI Citation Indexes 392 204 110 706

Encyclopaedia Britannica 359 235 45 639

Oxford Reference 354 215 64 633

Three E-Use Profiles by Residence: Top 15 Titles Used (by login count)

ALCTS | Summer 2005