a study of citations in users’ online personal collections

ECDL, Budapest,Hungary, Sep 16 – 21, 2007

A Study of Citations in Users’ Online Personal Collections

Nishikant KapoorJohn T Butler, Sean M McNee, Gary C

FoutyJames A Stemper, Joseph A Konstan

GroupLens Research Group and University LibrariesUniversity of Minnesota


Recommenders Systems

Hoop Dreams

Star Wars

Pretty Woman

Titanic Blimp Rocky XV

Joe D A B D ? ? John A F D F Susan A A A A A A Pat D A C Jean A C A C A Ben F A F Nathan D A A


Motivation

Citation web data can be used to effectively generate recommendations for the technical paper

“On the Recommending of Citations for Research Papers”, McNee, et al. (CSCW 2002)

“Enhancing Digital Libraries with TechLens+”, Torres, et al. (JCDL 2004)


Research Objectives

Design & Develop Personalized digital library services

Understand Users’ research interests


Research Questions

Can we utilize users’ personal citation collections to offer them personalized DL services?

Can citations in users’ personal collections be resolved to unique identifiers? How many of those do actually

resolve to a unique online identifier? How many of the resolved citations do actually lead to an online source for their content or metadata?


RefWorks


Citation Collections

RefWorks users 96 collections, 30,336 citations Two outliers (4000+ and 7000+)

316


Citation Types


User Profile

Users’ personal citation collections

Represent users’ profile Research interests, Research collaborations

Are single collections Related, Diverse

Are multiple collections Task based, Workgroup based


Citation Types

B

DS

N

J

R


0% 20% 40% 60% 80% 100%

Citation Types

Undergraduate

U=15 C=725

Graduate

U=64 C=11,751

Researcher

U=6 C=1941

Faculty

U=8 C=3626 D

B

J

R N

W


Resolvability

A citation is resolvable if it has A valid unique ID : DOI for articles, ISBN for books Enough information to resolve it to a unique ID

All citations that can be represented using a valid unique ID, are potentially resolvable.


External Resolvers

DOI and OpenURL Query Interfaces Citation resolvers at crossref.org (CR)

ISBN Query Interfaces Citation resolver at worldcat.org (WC)


Validity

A URL is valid if it leads to a citation’s source online

URLs : URL may or may not be unique

Validated existence of URL, not its accuracy

Did not attempt to retrieve ID for citation


DOI Resolvability

Citations

Resolved RW only

Both

CR only

Science/Technology

4,027 2,637 (65%)

10 100 2,527

Natural Resources 1,346 754 (56%) 4 76 674Business 518 283 (55%) 283Public Health 1,229 562 (46%) 2 58 502Medicine 2,326 1,033

(44%)2 109 922

Other 1,158 487 (42%) 12 81 394Social Sciences 3,015 1,216

(40%)1 59 1,156

Humanities 189 73 (39%) 2 14 57Public Affairs 552 158 (29%) 158Faculty 2,694 1,718

(64%)1 39 1,678

Researcher 1,584 798 (50%) 14 92 692Graduate 10,170 4,716

(46%)18 388 4,310

Undergraduate 562 234 (42%) 11 223Totals 15,010 7,466

(50%)33 530 6,903

0 0

0 0


ISBN Resolvability

Citations

Resolved RW only

Both

WC only

Other 98 83 (85%) 23 42 18Business 97 81 (84%) 13 38 30Public Affairs 258 215 (83%) 24 121 70Social Sciences 414 301 (73%) 70 173 58Science/Technology

236 164 (69%) 23 84 57

Humanities 78 53 (68%) 10 37 6Medicine 16 10 (63%) 4 6Natural Resources 51 22 (43%) 5 8 9Public Health 36 7 (19%) 1 6Graduate 916 713 (78%) 144 419 150Faculty 180 126 (70%) 11 31 84Undergraduate 53 36 (68%) 9 11 16Researcher 183 102 (56%) 19 68 15Totals 1,332 977

(73%)183 529 265


URL Validity

Citations

Valid

Humanities 13 13 (100%)Public Health 8 8 (100%)Medicine 2 2 (100%)Social Sciences 178 175 (98%)Natural Resources 71 69 (97%)Business 174 160 (92%)Public Affairs 95 77 (81%)Other 10 7 (70%)Science/Technology

62 31 (50%)

Graduate 441 417 (95%)Faculty 105 89 (85%)Researcher 63 34 (54%)Undergraduate 4 2 (50%)Totals 613 542

(88%)


Resolvability Overlap


Resolvability Summary

Number of resolved citations

Before After Gain

DOI 563 (4%) 7,466 (50%) 6,383 (43%)

ISBN 712 (53%) 977 (73%) 265 (20%)

URL 542 (88%) 542 (88%) N/A

Totals 2,337 (13%)

8,540 (47%)

6,203 (34%)8,540

(47%)


Very limited resolvers were used Additional resolvers such as the Citation Matcher from PubMed could enhance resolvability further

Dataset too small and diverse Difficult to find correlation among users CF based services work better with larger dataset

Privacy concerns Users want (a) control (b) anonymity

Limitations & Concerns


Future Work

Survey - Users’ willingness to share their personal collections

Understand how truly do users’ personal collections represent their profile?

Prototype of CF based DL serviceshttp://techlens.cs.umn.edu/ RecSys, Minneapolis, Oct 19-20, 2007


Acknowledgements

NSF grant IIS-0534939

RefWorks (http://www.refworks.com/)


A Study of Citations in Users’ Online Personal Collections

Nishikant Kapoor

Questions?

a study of citations in users’ online personal collections

Documents