usage statistics & information behaviors:
DESCRIPTION
Usage Statistics & Information Behaviors: Understanding User Behavior with Quantitative Indicators John McDonald Assistant Director for User Services & Technology Innovation The Libraries of the Claremont Colleges November 2, 2007 NISO Usage Data Forum. - PowerPoint PPT PresentationTRANSCRIPT
-
Usage Statistics & Information Behaviors: Understanding User Behavior with Quantitative Indicators
John McDonaldAssistant Director for User Services & Technology InnovationThe Libraries of the Claremont CollegesNovember 2, 2007NISO Usage Data Forum
-
Correlation: Boba Fett and Ladybugs
-
We have the data, now what do we do?What we have done: Cancel journals Inform purchase decisions
What we should do: Understand usage behaviors Guide our decision making processes Understand our impact on our patrons
-
Information Usage BehaviorsStartingBrowsingAccessingChainingDifferentiatingExtractingEllis (1993), Ellis & Haugan (1997) & Meho & Tibbo (2003), McDonald (2007)VerifyingNetworkingMonitoringManagingManipulatingTeachingEnding
-
AccessingManaging & EndingChaining & DifferentiatingAccessing & Browsing
-
How do we observe & measure? Pose a QuestionHow will a new service affect our users? Develop a Theory Explain what you think happened. Test the TheoryDevelop metrics, collect data, analyze.
-
Example 1: Starting & Accessing Question: How will a new service affect our users? Theory: If we improve the users ability to identify relevant material (starting) and retrieve it (accessing), we either save them time or effort and allow them to access more material. Test: There will be a significant increase in the usage of material.
-
Starting & Accessing: Use Before & After OpenURL*significant at .05 level **significant at .01 level
-
Example 2: Differentiating Question: Do our choices affect our users ability to differentiate between resources? Theory: If we group resources together, we allow users to identify relevant resources and provide efficient methods to differentiate between resources. Test: There is significant increase in searches across common resource groupings.
-
Differentiating: Federated Search Statistics
DatabaseSearchesWeb of Science3823OPAC3314WorldCat3267PubMed238INSPEC233MathSciNet183Faculty of 1000 Biology176Compendex132
-
Differentiating: OPAC Searches (2005 v. 2006)
Chart3
151894110040
138757100141
13841498020
9330091388
9334479814
7622967565
10083690665
114541100108
111670108868
10537787865
10023899968
2005
2006
Chart1
129061
151894
138757
138414
59288
93344
76229
100836
114541
111670
105377
100238
102745
110040
100141
98020
91388
79814
67565
90665
100108
111670
87865
99968
Months
Searches
USER SEARCHES IN CLAS
Chart2
1518940110040
1387570100141
138414098020
93300091388
93344079814
76229067565
100836090665
1145410100108
1116700108868
105377087865
100238099968
Sheet1
20052005200520052005200620062006
dec, jan, febjan, feb, marfeb, mar, aprmar, apr, mayapr, may, junjul, augjun, jul, augjul, aug, sepaug, sep, octsep, oct, novoct, nov, decnov, dec, jandec, jan, febjan, feb, marfeb, mar, aprmar, apr, mayapr, may, junmay, jun, juljun, jul, augjul, aug, sepaug, sep, octsep, oct, novoct, nov, decnov, dec, jan
Number of User Keyed Searches129,061151,894138,757138,41459,28893,34476,229100,836114,541111,670105,377100,238102,745110,040100,14198,02091,38879,81467,56590,665100,108111,67087,86599,968
Number of System Suggested Searches4,7435,4314,5552,9129281,4812,3782,4842,1682,1693,9816,1046,5215,0163,5743,1023,0302,2301,6843,6713,8022,1691,7283,489
Number of Records Retrieved23,163,11725,929,52026,001,27026,593,41711,870,32711,840,32917,416,27520,903,520-2,018,188,315-2,016,029,203-2,017,006,38121,543,41920,076,68231,308,90125,281,63623,862,67122,354,10919,077,87218,019,45518,614,53327,435,306-2,016,029,20347,608,03550,681,196
Number of Searches Limited1,4741,7241,5021,4356135117901,2091,3291,7711,5761,7401,5491,6811,2719315092782462332231,771231227
Number of Exported Records1,2911,1288648104339191,3061,5501,0501,3261,5651,4511,3141,3861,1777212,0522,2852,2596438091,3261,3513,665
Number of Displays Invoked226,132457,961453,725450,537123,59079,865130,249171,133190,367187,076179,154179,555184,048194,280173,384166,730153,584131,572110,632157,786172,469187,076138,469159,147
Jul-05Aug-05Sep-05Oct-05
336123361233,61247,31730,741
Dec-0443020
Jan-0543020
Feb-0543020
Mar-0565423
Apr-0530314
May-0542677dec, jan, febjan, feb, marfeb, mar, aprmar, apr, mayapr, may, junjul, augjun, jul, augjul, aug, sepaug, sep, octsep, oct, novoct, nov, decnov, dec, jan
Jun-05-13,703129,061151,894138,757138,41493,30093,34476,229100,836114,541111,670105,377100,238
Jul-0564,3704,7435,4314,5552,9129281,4812,3782,4842,1682,1693,9816,104
Aug-0525,56223,163,11725,929,52026,001,27026,593,41711,870,32711,840,32917,416,27520,903,520-2,018,188,315-2,016,029,203-2,017,006,38121,543,419
Sep-0510,9041,4741,7241,5021,4356135117901,2091,3291,7711,5761,740
Oct-0578,0751,2911,1288648104339191,3061,5501,0501,3261,5651,451
Nov-0522,691226,132457,961453,725450,537123,59079,865130,249171,133190,367187,076179,154179,555
Dec-054,611
Jan-0672,936200620062006
Feb-0625,198DecJanFebMarAprMayJunJulAugSeptOctNov
Mar-06102,745110,040100,14198,02091,38879,81467,56590,665100,108108,86887,86599,968
Apr-066,5215,0163,5743,1023,0302,2301,6843,6713,8023,9371,7283,489
May-0620,076,68231,308,90125,281,63623,862,67122,354,10919,077,87218,019,45518,614,53327,435,30640,123,55547,608,03550,681,196
Jun-061,5491,6811,271931509278246233223280231227
Jul-061,3141,3861,1777212,0522,2852,2596438091,1471,3513,665
Aug-06184,048194,280173,384166,730153,584131,572110,632157,786172,469185,082138,469159,147
Sep-06
Oct-06200520052005
Nov-06
Dec-06Point to Point Differential41,85438,61640,3941,91213,5308,66410,17114,4332,80217,512270
Jan-07as %28%28%29%2%14%11%10%13%3%17%0%
Sheet2
Sheet3
-
Differentiating: WorldCat Searches
Chart4
413207
672336
701323
478503
677295
497244
662422
659566
1331457
2041533
979408
1398288
2006
2005
Chart1
129061
151894
138757
138414
59288
93344
76229
100836
114541
111670
105377
100238
102745
110040
100141
98020
91388
79814
67565
90665
100108
111670
87865
99968
Months
Searches
USER SEARCHES IN CLAS
Chart2
1518940110040
1387570100141
138414098020
93300091388
93344079814
76229067565
100836090665
1145410100108
1116700108868
105377087865
100238099968
Chart3
151894110040
138757100141
13841498020
9330091388
9334479814
7622967565
10083690665
114541100108
111670108868
10537787865
10023899968
2005
2006
CLAS
20052005200520052005200620062006
dec, jan, febjan, feb, marfeb, mar, aprmar, apr, mayapr, may, junjul, augjun, jul, augjul, aug, sepaug, sep, octsep, oct, novoct, nov, decnov, dec, jandec, jan, febjan, feb, marfeb, mar, aprmar, apr, mayapr, may, junmay, jun, juljun, jul, augjul, aug, sepaug, sep, octsep, oct, novoct, nov, decnov, dec, jan
Number of User Keyed Searches129,061151,894138,757138,41459,28893,34476,229100,836114,541111,670105,377100,238102,745110,040100,14198,02091,38879,81467,56590,665100,108111,67087,86599,968
Number of System Suggested Searches4,7435,4314,5552,9129281,4812,3782,4842,1682,1693,9816,1046,5215,0163,5743,1023,0302,2301,6843,6713,8022,1691,7283,489
Number of Records Retrieved23,163,11725,929,52026,001,27026,593,41711,870,32711,840,32917,416,27520,903,520-2,018,188,315-2,016,029,203-2,017,006,38121,543,41920,076,68231,308,90125,281,63623,862,67122,354,10919,077,87218,019,45518,614,53327,435,306-2,016,029,20347,608,03550,681,196
Number of Searches Limited1,4741,7241,5021,4356135117901,2091,3291,7711,5761,7401,5491,6811,2719315092782462332231,771231227
Number of Exported Records1,2911,1288648104339191,3061,5501,0501,3261,5651,4511,3141,3861,1777212,0522,2852,2596438091,3261,3513,665
Number of Displays Invoked226,132457,961453,725450,537123,59079,865130,249171,133190,367187,076179,154179,555184,048194,280173,384166,730153,584131,572110,632157,786172,469187,076138,469159,147
Jul-05Aug-05Sep-05Oct-05
336123361233,61247,31730,741
Dec-0443020
Jan-0543020
Feb-0543020
Mar-0565423
Apr-0530314
May-0542677dec, jan, febjan, feb, marfeb, mar, aprmar, apr, mayapr, may, junjul, augjun, jul, augjul, aug, sepaug, sep, octsep, oct, novoct, nov, decnov, dec, jan
Jun-05-13,703129,061151,894138,757138,41493,30093,34476,229100,836114,541111,670105,377100,238
Jul-0564,3704,7435,4314,5552,9129281,4812,3782,4842,1682,1693,9816,104
Aug-0525,56223,163,11725,929,52026,001,27026,593,41711,870,32711,840,32917,416,27520,903,520-2,018,188,315-2,016,029,203-2,017,006,38121,543,419
Sep-0510,9041,4741,7241,5021,4356135117901,2091,3291,7711,5761,740
Oct-0578,0751,2911,1288648104339191,3061,5501,0501,3261,5651,451
Nov-0522,691226,132457,961453,725450,537123,59079,865130,249171,133190,367187,076179,154179,555
Dec-054,611
Jan-0672,936200620062006
Feb-0625,198DecJanFebMarAprMayJunJulAugSeptOctNov
Mar-06102,745110,040100,14198,02091,38879,81467,56590,665100,108108,86887,86599,968
Apr-066,5215,0163,5743,1023,0302,2301,6843,6713,8023,9371,7283,489
May-0620,076,68231,308,90125,281,63623,862,67122,354,10919,077,87218,019,45518,614,53327,435,30640,123,55547,608,03550,681,196
Jun-061,5491,6811,271931509278246233223280231227
Jul-061,3141,3861,1777212,0522,2852,2596438091,1471,3513,665
Aug-06184,048194,280173,384166,730153,584131,572110,632157,786172,469185,082138,469159,147
Sep-06
Oct-06200520052005
Nov-06
Dec-06Point to Point Differential41,85438,61640,3941,91213,5308,66410,17114,4332,80217,512270
Jan-07as %28%28%29%2%14%11%10%13%3%17%0%
WorldCat
DatabaseJanFebMarAprMayJunJulAugSepOctNovDec
2006413672701478677497662659133120419791398
2005207336323503295244422566457533408288
Sheet3
-
Example 3: Chaining Question: Do our users move from one information resource to another?Theory: If users are moving from resource to resource, usage of resources in the same environment (one provider) and results of that usage (citations) will increase. Test: There will be a significant increase in the usage and/or results of usage of a resources material.
-
Chaining: JSTOR Citations (2000 v. 2004)
Chart4
3442
3425
2023
3015
17
166
24
83
13
13
53
12
12
12
22
12
82
12
111
31
51
31
21
21
11
21
41
2000
2004
Chart1
129061
151894
138757
138414
59288
93344
76229
100836
114541
111670
105377
100238
102745
110040
100141
98020
91388
79814
67565
90665
100108
111670
87865
99968
Months
Searches
USER SEARCHES IN CLAS
Chart2
1518940110040
1387570100141
138414098020
93300091388
93344079814
76229067565
100836090665
1145410100108
1116700108868
105377087865
100238099968
Chart3
151894110040
138757100141
13841498020
9330091388
9334479814
7622967565
10083690665
114541100108
111670108868
10537787865
10023899968
2005
2006
CLAS
20052005200520052005200620062006
dec, jan, febjan, feb, marfeb, mar, aprmar, apr, mayapr, may, junjul, augjun, jul, augjul, aug, sepaug, sep, octsep, oct, novoct, nov, decnov, dec, jandec, jan, febjan, feb, marfeb, mar, aprmar, apr, mayapr, may, junmay, jun, juljun, jul, augjul, aug, sepaug, sep, octsep, oct, novoct, nov, decnov, dec, jan
Number of User Keyed Searches129,061151,894138,757138,41459,28893,34476,229100,836114,541111,670105,377100,238102,745110,040100,14198,02091,38879,81467,56590,665100,108111,67087,86599,968
Number of System Suggested Searches4,7435,4314,5552,9129281,4812,3782,4842,1682,1693,9816,1046,5215,0163,5743,1023,0302,2301,6843,6713,8022,1691,7283,489
Number of Records Retrieved23,163,11725,929,52026,001,27026,593,41711,870,32711,840,32917,416,27520,903,520-2,018,188,315-2,016,029,203-2,017,006,38121,543,41920,076,68231,308,90125,281,63623,862,67122,354,10919,077,87218,019,45518,614,53327,435,306-2,016,029,20347,608,03550,681,196
Number of Searches Limited1,4741,7241,5021,4356135117901,2091,3291,7711,5761,7401,5491,6811,2719315092782462332231,771231227
Number of Exported Records1,2911,1288648104339191,3061,5501,0501,3261,5651,4511,3141,3861,1777212,0522,2852,2596438091,3261,3513,665
Number of Displays Invoked226,132457,961453,725450,537123,59079,865130,249171,133190,367187,076179,154179,555184,048194,280173,384166,730153,584131,572110,632157,786172,469187,076138,469159,147
Jul-05Aug-05Sep-05Oct-05
336123361233,61247,31730,741
Dec-0443020
Jan-0543020
Feb-0543020
Mar-0565423
Apr-0530314
May-0542677dec, jan, febjan, feb, marfeb, mar, aprmar, apr, mayapr, may, junjul, augjun, jul, augjul, aug, sepaug, sep, octsep, oct, novoct, nov, decnov, dec, jan
Jun-05-13,703129,061151,894138,757138,41493,30093,34476,229100,836114,541111,670105,377100,238
Jul-0564,3704,7435,4314,5552,9129281,4812,3782,4842,1682,1693,9816,104
Aug-0525,56223,163,11725,929,52026,001,27026,593,41711,870,32711,840,32917,416,27520,903,520-2,018,188,315-2,016,029,203-2,017,006,38121,543,419
Sep-0510,9041,4741,7241,5021,4356135117901,2091,3291,7711,5761,740
Oct-0578,0751,2911,1288648104339191,3061,5501,0501,3261,5651,451
Nov-0522,691226,132457,961453,725450,537123,59079,865130,249171,133190,367187,076179,154179,555
Dec-054,611
Jan-0672,936200620062006
Feb-0625,198DecJanFebMarAprMayJunJulAugSeptOctNov
Mar-06102,745110,040100,14198,02091,38879,81467,56590,665100,108108,86887,86599,968
Apr-066,5215,0163,5743,1023,0302,2301,6843,6713,8023,9371,7283,489
May-0620,076,68231,308,90125,281,63623,862,67122,354,10919,077,87218,019,45518,614,53327,435,30640,123,55547,608,03550,681,196
Jun-061,5491,6811,271931509278246233223280231227
Jul-061,3141,3861,1777212,0522,2852,2596438091,1471,3513,665
Aug-06184,048194,280173,384166,730153,584131,572110,632157,786172,469185,082138,469159,147
Sep-06
Oct-06200520052005
Nov-06
Dec-06Point to Point Differential41,85438,61640,3941,91213,5308,66410,17114,4332,80217,512270
Jan-07as %28%28%29%2%14%11%10%13%3%17%0%
WorldCat Use
413207
672336
701323
478503
677295
497244
662422
659566
1331457
2041533
979408
1398288
2006
2005
WorldCat
DatabaseJanFebMarAprMayJunJulAugSepOctNovDec
2006413672701478677497662659133120419791398
2005207336323503295244422566457533408288
jstor
journal20002004
ECONOMETRICA3442
AM POLSCI REV3425
AM ECON REV2023
AM J OF POLSCI3015
MIND17
Q J OF ECON166
BR J FOR PHIL SCI24
J OF FIN83
J OF THE AM STAT ASSN13
PHIL REV13
J OF POL ECON53
INTRNL ECON REV12
PHIL OF SCI12
ANNALS OF MATH STATS12
PROC ROY SOC LNDN B22
J OF ECON PERSP12
REV OF ECON ST82
ANNALS OF STATS12
BRIT J OF POL SCI111
ECONOMIC J31
J OF BUSINESS51
J OF POLITICS31
SOC STUDIES OF SCI21
LEGIS STUD Q21
PROC AM MATH SOC11
REV OF ECON & STATS21
J OF PHILO41
-
Example 4: Managing, Teaching Question: Are our users managing or utilizing content differently?Theory: A stable online archive allows users to re-access or re-use content more efficiently (utility usage or virtual vertical file), or utilize it for instructional purposes in different ways (virtual syllabus). Test: There will be a significant increase in the systematic re-use of current, locally produced content.
-
Managing, Teaching: Use of local content
-
Example 5: Service EffectsQuestion: How do our choices in libraries affect user behavior?Theory: When we change the display options (e.g. cataloging) for journals, did that affect either publisher usage or SFX usage?Test: Changing cataloging results in decreased local journal usage as measured by the publisher and SFX.
-
Service Effects: Usage of Journals (2005 v. 2006)
Chart2
668580
625587
587726
603538
656663
660587
487773
527575
461511
523676
579648
440407
2005
2006
Chart1
3315850204503
800126405571082
956136706931218
742129704761046
763133904941084
98811180692897
83311950540942
81010570546872
7359680464733
88310880608823
7258020493558
3995140209371
6789990435743
Cancelled
48 List
Cancelled
48 List
Clickthrough Rate
0.61631419940.8598290598
0.696250.8560126582
0.72489539750.8910021946
0.6415094340.8064764842
0.64744429880.8095593727
0.70040485830.8023255814
0.64825930370.7882845188
0.67407407410.8249763482
0.6312925170.757231405
0.68856172140.7564338235
0.680.6957605985
0.52380952380.7217898833
0.64159292040.7437437437
Clickthrough Rate
Sheet1
LCCNCountOfPrint ISSNSumOfJan-2005SumOfFeb-2005SumOfMar-2005SumOfApr-2005SumOfMay-2005SumOfJun-2005SumOfJul-2005SumOfAug-2005SumOfSep-2005SumOfOct-2005SumOfNov-2005SumOfDec-2005
Cancelled87668625587603656660487527461523579440
48 List46256625832682232625462215231125362313237922941957
87JanFebMarAprMayJunJulAugSepOctNovDec
Cancelled46580587726538663587773575511676648407
48 List271027643091247926022365243423942553276823241952
Usage Change-13.2%-6.1%23.7%-10.8%1.1%-11.1%58.7%9.1%10.8%29.3%11.9%-7.5%
Winter97.9%
Fall
SumOf01-RequestsSumOf02-RSumOf03-RSumOf04-RSumOf05-RSumOf06-RSumOf07-RSumOf08-RSumOf09-RSumOf10-RSumOf11-RSumOf12-RSumOf13-R
Cancelled331800956742763988833810735883725399678
48 List58512641367129713391118119510579681088802514999
SumOf01-ClickthroughsSumOf02-CSumOf03-CSumOf04-CSumOf05-CSumOf06-CSumOf07-CSumOf08-CSumOf09-CSumOf10-CSumOf11-CSumOf12-CSumOf13-C
Cancelled204557693476494692540546464608493209435
48 List5031082121810461084897942872733823558371743
61.6%69.6%72.5%64.2%64.7%70.0%64.8%67.4%63.1%68.9%68.0%52.4%64.2%
86.0%85.6%89.1%80.6%81.0%80.2%78.8%82.5%75.7%75.6%69.6%72.2%74.4%
Sheet2
Sheet3
-
Service Effects: SFX Clickthrough Rate (Local v. Shared)
Chart2
0.61631419940.8598290598
0.696250.8560126582
0.72489539750.8910021946
0.6415094340.8064764842
0.64744429880.8095593727
0.70040485830.8023255814
0.64825930370.7882845188
0.67407407410.8249763482
0.6312925170.757231405
0.68856172140.7564338235
0.680.6957605985
0.52380952380.7217898833
0.64159292040.7437437437
SCELC Titles
CIT Titles
Chart1
3315850204503
800126405571082
956136706931218
742129704761046
763133904941084
98811180692897
83311950540942
81010570546872
7359680464733
88310880608823
7258020493558
3995140209371
6789990435743
Cancelled
48 List
Cancelled
48 List
Chart3
7.67816091956.6666666667
7.1839080466.7471264368
6.74712643688.3448275862
6.93103448286.183908046
7.54022988517.6206896552
7.58620689666.7471264368
5.59770114948.8850574713
6.05747126446.6091954023
5.29885057475.8735632184
6.01149425297.7701149425
6.65517241387.4482758621
5.05747126444.6781609195
2005
2006
Sheet1
LCCNCountOfPrint ISSNSumOfJan-2005SumOfFeb-2005SumOfMar-2005SumOfApr-2005SumOfMay-2005SumOfJun-2005SumOfJul-2005SumOfAug-2005SumOfSep-2005SumOfOct-2005SumOfNov-2005SumOfDec-2005
Cancelled877.77.26.76.97.57.65.66.15.36.06.75.1
48 List4655.825832682232625462215231125362313237922941957
Use per JournalJanFebMarAprMayJunJulAugSepOctNovDec
Cancelled876.76.78.36.27.66.78.96.65.97.87.44.7
48 List46271027643091247926022365243423942553276823241952
Usage Change-13.2%-6.1%23.7%-10.8%1.1%-11.1%58.7%9.1%10.8%29.3%11.9%-7.5%
Winter97.9%
Fall
SumOf01-RequestsSumOf02-RSumOf03-RSumOf04-RSumOf05-RSumOf06-RSumOf07-RSumOf08-RSumOf09-RSumOf10-RSumOf11-RSumOf12-RSumOf13-R
Cancelled331800956742763988833810735883725399678
48 List58512641367129713391118119510579681088802514999
SumOf01-ClickthroughsSumOf02-CSumOf03-CSumOf04-CSumOf05-CSumOf06-CSumOf07-CSumOf08-CSumOf09-CSumOf10-CSumOf11-CSumOf12-CSumOf13-C
Cancelled204557693476494692540546464608493209435
48 List5031082121810461084897942872733823558371743
61.6%69.6%72.5%64.2%64.7%70.0%64.8%67.4%63.1%68.9%68.0%52.4%64.2%
86.0%85.6%89.1%80.6%81.0%80.2%78.8%82.5%75.7%75.6%69.6%72.2%74.4%
668625587603656660487527461523579440
256625832682232625462215231125362313237922941957
JanFebMarAprMayJunJulAugSepOctNovDec
580587726538663587773575511676648407
271027643091247926022365243423942553276823241952
Sheet2
Sheet3
-
Example 5: Services Related BehaviorsWhat else do users want or need? Are there services related behaviors that we can observe? Providing content is one option, but how are researchers using associated information services? If we provide them the article they want in fulltext, we see that sometimes they ask for other types of things. Can we match these things to those user behaviors?
-
Services Related Behaviors
Information Service RequestsOrder Article via Document Delivery951See References for this Article790Search Library Catalog580Read Abstract283Search Article Title on the Web170Send Feedback to Library15See Articles citing this Article11
-
What else could we be studying?Monitoring Many information providers have e-alerts, repeat saved searches, etc. Networking Users may want to email a citation to a colleague or another student.Extracting Passing the bibliographic information to another database to search.Analyzing Including user behavior information in the statistical measurement tools.
-
Questions?John McDonaldNovember 2, 2007
*Good morning and Id like to thank you all for coming. My name is John McDonald and I am the Assistant Director for User Services & Technology Innovation at the Libraries of the Claremont Colleges. Im pleased that the forum organizers have invited me to present about committee for this position has invited me to campus today for this interview.
Today Im going to talk about the Librarys Dilemma: The Future of Innovative Library Services in the Academic Environment. Ill be covering a lot here today, but feel free to ask questions at any time.*The Librarys Dilemma is a phrase that Ive co-opted from the Innovators Dilemma, a book by Clayton Christensen, where he argues that disruptive technologies help new companies bring about products that can challenge and eventually replace established products and businesses. The dilemma is how does the established company or business innovate, while still producing their products or services, in order to avoid being replaced by new companies that do not have the same established product to continue to sell.
This chart is one Ive adjusted from a presentation by Gary Flake, a microsoft researcher, who was writing about Internet singularity. He was comparing the offline world to the online world, but Ive adjusted it to compare the Library Environment to the Internet Environment, relative to academic research. You can see the dichotomy between each of these problematic points. For libraries to innovate, we have huge costs, in real dollars, personnel, or space. Web applications have no such costs.
The tail of the librarys content is limited by our space, our selection criteria and mission statements, and our organization schemes. On the web, the tail is theoretically unlimited, even if it is functionally limited by the available retrieval systems and abilities of users.
In libraries, to innovate, we usually need to work harder, since new and better services or systems require more employees or require those employees to develop new skills. On the web, innovation is driven by working smarter using technological advances to improve the efficiency of the worker or user.
To provide quality products and services, libraries typically have to have high intervention much manual labor and interaction with our users or our resources to design innovative services. On the web, quality is improved through technology developed outside the information world or through integrating tools and services.
And finally, in libraries innovation follows demand. And quite often, thats great demand, or demand followed by financial support, or the demand is so great that it imposes innovation. On the web, innovation predicts and precedes demand. Innovation is typically driven by single users and can be developed, tested, and then accepted or rejected with few costs to the network.
The solution to this dilemma is for the library to move along a continuum on each of these points from the left to the right. And the challenge is how to do this while still serving our users to the same standard as we have in the past.
*The Librarys Dilemma is a phrase that Ive co-opted from the Innovators Dilemma, a book by Clayton Christensen, where he argues that disruptive technologies help new companies bring about products that can challenge and eventually replace established products and businesses. The dilemma is how does the established company or business innovate, while still producing their products or services, in order to avoid being replaced by new companies that do not have the same established product to continue to sell.
This chart is one Ive adjusted from a presentation by Gary Flake, a microsoft researcher, who was writing about Internet singularity. He was comparing the offline world to the online world, but Ive adjusted it to compare the Library Environment to the Internet Environment, relative to academic research. You can see the dichotomy between each of these problematic points. For libraries to innovate, we have huge costs, in real dollars, personnel, or space. Web applications have no such costs.
The tail of the librarys content is limited by our space, our selection criteria and mission statements, and our organization schemes. On the web, the tail is theoretically unlimited, even if it is functionally limited by the available retrieval systems and abilities of users.
In libraries, to innovate, we usually need to work harder, since new and better services or systems require more employees or require those employees to develop new skills. On the web, innovation is driven by working smarter using technological advances to improve the efficiency of the worker or user.
To provide quality products and services, libraries typically have to have high intervention much manual labor and interaction with our users or our resources to design innovative services. On the web, quality is improved through technology developed outside the information world or through integrating tools and services.
And finally, in libraries innovation follows demand. And quite often, thats great demand, or demand followed by financial support, or the demand is so great that it imposes innovation. On the web, innovation predicts and precedes demand. Innovation is typically driven by single users and can be developed, tested, and then accepted or rejected with few costs to the network.
The solution to this dilemma is for the library to move along a continuum on each of these points from the left to the right. And the challenge is how to do this while still serving our users to the same standard as we have in the past.
*Starting: activities associated with the initiation of information seekingBrowsing: scanning for information in areas of interest or near relevant itemsAccessing: physical & intellectual act of locating & acquiring informationChaining: following chains of citations and hyperlinksDifferentiating: using differences between resources to filter informationExtracting: identifying and selecting information systematically from a sourceVerifying: using other resources to establish the authenticity of informationNetworking: communicating and interacting with others to build information archives, gather information, and share informationMonitoring: keeping up to date with information in the areaManaging: filing, organizing, and storing information for later use or re-useManipulating: re-use and re-purposing of the data, or in Web 2.0 terms, mash-upsEnding: making sure nothing was missed earlier, or finalizing the projectOne set of methods is to match our information systems and the metrics they produce to these behaviors. For weve designed these based on research, observation, and through analysis of our researchers needs and expectations.**The first question is What was the impact of a new access tool on resource usage? Specifically, I wondered what affect SFX had on the usage of journals in our collection. I speculated that the increase in discoverability and accessibility provided by SFX led to our faculty and students using journals more heavily than they had in the past. The test of this hypothesis is that there is no relationship between the release of the resolver and journal usage.*The first question is What was the impact of a new access tool on resource usage? Specifically, I wondered what affect SFX had on the usage of journals in our collection. I speculated that the increase in discoverability and accessibility provided by SFX led to our faculty and students using journals more heavily than they had in the past. The test of this hypothesis is that there is no relationship between the release of the resolver and journal usage.*This table shows the results of this analysis. Journals were grouped into 9 broad subject categories and I analyzed publisher provided online journal usage for our collection from before we had the resolver (in 2000) to after we had it (2002). The results were significant. The negative and large z-scores indicate that usage was higher after SFX was released and the results were significant for almost all subject areas at the .05 or .01 level. The three disciplines that had no signficance to the increase in usage were the 3 with the fewest number of journals with publisher provided usage.
It is clear that SFX led to more article downloads by our faculty and students.*Further extending the theme of the effect of one decision on other resources, I wondered what the impact of a new access tool might have on the usage of other access tools. Specifically I wondered what the impact of resource groupings in the quicksets, subject categories might have on usage of those resources. Metalib is a powerful discovery and access tool and the metasearch functionality exposes more content to the user and promotes resources that might otherwise have been overlooked.*This table shows the stats for Metalib since its public release. Its interesting to note that WoS, CLAS, and WorldCat are virtually identical. As you know, those are the three databases grouped into the default Basic resources quickset. So, it led me to wonder how the use of Metalib and its return of records *Here is an illustration of the number of searches in CLAS in 3 month blocks beginning with the month on the x-axis. The blue line is the searches in 2005 while the orange line is 2006. Generally, this shows a drop in total searches in CLAS, maybe explainable by the Google effect or lower reliance on books by our community. But, more interestingly is that the trendlines indicate that the searches for 2006, while initially lagging well behind the 2005 rate until an increase in august and a spike in September. By November, the search rate had returned to previous levels. In September we released Metalib and this could explain some of the effect.*Here is an illustration of the number of searches in WorldCat for 2005 and 2006. The blue line is the searches in 2005 while the orange line is 2006. Generally, this shows an increase in searches for 2006, but a drastic jump in searches starting in September and continuing into October, November and December. Again, this mirrors the release of Metalib and indicates that the positive recommending of both CLAS and WorldCat as part of the default Basic Quickset.
DatabaseJanFebMarAprMayJunJulAugSepOctNovDec20064136727014786774976626591331204197913982005207336323503295244422566457533408288
*The first question is What was the impact of a new access tool on resource usage? Specifically, I wondered what affect SFX had on the usage of journals in our collection. I speculated that the increase in discoverability and accessibility provided by SFX led to our faculty and students using journals more heavily than they had in the past. The test of this hypothesis is that there is no relationship between the release of the resolver and journal usage.*The final example that I wanted to discuss today is the effect of adding JSTOR on our faculty information usage behavior, specifically its affect on citations on Caltech authored publications. We bought JSTOR in 2002, so I selected articles published in 2000 (in blue) compared to those published in 2004 (in orange). Overall, the test of the entire set of journals in the group was not significant since there were only 51 articles in each sample year with about 200 citations to JSTOR journals each year. There were a total of 27 journals from JSTOR cited in both samples, and this chart illustrates the ranked values of the citations plus a powerlaw trend line. This illustration indicates the the most popular journals were positively affected by provision of JSTOR, while there was no effect on the little used titles. *The first question is What was the impact of a new access tool on resource usage? Specifically, I wondered what affect SFX had on the usage of journals in our collection. I speculated that the increase in discoverability and accessibility provided by SFX led to our faculty and students using journals more heavily than they had in the past. The test of this hypothesis is that there is no relationship between the release of the resolver and journal usage.*The first question is What was the impact of a new access tool on resource usage? Specifically, I wondered what affect SFX had on the usage of journals in our collection. I speculated that the increase in discoverability and accessibility provided by SFX led to our faculty and students using journals more heavily than they had in the past. The test of this hypothesis is that there is no relationship between the release of the resolver and journal usage.*Example 2 is the effect that Metadata notation has on information usage.
In July 2006, we concluded negotiations with Wiley for our site license to their journals for 2006 and 2007. We had access to a shared title list through the SCELC consortia, but our new license was considered outside the consortia by Wiley and thus our shared title access ended. We changed the SFX thresholds on July 1st, 2006. Users would not be presented with a Fulltext link for those 200+ titles that we no longer had 2006 access to, although links to the 1997-2005 material was still presented.
Fortunately for us, Wiley never stopped online access to most titles. So this provides an interesting live experiment in how library service decisions affect usage behaviors, particularly through the previously mentioned access and discovery tools. I would expect, from the prior research, that reducing the active presentation of the links to 2006 material reduced the number of downloads of Wiley articles and the amount of use recorded through SFX.*This Chart shows the usage, as reported by Wiley, of their Journals by us in 2005 and 2006. It is limited to just the SCELC shared list that we no longer have a valid subscription for. Its a little hard to see, but the Blue lines are the 2005 usage and a linear trendline. Compare that with the 2006 usage, in orange. Youll notice a huge spike in July in 2006 usage. Thats contrary to our theory that our use should have dropped. This was because Melody individually checked access to each journal to verify that Wiley was not reducing our valid access. But more important is the trend that shows that while the usage does trend down, it should be dropping faster, and 2006 should be less than 11% more than 2005 (We expect usage to increase about 11% per year as additional years of online material were added to the database). Therefore, our lack of access to 2006 material should have affected our overall stats. It didnt because Wiley didnt really reduce our access.*This is just 2006 represented, with the blue being the SCELC titles and Orange our titles. Here we see a similar trend, but without the spike in July (since Melody wasnt using SFX to get to the PDFs). We see a general downward trend in the Clickthrough Rate. I used Clickthrough rate because that signifies a user clicking from the Menu to the fulltext. We would expect this to go down, since there are fewer fulltext presentations since 2006 wasnt being offered. And indeed, the rate dropped in July and never returned to the previous levels. This indicates that the lack of presentation of FT affected SFX users and their usage of these titles. Contrast this to the previous slide where we found that Wiley reported usage didnt suffer to the extent that we thought and a clearer picture exists that users are still getting articles, even when we dont indicate ownership.*The first question is What was the impact of a new access tool on resource usage? Specifically, I wondered what affect SFX had on the usage of journals in our collection. I speculated that the increase in discoverability and accessibility provided by SFX led to our faculty and students using journals more heavily than they had in the past. The test of this hypothesis is that there is no relationship between the release of the resolver and journal usage.*The first question is What was the impact of a new access tool on resource usage? Specifically, I wondered what affect SFX had on the usage of journals in our collection. I speculated that the increase in discoverability and accessibility provided by SFX led to our faculty and students using journals more heavily than they had in the past. The test of this hypothesis is that there is no relationship between the release of the resolver and journal usage.*Any questions, thoughts, or reasons to call me crazy?