libqual+ ® 2007: an introduction american library association (ala) annual conference washington,...
TRANSCRIPT
LibQUAL+LibQUAL+®® 2007: An Introduction 2007: An Introduction
American Library Association (ALA)Annual Conference
Washington, DCJune 25, 2007
MaShana Davis and Martha KyrillidouAssociation of Research Libraries
old.libqual.org
old.libqual.org
OverviewOverview
• Research Foundation and Current Uses of LibQUAL+®
• Preparing to Implement the Survey
• The Survey Process: Initial Steps
• Mapping Disciplines
• Approving and Running Your Survey
• General Discussion and Q&A
old.libqual.org
LibQUAL+LibQUAL+®® Background and OverviewBackground and Overview
old.libqual.org
Reference TransactionsReference Transactions
old.libqual.org
AssessmentAssessment
“The difficulty lies in trying to find a single model or set of simple indicators that can be used by different institutions, and that will compare something across large groups that is by definition only locally applicable—i.e., how well a library meets the needs of its institution. Librarians have either made do with oversimplified national data or have undertaken customized local evaluations of effectiveness, but there has not been devised an effective way to link the two.”
Sarah Pritchard, Library Trends, 1996
old.libqual.org
Multiple MethodsMultiple Methodsof Listening to Customersof Listening to Customers• Transactional surveys*• Mystery shopping• New, declining, and lost-customer surveys• Focus group interviews• Customer advisory panels• Service reviews• Customer complaint, comment, and inquiry capture• Total market surveys*• Employee field reporting• Employee surveys• Service operating data capture
*A SERVQUAL-type instrument is most suitable for these methods
Note. A. Parasuraman. The SERVQUAL Model: Its Evolution And Current Status. (2000).
Paper presented at ARL Symposium on Measuring Service Quality, Washington, D.C.
old.libqual.org
World World LibQUALLibQUAL++®® Survey Survey
old.libqual.org
Premises
Three Seminal Quotations
old.libqual.org
PERCEPTIONS SERVICE “….only customers judge quality;
all other judgments are essentially
irrelevant”
Note. Zeithaml, Parasuraman, Berry. (1999). Delivering quality service. NY: The Free Press.
LibQUAL+™ Premise #1
old.libqual.org
LibQUAL+™ Premise #2
“Il est plus nécessaire d'étudier
les hommes que les livres”
—FRANÇOIS DE LA ROCHEFOUCAULD
old.libqual.org
“We only care about the things we measure.”
--Bruce Thompson, CASLIN, 2006
LibQUAL+™ Premise #3
old.libqual.org
DimensionsDimensions
2000 2001 2002 2003-200741 items 56 items 25 items 22 items
Affect of Service Affect of Service Affect of Service Affect of Service
Library as Place Library as Place Library as Place Library as Place
Reliability Reliability Personal ControlInformation Control
Provision of Physical Collections
Self-RelianceInformation Access
Access to Information
Access to Information
old.libqual.org
Interpreting Service Quality Data
Three Interpretation Frameworks
old.libqual.org
Benchmarking Against Peer Institutions
--1,000,000 Users; 1,000 Institutions!
NORMS! NORMS! NORMS!
Interpretation Framework #1
old.libqual.org
Score Norms
• Norm Conversion Tables facilitate the interpretation of observed scores using norms created for a large and representative sample.
• LibQUAL+™ norms have been created at both the individual and institutional level
old.libqual.org
Institutional Norms for PerceivedMeans on 25 Core Questions
Note: Thompson, B. LibQUAL+ Spring 2002 Selected Norms, (2002).
old.libqual.org
Benchmarking Against Self, Longitudinally
“Nobody is more like me than me!”--Anonymous
Interpretation Framework #2
old.libqual.org
Interpreting Perceived Scores Against Minimally-Acceptable and Desired
Service Levels (i.e., “Zones of Tolerance”)
Interpretation Framework #3
old.libqual.org
old.libqual.org
old.libqual.org
Survey Structure Survey Structure (Detail View)(Detail View)
old.libqual.org
Rapid Growth in Other AreasRapid Growth in Other Areas• Languages
– Afrikaans– American English– British English– Danish– Dutch – Finnish– French– German– Norwegian– Swedish
• Consortia*Each may create 5 local questions to add
to their survey
• Countries– Australia, Canada, Denmark, Egypt,
Finland, France, Hong Kong, New Zealand, the Netherlands, Norway, South Africa, Sweden, Switzerland, UAE, U.K., U.S.
• Types of Institutions– Academic Health Sciences– Academic Law– Academic Military– College or University– Community College– Electronic– European Business– European Parliament– Family History– Research Centers (FFRDC) Libraries– High School (2007)– Hospital– National Health Service England– Natural Resources– New York Public– Public– Smithsonian– State– University/TAFE
old.libqual.org
250
218
307308
204
164
13
43
152,111
176,360
151,460
113,480
78,863
4,407
20,416
128,958
0
50
100
150
200
250
300
350
2000 2001 2002 2003 2004 2005 2006 2007
Survey Year
Number ofInstitutions
0
20,000
40,000
60,000
80,000
100,000
120,000
140,000
160,000
180,000
200,000
Number ofResponses
Number of Institutions
Number of Responses
LibQUALLibQUAL++®® Participants Participants
* 2007 data reflects Session I data only
old.libqual.org
LibQUALLibQUAL++®® First Year Participants First Year Participants
* 2007 data reflects Session I data only
old.libqual.org
LibQUALLibQUAL++®® Surveys by Type Surveys by Type Year
Type 2000 2001 2002 2003 2004 2005 2006 2007
Academic Health Sciences 1 35 23 13 13 10 8
Academic Law 1 25 10 6 2
Academic Military 6 1 1
College or University 13 41 111 244 150 201 226 177
Canadian Government 6
Community College 16 29 3 15 27 18
Electronic 1
European Business 5 16
Family History 1 2
Hospital 10 1 1
National Health Services England 10
Natural Resources 4
New York Public 1
Public 4 1 1 3 2
Research Centers (FFRDC) Libraries 5 2 1
Smithsonian 1 1
State 1 1 3 2
University/TAFE 2 1 1
* 2007 data reflects Session I data only
old.libqual.org
LibQUALLibQUAL++®® Languages Languages
American English
Dutch EnglishFrench Canadian DutchSwedish
Swedish(British English)
Afrikaans
DanishFinnishGerman Norwegian
British English
Continental French
old.libqual.org
LibQUALLibQUAL++®® Surveys by Language Surveys by Language Year
Language 2000 2001 2002 2003 2004 2005 2006 2007
Afrikaans 4 1
American English 13 42 164 285 176 207 236 179
British English 20 22 31 50 18
Danish 1
Dutch 1 2
Dutch English 1 1 2
Finnish 1
French Canadian 2 1 4 20
French Continental 1 1
German 1
Norwegian 1
Swedish 5 2 1 1
Swedish British English 1
Swedish English (A.E.) 2 1
* 2007 data reflects Session I data only
old.libqual.org
LibQUALLibQUAL++®® Surveys by Consortia Surveys by Consortia Year
Consortium 2000 2001 2002 2003 2004 2005 2006 2007
AAHSL 1 35 21 14 12 7 8
Alabama Academic (NAAL) 10 1 8 2 11
AJCU - Academic Libraries 20 21
AJCU - Law Libraries 1
Connecticut State University Library System (CONSULS) 5
Consortium of Church Libraries and Archives 7
CUC Group (formerly CES) 6 8 1
CUNY Libraries 19
Department of Justice Canada Libraries 6 17
European Business Schools Librarians Group
Georgia Consortium 19 1
Harrisburg Area Community Colleges 5
Hospital/MLA 7
Keystone Library Network 15
LibQUAL Canada 63
* 2007 data reflects Session I data only
old.libqual.org
LibQUALLibQUAL++®® Surveys by Consortia Surveys by Consortia(cont’d)(cont’d)
Year
Consortium 2000 2001 2002 2003 2004 2005 2006 2007
Maine URSUS Libraries 13
Massachusetts LSTA 5
MCCLPHEI 14
MERLN (Military Education & Research Library Network) 6
National Health Service England 10
New England Law Library Consortium (NELLCO) 8
North Carolina Community Colleges 15
NY3Rs College and University Libraries 71 2 1 1
NY3Rs Public Libraries 5
Oberlin Libraries Group 12 9 13 2 1
OhioLINK 57 45 1 14 2 6
Research Centers (FFRDC) Libraries
SCONUL 20 17 16 20 14
State University Libraries of Florida 6 2 1 1
University of Wisconsin System 14
VALE 12 1
* 2007 data reflects Session I data only
old.libqual.org
Participating Libraries by CountryParticipating Libraries by Country Year
Country 2000 2001 2002 2003 2004 2005 2006 2007
Australia 1 6 2 1
Canada 1 3 4 8 10 15 11 63
Denmark 1 2
Egypt 1 2 1
Finland 2
France 1 2 1
Ireland 1 1 2 1
Mexico 1
Netherlands 1 1 5
New Zealand 1 1
Norway 2
South Africa 12 8 2
Sweden 3 4 4 2
Switzerland 2 2
U.A.E. 1
U.K. 20 17 16 33 14
* 2007 data reflects Session I data only
old.libqual.org
Surveys by Session: 2004-2007Surveys by Session: 2004-2007
Year Session 1 Session 2
2004 202 2
2005 199 56
2006 205 93
2007 218
old.libqual.org
Preparing for the Survey Preparing for the Survey ImplementationImplementation
old.libqual.org
Why is Your Library Participating Why is Your Library Participating in in LibQUAL+LibQUAL+® ® ??
• Institutional goals:
– what do you want to get out of the survey?
• Survey requirements:
– people, supplies, technology
old.libqual.org
Get Permission from your Get Permission from your Institutional Review BoardInstitutional Review Board
• Local group that approves human subject research
• May also be called
– Committee for Human Subject Research
– Committee for the Protection of Human Subjects
• Not all institutions need to obtain permission
old.libqual.org
Get Permission from yourGet Permission from yourInstitutional Review BoardInstitutional Review Board
• If permission is required:
– Seek well in advance of survey
– Supply a copy of the survey, if requested
– Inform them that results will be shared among participants
• No need to inform LibQUAL+® of decision
old.libqual.org
Determine Whom to SurveyDetermine Whom to Survey
• Random sample
or
• Entire population
old.libqual.org
If You Sample…If You Sample…
• Recommendations: – At least 1,200 random email addresses for
each user group– Separate sample groups for
undergraduates, graduates, faculty, and staff
– Think about survey fatigue if you want to repeat the survey regularly
• Keep notes on your methodology as you will be asked to describe it in the Post Hoc Questionnaire
old.libqual.org
If You Survey the If You Survey the Entire Population….Entire Population….
• Recommendations:
– Think about survey fatigue if you want to repeat the survey regularly
• Keep notes on your methodology as you will be asked to describe it in the Post Hoc Questionnaire
old.libqual.org
Obtain Email AddressesObtain Email Addresses
• Typical sources include:
– Campus computing office
– Campus administrative records office
– Library patron database
old.libqual.org
Marketing Your SurveyMarketing Your Survey
– Place ads in campus newspaper
– Write article for library newsletter
– Post flyers around campus
– Present at faculty meetings & student orientations
– Create a survey Web site and feature on library’s home page
– Take the survey to where users are: dining halls, study rooms, dorms
old.libqual.org
Become Familiar with Become Familiar with LibQUAL+LibQUAL+®® Resources Resources
• Management Centerhttp://www.libqual.org/Manage/index.cfm
• LibQUAL+® Procedures Manualhttp://www.libqual.org/Manual/index.cfm – Updated Version – September 2006– Much more detail
• Discussion [email protected]
old.libqual.org
The The LibQUAL+LibQUAL+®® Management CenterManagement Center
old.libqual.org
The Survey Process: Initial StepsThe Survey Process: Initial Steps
old.libqual.org
Survey Process: Manage Your SurveySurvey Process: Manage Your Survey
Eight Sections:
• Preferences
• Customization
• Preview
• Representativeness
• Monitor Survey Progress
• Incentive Winners
• Post Hoc
• Evaluation
*Sections must be completed in order
old.libqual.org
PreferencesPreferences
Fields marked with * are required
old.libqual.org
Customization: Optional/Local Customization: Optional/Local QuestionsQuestions
old.libqual.org
Customization: DisciplinesCustomization: Disciplines
• Results notebooks summarize findings by user group and provide a chart for, both, standard and custom disciplines
• Standard disciplines (based on your institution type, i.e., College/University)
• Customized disciplines
– Recommend no more than 16 disciplines, if possible
old.libqual.org
The Standard Disciplines – The Standard Disciplines – College/UniversityCollege/University• Agriculture/Environmental
Studies• Architecture• Business• Communications/
Journalism• Education• Engineering/Computer
Science• General Studies• Health Sciences
• Humanities• Law• Military/Naval Science• Other• Performing & Fine Arts• Science/Math• Social Sciences/
Psychology• Undecided
old.libqual.org
Customized DisciplinesCustomized Disciplines
• Use your local terminology to map to the standard disciplines
• Cautions:
– Need to provide representativeness data for each discipline
– Too many choices present challenges to users
old.libqual.org
Standard Disciplines: Standard Disciplines: An Academic LibraryAn Academic Library
old.libqual.org
Customized Disciplines: Customized Disciplines: An Academic LibraryAn Academic Library
old.libqual.org
Customization: Branch LibrariesCustomization: Branch Libraries
The library you use most often.
old.libqual.org
Approving and Running Approving and Running Your SurveyYour Survey
old.libqual.org
Previewing and ApprovingPreviewing and ApprovingYour SurveyYour SurveyPreview• Complete at least one full run
of your preview survey• Test in different settings,
using different platforms and Web browsers
• Get library staff involved in testing
Approved• Can no longer make changes• Live survey URL will appear at
the top of the “Manage Your Survey” page marked as Survey URL
old.libqual.org
Reviewing CommentsReviewing Comments
• About one-half of users include comments on their survey
• Download via the Monitor Survey Progress page (Excel file or text file)http://old.libqual.org/Manage/Results/Comments/index.cfm
old.libqual.org
Sending RemindersSending Reminders
• 3-5 reminders to sample populations
• Thank respondents who have completed the survey
• Boost up marketing efforts around campus– More fliers, table tents, ads in campus
newspaper– Get professors and other staff involved– Increase number or types of incentives
* More examples of marketing efforts featured at the Share Fair
old.libqual.org
Completing Your Completing Your Representativeness QuestionnaireRepresentativeness Questionnaire• Determines how your institutional profile compares to
your survey data• Requires the following information:
– # of individuals per user group– # of individuals within each discipline– # of males and females– Library Statistics
• Volumes added during the year – Gross/Total (including e-books)• Total number of current serials received (including electronic serials)• Total library expenditures (U.S. dollars)• Personnel – professional staff, full-time equivalent (FTE)• Personnel – support staff, full-time equivalent (FTE)
• Must be complete BEFORE closing survey
old.libqual.org
Completing Your Completing Your Representativeness QuestionnaireRepresentativeness Questionnaire
Representativeness Representativeness CompletedCompleted
Representativeness Representativeness NOT CompletedNOT Completed
old.libqual.org
Closing Your SurveyClosing Your Survey
• We recommend a survey run of at three weeks
• Once you close your survey:– Retrieve list of incentive winners– Fill out Post Hoc and Evaluation
Questionnaires
old.libqual.org
Completing Post Hoc and Evaluation Completing Post Hoc and Evaluation QuestionnairesQuestionnaires
Post Hoc Questionnaire• Information about your
survey– Sample size– # of e-mails sent– #of invalid e-mail
addresses– Incentives offered– Marketing techniques– Etc.
Evaluation Questionnaire• Feedback about your
LibQUAL+® experience• All survey liaisons and
assistants are encouraged to complete this questionnaire
old.libqual.org
General Discussion General Discussion and Q&Aand Q&A
old.libqual.org
Summary and ClosureSummary and Closure
old.libqual.org
LibQUAL+LibQUAL+®® Resources Resources
• LibQUAL+® Web site:http://old.libqual.org
• Publications:http://old.libqual.org/publications
• Events and Training:http://old.libqual.org/events
• Gap Theory/Radar Graph Introduction:http://old.libqual.org/Information/Tools/libqualpresentation.cfm
• LibQUAL+® Procedures Manual:http://old.libqual.org/Publications/index.cfm
old.libqual.org
The The LibQUAL+LibQUAL+®® Team Team• MaShana Davis
Technical Communications [email protected]
• Yolanda GlassAdministrative [email protected]
• Kristina JusthCustomer Relations [email protected]
• Martha KyrillidouDirector, Statistics and Service Quality [email protected]
• Khyati NayakSenior Applications [email protected]
• Gary RoebuckTechnical Operations [email protected]