Review Article
Do quality improvement systems improve healthlibrary services? A systematic reviewHannah Gray*, Gary Sutton† & Victoria Treadway‡*Corporate Nursing Warrington & Halton Hospitals NHS Foundation Trust, Warrington, Cheshire, UK, †Knowledge & Library
Service Warrington & Halton Hospitals NHS Foundation Trust, Warrington, Cheshire, UK, and ‡Integrated Library Service
(ILS) Wirral University Teaching Hospital NHS Foundation Trust, Wirral, Cheshire, UK
Abstract
Background: A turbulent financial and political climate requires health libraries to be more accountablethan ever. Quality improvement systems are widely considered a ‘good thing to do’, but do they produceuseful outcomes that can demonstrate value?Objectives: To undertake a systematic review to identify which aspects of health libraries are beingmeasured for quality, what tools are being used and what outcomes are reported following utilisation ofquality improvement systems.Results: Many health libraries utilise quality improvement systems without translating the data into serviceimprovements. Included studies demonstrate that quality improvement systems produce valuable outcomesincluding a positive impact on strategic planning, promotion, new and improved services and staffdevelopment. No impact of quality improvement systems on library users or patients is reported in theliterature.Conclusions: The literature in this area is sparse and requires updating. We recommend further primaryresearch is conducted in health libraries focusing upon the outcomes of utilising quality improvementsystems. An exploration of quality improvement systems in other library sectors may also providevaluable insight for health libraries.
Keywords: evaluation, health care, impact factor, libraries, qualitative, quantitative, review, systematic
Key Messages
Implication for Practice
• Many health libraries utilise quality improvement systems without translating the data into servicechanges. Health libraries should seek to translate resulting data into action planning and report onthe outcomes.
• Further research should be undertaken tomeasure the outcomes of utilising quality improvement systemsaswell as thebarriers encounteredby libraryand informationprofessionals to implementingsuchsystems.
• Health libraries should align quality improvement processes to the objectives of the wider organisa-tion to gain support and buy-in from senior management.Implications for Policy
• Strategic development of health library services should promote the importance of quality improve-ment outcomes (rather than processes) as the key to improving services.
• An holistic approach to quality improvement systems should be adopted by library managers toensure that valuable performance data is utilised effectively.
Correspondence: Gary Sutton, Knowledge & Library Service, Warrington & Halton Hospitals NHS Foundation Trust, Lovely Lane, Warrington, Cheshire,UK, WA5 1QG. E-mail: [email protected]
© 2012 The authors. Health Information and Libraries Journal © 2012 Health Libraries Group
Health Information and Libraries Journal 1
DOI: 10.1111/j.1471-1842.2012.00996.x
Introduction
Accepted wisdom dictates that measuring anddemonstrating quality is a ‘good thing to do’ inlibrary and information practice. The current politi-cal and financial climate also requires libraries tobe accountable and provide good value for money.Quality improvement systems are one way inwhich we can do this; they allow libraries togather data about their performance that may beused to demonstrate value to stakeholders, identifygaps in service provision and inform action plan-ning and service improvement.A wide range of quality improvement systems are
available for use in libraries: benchmarking, satis-faction questionnaires, accreditation criteria, perfor-mance indicators and ‘off the shelf’ products suchas LibQUAL+TM. Implementing a quality improve-ment system and gathering the resulting data is a rel-atively straightforward exercise. But what happensnext? How are libraries using the wealth of usageand performance data that are collected each year?Do quality improvement systems actually improvepractice or are they merely carried out as an habitualexercise? This review hopes to uncover ways inwhich health library services, their users and ulti-mately patients have benefitted from the use of qual-ity improvement systems. See Box 1 for a fulldescription of the review’s aim and objectives.
Current context
There is huge variety in the structure and deliveryof health library services in the UK. That com-bined with the assortment of quality improvementsystems available means that there is great diver-sity in the way that quality is approached. Anadditional challenge to the modern health libraryservice is to report on quality outcomes to demon-strate value and fitness for purpose. Projects haveemerged that explore ways to illustrate value andimpact, such as the Making Alignment a Priority(MAP) toolkit
1
that enables libraries to demonstratetheir alignment with strategic objectives. A stan-dardised approach has been taken nationally withthe implementation of the NHS Library QualityAssurance Framework (LQAF)
2
that requireslibraries to measure their compliance against a setof national standards.
Defining quality
Defining quality and what constitutes a good qual-ity library service has been a long running subjectof debate amongst library and information profes-sionals. Booth
3
observes that there are three per-spectives to defining the concept: a ‘top-down’,‘sideways’ and ‘bottom-up’ approach.
3
A top-downapproach implements standards from an organisa-tional level with an emphasis on structures andprocesses that does not necessarily take intoaccount local circumstances. A sideways approachenables benchmarking and peer comparison butmay concentrate too heavily on ‘what is done, notwhat ought to be done’.
3
A bottom-up perspectivemeasures customer satisfaction against customerexpectation but may be too subjective a measureand does not necessarily consider value for money.For the purposes of this review, we have defined aquality improvement system as an evaluation ofone or more facets of a library service that pro-duces data about library performance.This review hopes to explore some of the benefits
that libraries can obtain from implementing qualityimprovement systems. Does the literature reportpositive effects on the knowledge and developmentof library staff? Are there perceived benefits to thecustomer? What about the ultimate measure ofvalue: impact on patient care? In addition, is thereany evidence of positive outcomes on the homeorganisation, the NHS itself and how are thesebeing reported? There have been no systematicreviews published on this subject so it is hoped thatthis review will illuminate some of these issues.
Box 1: Aim and Objectives
AimTo identify the reported outcomes of utilising
quality improvement systems in health libraries.Objectives1 Identify aspects of library services that arebeing measured for quality.
2 Ascertain which tools are being used tomeasure quality.
3 Explore how quality improvement systemsaffect library staff, library users, patientsand the organisation.
© 2012 The authors. Health Information and Libraries Journal © 2012 Health Libraries Group
Health Information and Libraries Journal
Do quality systems improve health libraries?, Hannah Gray et al.2
Methods
The review was carried out by three librariansworking across two NHS Trusts in the North Westof England. Owing to a lack of expertise amongstthe group in conducting systematic reviews, advicewas sought from health information professionalswith more experience in this area. These informa-tion professionals formed an advisory board fromwhich the review team could gain further input toensure a robust approach and effective methodol-ogy for the review. The advisory group consistedof the North West LIHNN (Library and Informa-tion Health Network North West) Quality Group,members of the North West Health Care LibrariesUnit (HCLU) and information professionals at theNational Institute for Health and Clinical Excel-lence (NICE). A review protocol was developedby the review group, and all members wereinvolved at every stage to ensure a standardisedapproach throughout the review and to enable allthree reviewers to gain experience of the fullreview process. The three reviewers began eachstage together to build confidence and to gain ashared understanding of the process (for example,application of the inclusion and exclusion criteria),and subsequently, the tasks were completed inpairs or individually.
Criteria for inclusion and exclusion
The key challenge for the review team in defininginclusion and exclusion criteria was to agree whattypes of quality improvement system would be thefocus of the review. Following much discussionbased on the literature and focussing specificallyon quality improvement systems utilised in healthlibraries, a decision was made to include qualityimprovement systems led by the library (Box 2).The group considered widening the criteria to
encompass library services in sectors beyondhealth but agreed that the literature would be tooextensive to conduct a thorough review with usefulconclusions and recommendations.To capture studies that measured the outcomes
of utilising quality improvement systems, thereview team included only those studies that evalu-ated the effect of quality improvement systems andreported outcomes relating to library staff, the
library service, library users, patients and the widerorganisation.
Search strategy
The search strategy was developed by the reviewteam using the inclusion and exclusion criteria asa framework for discussion and identification ofkey terms. Appropriate adaptations, including theuse of subject headings, were made depending onthe source used. The search was run in key data-bases in the fields of library and information man-agement, education, social sciences and business.The group also performed hand searches of theHealth and Information Libraries Journal (HILJ)1995–2010 and Performance Measurement andMetrics 2000–2010. References of the finalincluded studies were scanned for any furtherpapers that met the specified criteria. To identifygrey literature, searches were conducted inGoogle, Google Scholar, Index to Theses and In-tute and a request was made for relevant materialvia e-mail discussions lists including the CILIPHealth Libraries Group. All searches were runbetween June and September 2010. The fullsearch strategy and sources searched are availablein Appendix 1.
Filtering and selection
Figure 1 illustrates the flow of papers through thefiltering process, from the search stage to the iden-tification of papers to be included in the review.The review team filtered approximately 20% of theresults by title/abstract collectively to gain a sharedunderstanding of the inclusion and exclusion crite-ria. The remaining results were shared out and fil-tered individually by the review team members.Search terms were intentionally broad to ensurewe did not miss useful results; this meant that alarge number of results relating to libraries in othersectors were omitted at the filtering stage. The fulltext of papers was then obtained. Each study wasassessed by two review team members for confir-mation of exclusion or inclusion. Where there wasdisagreement between the two reviewers, the studywas discussed at a review meeting and consensusreached. In total, 16 studies were located for inclu-sion in the review.
4–20
These are summarised in the
© 2012 The authors. Health Information and Libraries Journal © 2012 Health Libraries Group
Health Information and Libraries Journal
Do quality systems improve health libraries?, Hannah Gray et al. 3
results table (Table 1) and inform the resultsdiscussion.
Assessment of study quality and data extraction
The review team completed the appraisal and dataextraction of three papers collectively to ensureconsistency of approach, the remaining paperswere then divided amongst the team and the pro-cess repeated individually. Two members of theteam then reviewed all the data (using the full textof the included papers) for accuracy and to iden-tify themes to be drawn out for the discussion andconclusions of the review.
Figure 1 Flow of literature through review process
Box 2: Inclusion and Exclusion Criteria
Inclusion criteria1 Studies featuring libraries that providelibrary and information services to practis-ing healthcare professionals.
2 Studies that feature any quality improve-ment system involving libraries, for exam-ple:• accreditation;• standards;• audit;• indicators;
• tools / systems to improve quality;• frameworks;• benchmarking.
3 Quality systems led by the library (ratherthan by the parent organisation).
4 Studies with an evaluation element of theeffects of the quality improvement systemon:• library staff;• library services;• library users;• patients;• the organisation.
5 English language studies.6 Published and unpublished items.Exclusion criteria1 Any organisation-led quality improvementsystems that do not involve libraries.
2 Discussion articles.3 Non-English language studies.
Results
The quality improvement systems used to measurehealth library services are diverse. Of the studiesthat employed a single method of measurement,seven
5,7,8,11,12,16,17
used questionnaires, three4,7,15
usedbenchmarking, one
10
used accreditation standardsand one
9
used LibQUAL+TM. Four studies13,18,19
used multiple methods including questionnaires,interviews, critical incident technique and statisti-cal data.The included studies measured different aspects
of the library service for quality. Eight stud-ies
4,6,7,9,12,13,17
measured multiple aspects of thelibrary service, three studies
8,18,20
measured outreachservices, two
16,19
measured searching services, one5
measured the library’s collection, one5
measured alibrary event and one
11
measured the Inter LibraryLoan service.The component of the library service that was
measured for quality was dependent on the aimsof the study. Only two studies
8,11
explicitly men-tioned service improvement as one of the aims oftheir study. The aims of other studies included themeasurement of library usage,
12,16
gathering dataabout the impact of the library service,
18,20
compar-
© 2012 The authors. Health Information and Libraries Journal © 2012 Health Libraries Group
Health Information and Libraries Journal
Do quality systems improve health libraries?, Hannah Gray et al.4
Tab
le1
Resultstable
Author
Yea
rMainaims
Methods
Library
staffoutcomes
Library
serviceoutcomes
Library
community
outcomes
Organ
isational
outcomes
Angel
4
1996
Ben
chmarkingofavarietyof
aspects
ofwomen
’s
hospital
library
services.
Ben
chmarking.
Nonestated
.Aben
efitofsurveyingthe
specialityhospitalswas
the
forum
itprovided
for
women
’shospital
librarian
s
tocommunicatewitheach
other
andlaygroundwork
forfuture
cooperation.
Theprototypic
ben
chmark
instrumen
tcould
be
adap
tedbyother
hospital
libraries
for
use
intheir
institutional
quality
andreen
gineering
initiatives.
Nonestated
.
Chen
g6
1996
Tooutlinehow
indicators
havebeendeveloped
to
allow
ben
chmarkingofall
aspects
ofservices
andthe
outcomes
ofthis
ben
chmarking.
Ben
chmarking.
‘Theprocess
itselfopen
sup
opportunitiesfor
improvemen
tan
dcantrigger
thecommitmen
tan
dthewill
todeliver
even
better
perform
ance’.
Foundthat
itisavaluab
letool
foreffectivecommunication
withthehospital
man
agem
entonlibrary
developmen
t’.
‘Wecanseethepositive
impactofourdevelopmen
t
strategiesin
less
than
3yearse.g.computer
searchinggrew
five
times’.
Nonestated
.Nonestated
.
© 2012 The authors. Health Information and Libraries Journal © 2012 Health Libraries Group
Health Information and Libraries Journal
Do quality systems improve health libraries?, Hannah Gray et al. 5
Table1.
(con
tinued)
Author
Yea
rMainaims
Methods
Library
staffoutcomes
Library
serviceoutcomes
Library
community
outcomes
Organ
isational
outcomes
Cundari7
1995
Evaluationofalibrary
service
usingaTQM
approach.
Questionnaire.
Thisprocess
has
allowed
the
professional
library
staffto
iden
tify
customer
needs,
makechan
ges
to
accommodatethose
needs
andestablishthemindset
that
qualityim
provemen
tis
anongoingprocess.
Theservicewas
able
to
respondto
unsatisfied
customersan
dto
developan
actionplanforservice
improvemen
t.
Immed
iate
chan
ges
included
anew
rangeoflibrary
fact
sheets,curren
taw
aren
ess
servicean
dim
provemen
ts
mad
eto
internal
procedures
(ILLsservice).Thelongterm
planhas
yetto
be
implemen
tedbutwillinvo
lve
utilisingtheintran
etfor
raisingtheprofile
ofthe
library.
Nonestated
.Nonestated
.
Davis
2004
Evaluationofuserperception/
satisfactionwithchan
ges
tothehospital
library’s
collections.
Questionnaire.
Nonestated
.En
abledrecommen
dationsto
bemad
eforservice
improvemen
t,e.g.survey
responseshighlightedthat
copyrighted
ucationan
d
guidan
ceisrequired
.
Nonestated
.Nonestated
.
© 2012 The authors. Health Information and Libraries Journal © 2012 Health Libraries Group
Health Information and Libraries Journal
Do quality systems improve health libraries?, Hannah Gray et al.6
Table1.
(con
tinued)
Author
Yea
rMainaims
Methods
Library
staffoutcomes
Library
serviceoutcomes
Library
community
outcomes
Organ
isational
outcomes
Fama8
2005
Toiden
tify
featuresof
existinglibrary
outreach
servicemodelsto
enab
le
localim
provemen
t.
Survey
of
outreach
library
staff.
Nonestated
.En
abledthem
tomakedetailed
recommen
dationsforservice
improvemen
t.
Nonestated
.Nonestated
.
Foss
9
2006
Evaluationofalibrary
service
usingtheLibQUAL+
TM
survey.
LibQUAL+
TM.
Instigated
stafftraining.
Followingthesurvey
astaff
meetingwas
heldto
iden
tify
customer
service
improvemen
tsan
dthe
actionsiden
tified
werestated
ascompleted.
Thedatacollected
was
used
toad
aptexistingservices:
review
andmodifycurren
t
policiesan
dproceduresto
ensure
excellentcustomer
service,
retrainstaffon
aspects
ofcirculationduty,
cross-train
onresponsibilities,
attendlibrary
instruction
classes,
lookat
other
sources
toincrease
collectionbudget,
implemen
tationofnew
OPACsystem
,electronic
reserves,an
dhiringofnew
staff.Im
provedaccess
to
e-journals.
Nonestated
.Nonestated
.
© 2012 The authors. Health Information and Libraries Journal © 2012 Health Libraries Group
Health Information and Libraries Journal
Do quality systems improve health libraries?, Hannah Gray et al. 7
Table1.
(con
tinued)
Author
Yea
rMainaims
Methods
Library
staffoutcomes
Library
serviceoutcomes
Library
community
outcomes
Organ
isational
outcomes
Gluck
10
2001
Toevaluatetheinclusionof
stan
dardsforhospital
libraries
inexternal
med
ical
educationaccred
itation
schem
es.
Accreditation
stan
dards.
Nonestated
Onelibrary
received
additional
fundingforstaffingan
d
betterlibrary
facilities.
Numeroushospitalshave
raised
theirstaffingor
budget
levelsas
aresult.
Onelibrary
had
afulltime
librarian
added
tothe
staffingquota.
Hospital
administrators
approvedan
additional
5h
per
weekforprofessional
library
staffto
meetthe
minim
um
stan
dards.
Onelibrary
was
awarded
fundingforelectronic
resources.
Theben
efits
ofhavinga
professional
librarian
onthe
accred
itationboardwere
realised
andthishas
been
mad
eaperman
entfixture.
Theweightthat
anexternal
accred
itationreport
can
carrywas
ackn
owledged
.
Thereview
canserveas
a
forum
inwhichthesm
ooth
andefficien
tleveloflibrary
perform
ance
isbroughtto
light,providinginvaluab
le
library
promotionto
the
physiciansan
dad
ministrators
whoareinvo
lved
inthe
hospital’s
CMEreview
team
.
Nonestated
.Becau
seofthetrust
established
duringthe
review
process,
administrationhas
beenmore
supportive
ofthelibrary
and
receptive
tomost
of
itsrequests’
Thehospital
took
CSM
Sreview
svery
seriouslyan
d
accred
itationby
outsideag
encies
was
promotedthroughthe
med
iaas
partoftheir
qualitycampaign.
© 2012 The authors. Health Information and Libraries Journal © 2012 Health Libraries Group
Health Information and Libraries Journal
Do quality systems improve health libraries?, Hannah Gray et al.8
Table1.
(con
tinued)
Author
Yea
rMainaims
Methods
Library
staffoutcomes
Library
serviceoutcomes
Library
community
outcomes
Organ
isational
outcomes
Had
ley1
1
2006
ToevaluateHealthLibraries
Weekwiththeaim
of
inform
ingfuture
direction
oftheinitiative.
Questionnaire.
Nonestated
.‘From
thedatawewereab
le
toproduce
asetof
recommen
dationsaround
new
,im
provedwaysof
workingforfuture
campaigns’.
Nonestated
.Nonestated
.
Hen
ry12
1993
Toevaluatetheuse
ofthe
library
servicebyhospital
administrators.
Questionnaire.
Nonestated
.Su
rvey
highlightedlack
of
awaren
essofman
agem
ent
materialin
collection.This
ledto
thelibrary
mountinga
publicitycampaignre
these
resources.
Reviewed
locationof
organ
isational
materialsto
increase
access.
Expan
ded
curren
taw
aren
ess
service.
Intendto
produce
new
sletter.
Thestructure
ofaCQI
program
provides
a
fram
ework
forthelibrary
staffto
exam
ineallaspects
ofthelibrary
operation.
TheCQIprocess
enab
lesthe
librarian
sto
iden
tify
underserved
usergroupsan
d
theirneeds,
tomarketthe
library
anditsservices
to
thesegroupsan
dto
make
chan
ges
toaccommodate
those
needs.
Nonestated
.Nonestated
.
Huntley
13
1970
Pilotofaqualitytoolto
evaluatevarious
componen
tsoflibrary
services.
Questionnaires,
interviewsan
d
statistical
inform
ation.
Nonestated
Observed
effects:
Library
usageincreased,services
broad
ened
,qualityprogram
introduced.
Nonestated
.Nonestated
.
© 2012 The authors. Health Information and Libraries Journal © 2012 Health Libraries Group
Health Information and Libraries Journal
Do quality systems improve health libraries?, Hannah Gray et al. 9
Table1.
(con
tinued)
Author
Yea
rMainaims
Methods
Library
staffoutcomes
Library
serviceoutcomes
Library
community
outcomes
Organ
isational
outcomes
Leishman
15
1995
Ben
chmarkingofILLservices
provided
byaregional
consortium
oflibraries.
Ben
chmarking.
Nonestated
.So
meservices
havecompared
thedatawiththeirown
trackingstatistics
andhave
investigated
localprocedures
wherethey
did
notmeetthe
groupmed
ian.
Someservices
haveusedthe
datato
justifychan
ges
inILL
procedurese.g.from
phone
toe-mailrequests.
Datausedto
provideusers
witharealisticexpectation
regardingturnaroundtime.
Nonestated
.Nonestated
.
Lyon
16
1989
Evaluationofthe‘online
search
service’;to
assess
quality,
iden
tify
patternsof
use
andassess
interest
in
endusersearching.
Questionnaire.
Nonestated
.En
abledrecommen
dationsto
bemad
eforservice
improvemen
t.
Nonestated
.Nonestated
.
Mi17
2007
Needsassessmen
tofusers
of
oneservice;
gathering
baselinedataonclients’
needs,usagepatternsan
d
satisfactionwiththelibrary
service.
Questionnaire.
Nonestated
.Thequalitysystem
produced
eviden
ceforthelibrary
abouthow
itisad
dressing
theneedsofusers,provided
justificationforrequesting
additional
funding;inform
ed
decisionsonnew
resources,
library
trainingdesignan
d
IT.Overallitserved
asa
‘goodstartfortheongoing
process
oflibrary
service
improvemen
t’.
Thelibrary
mad
ealleB
ooks
accessible
viathelibrary
catalogue.
Nonestated
.Nonestated
.
© 2012 The authors. Health Information and Libraries Journal © 2012 Health Libraries Group
Health Information and Libraries Journal
Do quality systems improve health libraries?, Hannah Gray et al.10
Table1.
(con
tinued)
Author
Yea
rMainaims
Methods
Library
staffoutcomes
Library
serviceoutcomes
Library
community
outcomes
Organ
isational
outcomes
Robinson
18
2007
Toevaluatetheim
pactof
library
outreach
services.
Questionnaire,
interviews.
Nonestated
.A
series
ofrecommen
dations
forgoodpracticein
outreach
services
havebeen
developed
based
onthe
resultsofthisstudy,
plus
implicationsforpolicyan
d
practiceiden
tified
.
Nonestated
.Nonestated
.
Urquhart1
9
1996
Todeterminethevalueof
inform
ationprovided
by
NHSlibrary
services
to
clinical
decisionmaking(of
med
ics)
andto
developa
toolkitforau
ditingthis
aspectofinform
ation
services.
Surveys,
interviews,
critical
inciden
tstudy,
questionnaires.
Nonestated
.Developmen
tofatoolkitto
enab
leinform
ationservices
notonly
toassess
their
impactonclinical
decision
making,butalso
toiden
tify
waysin
whichthat
impact
could
been
han
cedthrough
improvedtargetingof
services.
Nonestated
.Nonestated
.
Yeo
man
20
2003
Developan
devaluate
methodsfordetermining
thevaluean
dim
pactof
library
outreach
services.
Questionnaires,
interviews,
critical
inciden
t
technique,
vignettes,
cost
study.
Project
staffap
preciated
closer
invo
lvem
entin
theoperation
ofaresearch
project
and
severalofthose
invo
lved
haveconsolidated
theirskills
innew
projects.
Oneoftheoutreach
service
man
agersnow
has
greater
responsibilities
ined
ucation
andtrainingforclinical
and
library
staff.
Librarian
scommen
tedthat
added
valuecamefrom
invo
lvem
entin
awider
study
that
raised
theprofile
and
hen
ceaw
aren
essoftheir
projectsat
localan
dregional
levels.
Evaluation:work
extended
toprimarycare
atonesite.
VIVOSeviden
cesupported
outreach
serviceman
agersin
justifyingtheirneedfor
trainersan
dforparticular
trainingap
proaches
aswell
asindicatingtheneedfora
24/7
approachto
support.
Methodologies
developed
inVIVOS
appliedto
evaluation
oftheNELH.
Tobedeveloped
asa
dissertationproject
forapost
graduate
studen
t.
Experience
inform
ed
thepreparationofa
research
proposalon
communityhealth
inform
ation.
Helped
guidefurther
developmen
tof
collaborative
research
fram
eworks.
Nonestated
.
© 2012 The authors. Health Information and Libraries Journal © 2012 Health Libraries Group
Health Information and Libraries Journal
Do quality systems improve health libraries?, Hannah Gray et al. 11
ing data with other library services,4,6,15
measuringuser satisfaction,
5,17
or simply ‘evaluation’ of thelibrary service.
7,9
Whether service improvement isan implicit objective in these studies is a matterfor debate. What is noticeable in the literature is afocus on the methodology surrounding qualitymeasurement and the resulting data gathered,rather than the adoption of a holistic stance toquality improvement and the effects of such qual-ity measurement activities.In line with the objectives of this review, we
were interested in identifying reported outcomesrelating to library staff, the library service, thelibrary users, patients and the wider organisation.The most commonly reported outcomes were
related to the library service and how it is deliv-ered. Data gathered as a result of quality improve-ment systems can be used to justify funding foradditional services or resources. Gluck
10
recountshow involvement of the library in a medical edu-cation accreditation scheme provided ‘an excellentopportunity for an audit of library services andlevels of support, performed by outside profession-als’. The accreditation process enabled the libraryservices involved to negotiate extra working hoursfor professional library staff, improved IT facilitiesand an increased library budget.Several studies
5,7,8,11,16,18,19
reported that the qualitymeasurement process enabled the library service tomake recommendations for improved ways ofworking. Robinson
18
states that ‘a series of recom-mendations for good practice in outreach serviceshave been developed based on the results of thisstudy, plus implications for policy and practiceidentified’(p. 65). The potential of quality improve-ment systems to inform strategic action planning isacknowledged by a number of studies,
7,9,10,12,15,17
which reported tangible improvements to servicedelivery as a direct result of involvement in qualityinitiatives. As a result of a user questionnaire thathighlighted a lack of awareness amongst users aboutmanagement related material in the library’s collec-tion, Henry
12
mounted a promotional campaign topublicise these materials and reviewed the locationof its collection to improve access to these materi-als. Cundari
7
reports how the library service imple-mented a two-phase action plan for serviceimprovement as a result of a total quality manage-ment approach to service delivery. Immediate short-
term actions included a new range of library factsheets, current awareness service and improvementsmade to internal procedures. Long-term improve-ments involved utilising the intranet for raising theprofile of the library(p. 190). The use of LibQUAL+TM as a quality improvement tool enabled Foss
9
to fulfil a series of project recommendations withina 3-month period. Data collected from LibQUAL+TM were used to review and modify current poli-cies and procedures to ensure excellent customerservice, retrain staff on aspects of circulation duty,cross-train on responsibilities, attend library instruc-tion classes, look at other sources to increase collec-tion budget, implement a new OPAC system,introduce electronic reservations and hire newstaff(p. 13–14). All are examples that highlight thepotential for quality improvement systems to informshort-term and long-term service improvements.Many studies were excluded from the final review
because they failed to report on outcomes that werea result of utilising the quality improvement system.Implementing a quality improvement system withina health library may provide interesting usage andperformance data, but many studies did not translatethis knowledge into service changes and were,therefore, excluded from the review.An additional reported outcome was that the
quality improvement system provided the opportu-nity to raise the profile of the library service bothinternally and externally to the organisation.Librarians commented that as a result of involve-ment in an evaluation project
20
‘added value camefrom involvement in a wider study that raised theprofile and hence awareness of their projects atlocal and regional levels’(p. 431). Gluck
10
reportsthat externally to the organisation involvement inan accreditation programme ‘was promotedthrough the media as part of [the hospital’s] qual-ity campaign’(p. 274). A raised library profile withinthe organisation may also prompt discussions withsenior management; Cheng
6
states that undertakinga benchmarking activity proved to be ‘a valuabletool for effective communication with the hospitalmanagement on library development’(p. 6).Overall, reported outcomes of utilising quality
improvement systems were positive. Henry12
sug-gests that the structure of a total quality improve-ment system ‘provides a framework for the librarystaff to examine all aspects of the library’s opera-
© 2012 The authors. Health Information and Libraries Journal © 2012 Health Libraries Group
Health Information and Libraries Journal
Do quality systems improve health libraries?, Hannah Gray et al.12
tion’(p. 439). A quality improvement system may pro-vide an opportune pause for the library service toexamine service delivery. Mi
17
describes the imple-mentation of a quality improvement process as ‘agood start for the ongoing process of library serviceimprovement and for the continuous efforts of main-taining excellent customer satisfaction’(p. 51).Implementing quality improvement systems
undoubtedly requires a substantial degree of inputfrom library managers and library staff. An invest-ment of time and effort into quality improvementsystems can reap professional benefits for the staffinvolved. Studies reported various outcomes relat-ing to staff development, including expanding pro-fessional experiences and the acquisition of newskills. Yeoman
20
reports that ‘staff appreciated clo-ser involvement in the operation of a research pro-ject and several of those involved have consolidatedtheir skills in new projects’(p. 433). Cheng
6
recognis-es that staff involvement in quality improvementsystems encourages a team focus on serviceimprovement and that ‘the process itself opens upopportunities for improvement and can trigger thecommitment and the will to deliver even better per-formance’(p. 6). Cundari
7
agrees, stating that ‘thisprocess has allowed the professional library staff toidentify customer needs, make changes to accom-modate those needs and establish the mindset thatquality improvement is an ongoing process’(p. 191).Foss
9
illustrates practically how the quality measure-ment process enabled a stronger staff focus on cus-tomer service and describes how a staff meetingwas called ‘to communicate the results of the surveyand identify solutions to provide better customerservice while enforcing policies’(p. 13). So perhapsinvolvement in quality initiatives can not only allowlibrary and information professionals to expand thebreadth of their own experience and skills but alsoenable the library team to share a common directionin the improvement of service delivery.The effects of quality improvement systems on
library users are perhaps more difficult to quantify;none of the included studies reported any directimpact on customers. This may be for several rea-sons; it may be that direct impact of qualityimprovement systems on library users is difficult tomeasure. Perhaps positive effects on library usersare implicitly assumed within the notion of serviceimprovement. As Lyon
16
puts it, ‘readers are entitled
to the best possible service and, to ensure that theyreceive it, there is a constant need to evaluate andre-evaluate the quality of services on offer’(p. 3). Orperhaps library services concentrate too much onthe concept of quality as an internal mechanism anddo not adopt a customer-facing stance to improvingperformance. No effects on patients were reportedin any of the studies included in the review, possi-bly for the reasons outlined earlier.Only one study
10
reported any effects on thewider organisation. Following involvement in anaccreditation initiative the library service estab-lished a more positive working relationship withthe hospital administration that was consequently‘more supportive of the library and receptive tomost of its requests’(p. 275). Improved communica-tion between the library and senior managementabout the quality improvement process could per-haps enable the wider organisation to take a greaterinterest and assume more ownership in serviceimprovement within the library. To achieve thisultimately requires library managers to demonstratehow a library in pursuit of quality improvement isbeneficial to the wider health community by align-ing its objectives with those of the organisation.An effect that was reported in some studies
4,20
that was not anticipated at the beginning of thereview was the impact that quality improvementsystems can have on the wider health library com-munity. Yeoman
20
developed an evaluation method-ology that was later applied to other evaluationprojects(p. 432). The work undertaken also ‘helpedguide further development of collaborative researchframeworks’(p. 432). Angel
4
states that the bench-marking exercise undertaken provided a forum for‘hospital librarians to communicate with each otherand lay groundwork for future cooperation’(p. 584).Sharing any knowledge that is gained from under-taking quality improvement initiatives in healthlibraries can be valuable learning for other libraryand information professionals and can only serveto inform and improve good practice in this area.
Assessment of Quality
Study design characteristics
Half of the studies5,8–10,12,13,18,19
included in thisreview assumed a qualitative approach (n = 8),
© 2012 The authors. Health Information and Libraries Journal © 2012 Health Libraries Group
Health Information and Libraries Journal
Do quality systems improve health libraries?, Hannah Gray et al. 13
encompassing surveys (n = 3), a mix of surveys,questionnaires and critical incident techniques,
13,19
LibQUAL+TM9
and accreditation standards.Five studies used a mixture of qualitative and
quantitative methodologies to approach theirstudy,
7,11,16,17,20
of which four used question-naires
7,11,16,17
and one used a combination of ques-tionnaires, interviews, critical incident technique,vignettes and cost study.
20
Two studies used a case study methodology,6,15
using benchmarking as their main method of datacollection.One study solely employed quantitative analy-
sis,4
looking at benchmarking within a women’shospital.
Assessment of quality
No obvious critical appraisal tool was available,which met our requirements sufficiently. Therefore,the review team formulated and developed a qual-ity checklist based upon three existing criticalappraisal tools.
21–24
The results table summarisesthe studies meeting these quality criterions.Nine studies had clearly stated aims and objec-
tives,4,7,8,10,13,15,17–19
compared with seven5,6,9,11,12,16,20
thatdid not.
Quality of reporting
Three studies’ methods were not clearlydescribed,
6,10,20
and an element of uncertainty totheir design was apparent. One study’s reportingof methods was unclear, as there was uncertaintyas to whether the study instrument, a question-naire, had been sent to all participants.
12
Theremaining studies’
4,5,7–9,11,13,15–19
(n = 12) methodswere clearly described, thus providing clarity andallowing for future replication.There were studies that exhibited good research
practice; Leishmann15
carried out two studies atdifferent times of the year to test the reliabilityagainst each other. Cundari
7
utilised both purpose-ful and random sampling techniques to ensure theyreached an appropriate mix of participants whilstminimising the risk of introducing selection biasinto the sample. Urquhart
19
made sure all recipientsreceived a follow-up survey, respondents whoreported that the information provided was of
‘actual’ or ‘potential value’ was invited to inter-view. A random sample was also utilised in thecritical incident aspect of the study, again minimis-ing selection bias. In contrast, Huntley
13
did notinform who the interviewees were, Fana
8
clearlydescribed their methods, but the geographicalrestrictions they placed upon their selections mayhave inhibited the responses they received. Henry
12
appeared to survey all hospital administrators, butit was unclear as to what their sample size actuallywas. Yeoman
20
utilised a mix of selection methodsover seven sites, which hampered the quality oftheir results. Uniformity of selection methodswould have produced higher quality results.The majority of studies
4,7–9,13,15–20
made a goodcase for the approach they decided to take.Huntley
13
argued statistics and data alone wouldnot support any judgements made and Urquhart
19
based their research methodology upon analysis ofprevious studies’ methods, and successful method-ologies were employed. The studies
5,6,10–12
where acase was not made either decided not to justifytheir methodology or partly provided reasons forsome of their methods. Examples of such rangefrom Foss’
9
decision to email surveys but not pro-vide print copies for participants who may nothave access to their email to Henry’s
12
adaption oftheir survey from other, non-related MedicalLibrary Association surveys.Seven of the studies neglected to make their
aims and objectives clear5,6,9,11,12,16,20
compared withnine studies
4,7,8,10,13,15,17–19
who had them plainlystated.The majority of the studies’
4,5,7,8,11–13,15,16,18,19
(n = 12) results addressed their original researchquestion. The other five studies
6,9,10,17,20
either devi-ated from their original hypothesis
6,10,17,20
orappeared to have not posed one originally.
9
The conclusions drawn from their results werein the main based upon an objective interpretationof the results.
5,7–9,12,13,15,17–19
Where appropriate, theresearchers offered caution because of thematiccoding drawn from the researchers’ interpretation
8
or admittance that no evaluation mechanism is per-fect.
13
Three studies4,11,20
did not appear to basetheir conclusions objectively, Hadley’s
11
conclusionwas unrelated to the survey and Angel
4
inferredwhy certain statistics were lower than others. Itwas unclear with two of the included studies
6,10
© 2012 The authors. Health Information and Libraries Journal © 2012 Health Libraries Group
Health Information and Libraries Journal
Do quality systems improve health libraries?, Hannah Gray et al.14
whether their conclusions were truly objective. InGluck’s
10
case, the results were anecdotal so it wasdifficult to assess.Related to this was the appropriateness of the
sample size to conclusions drawn from theirresults. Ten studies
5–7,11,12,16–19
made conclusions thatwere apt, in relation to the size of the sample.Angel’s
4
conclusions were not in relation to sam-ple size, and it was unclear in the following stud-ies: Fama,
8
Foss,9
Gluck,10
Huntley,13
Leishmann15
and Yeoman.20
Strengths and Limitations
A key challenge in conducting the review wasagreeing a definition of quality to inform the inclu-sion and exclusion criteria. Although the reviewteam spent much time deliberating over the inclu-sion criteria regarding quality, the different experi-ences of the team coupled with input from theadvisory board provided a comprehensiveapproach that arguably would not have beenachieved by a single reviewer. The decision tolimit the review to health libraries and English lan-guage omitted potentially valuable literature fromother sectors and countries; this was done to limitthe literature to a manageable volume. The reviewteam worked collaboratively where time and logis-tics allowed with all members commencing eachstage of the review together to ensure a consistentapproach. The full text was filtered by at least twomembers of the team; however, most criticalappraisal was conducted by a single reviewer.
Conclusions
The literature examining the outcomes of the useof quality improvement systems in health librariesis sparse and needs updating. There may be qualityimprovement work being undertaken on a moreinformal level within health libraries. To make thisknowledge more explicit and accessible to a wideraudience, we recommend further primary researchis conducted in health libraries focusing uponthe outcomes of utilising quality improvementsystems.Several tools were used in the studies to mea-
sure quality in a wide range of library services.The literature reveals a number of positive out-
comes of utilising quality improvement systemsrelating to the library service, library staff and thewider health organisation. However, no studieswere able to identify any effects on library usersor patients. This may be due to the fact thatimproved service to customers (and thus improvedpatient care) is implied following the implementa-tion of quality improvement systems. It may, how-ever, point to a need for health libraries to viewquality improvement as a customer-focused exer-cise rather than an internal mechanism. One out-come that was not anticipated at the beginning ofthe review was that quality improvement systemsalso have an effect on the wider library commu-nity, prompting health librarians to collaborate,network and share professional knowledge aroundthe subject.One of the objectives of the review was to
examine the outcomes of quality improvement sys-tems on library staff, library users, patients and theorganisation. However, none of the included stud-ies set out to do this as their primary aim butinstead described the quality improvement process.There is a tendency in the literature to focus onthe methodology surrounding quality improvementsystems rather than what prompts services to adoptquality improvement systems and the impact oftheir introduction. Many studies were excludedfrom the review because of lack of reporting onthe outcomes of utilising quality improvement sys-tems.Initially the review team had hoped to gain
some understanding of the barriers that preventlibrary and information professionals from imple-menting quality improvement systems. With hind-sight, however, an answer to this question isunlikely to be acquired from the literature and maybe best addressed through further primary researchin this area. An exploration of the outcomes ofutilising quality improvement systems in otherlibrary sectors may also provide valuable insightfor health libraries.The literature demonstrates how quality
improvement systems can result in improvementsto strategic planning, promotion, new andimproved services and staff training. However,many health libraries utilise quality improvementsystems without translating the data into serviceimprovements. Health libraries need to go one step
© 2012 The authors. Health Information and Libraries Journal © 2012 Health Libraries Group
Health Information and Libraries Journal
Do quality systems improve health libraries?, Hannah Gray et al. 15
further by assessing and reporting on the outcomesof their quality improvement systems.
References
1 Booth, A. What is quality and how can we measure it? Shine2003. Available at: http://www.shinelib.org.uk/cpd/journal/volume_43/what_is_quality_and_how_can_we_measure_it(accessed 1 May 2012).
2 Strategic Health Authorities Leads. Library Quality Accredi-tation Framework. 2010. Available at: http://www.libraryser-vices.nhs.uk/forlibrarystaff/lqaf/ (accessed 1 May 2012).
3 Library and Information Health Network North West. Avail-able at: http://alignmentprojectnwlibrarians.pbworks.com/w/page/11046836/FrontPage (accessed 1 May 2012).
4 Koufogiannakis, D., Booth, A. & Brettle, A. ReLIANT:reader’s guide to the literature on Interventions Addressingthe Need for education and Training. Library and Informa-tion Research, 2006, 30, 44–51.
5 CriSTAL. Appraising a User Study. Key questions to helpyou make sense of a user study. Sheffield, UK: Universityof Sheffield, School of Health and Related Research(ScHARR). Available at: http://www.shef.ac.uk/scharr/eblib/use.htm (accessed August 2009).
6 Cawthorne, J. E. Leading from the Middle of the Organiza-tion: an examination of shared leadership in AcademicLibraries. The Journal of Academic Librarianship, 2010, 36,151–157.
7 Brettle, A., Maden-Jenkins, M., Anderson, L., McNally, R.,Pratchett, T., Tancock, J., Thornton, D. & Webb, A. Evalu-ating clinical librarian services: a systematic review. HealthInformation and Libraries Journal, 2011, 28, 3–22.
8 Angel, S. H. & Mackler, L. G. A benchmark instrumenttested in women’s hospital libraries. Bulletin of the MedicalLibrary Association, 1996, 84, 582–585.
9 Campbell, K., Davis, A., McGowan, J. & Shorr, R. A post-merger hospital library collection survey and data analysisindicated better resource allocation and user satisfaction.Journal of the Canadian Health Libraries Association,2004, 25, 29–38.
10 Cheng, G. Y. T. The use of benchmarking in improving thelibrary service quality of devolved management. Journal ofthe Hong Kong Library Association 1996, 18, 1–14.
11 Cundari, L. & Stutz, K. Enhancing library services: anexploration in meeting customer needs through total qualitymanagement.. Special Libraries, 1995, 86, 188–194.
12 Fama, J., Berryman, D., Harger, N., Julian, P., Peterson, N.,Spinner, M. & Varney, J. Inside outreach: a challenge forhealth sciences librarians. Journal of the Medical LibraryAssociation, 2005, 93, 327–337.
13 Foss, M., Buhler, A., Rhine, L. & Layton, B. HSCL Lib-QUAL+ 2004: from numbers and graphs to practical appli-cation. Medical Reference Services Quarterly, 2006, 25, 1–15.
14 Gluck, J. C. & Hassig, R. A. Raising the bar: the impor-tance of hospital library standards in the continuing medicaleducation accreditation process. Bulletin of the MedicalLibrary Association, 2001, 89, 272–276.
15 Hadley, S. Health libraries week 2005 – Evaluation usingan online survey tool. Health Information on the Internet,2006, 50, 9.
16 Henry, B. Continuous quality improvement in the hospitallibrary. Bulletin of the Medical Library Association, 1993,81, 437–439.
17 Huntley, J. L. & Orrok, P. The hospital library profile as anevaluation mechanism. Bulletin of the Medical Library Asso-ciation, 1970, 58, 403–414.
18 Lee, T. Exploring outcomes assessment: the AAHSL Lib-QUAL+ experience. Journal of Library Administration,2004, 40, 49–58.
19 Leishman, J. Improving interlibrary loan quality throughbench-marking: a case study from the Health Science Infor-mation Consortium of Toronto. Health Libraries Review,1995, 12, 215–218.
20 Lyon, E. The questionnaire – a quality control method foronline searching? Health Libraries Review, 1989, 6, 3–19.
21 Mi, M. & Gilbert, C. M. Needs assessment: prerequisite forservice excellence. Journal of Hospital Librarianship, 2007,7, 31–52.
22 Robinson, L. & Bawden, D. Evaluation of outreach servicesfor primary care and mental health; assessing the impact.Health Information and Libraries Journal, 2007, 24, 57–66.
23 Urquhart, C. J. & Hepworth, J. B. Comparing and usingassessment of the value of information to clinical decision-making. Bulletin of the Medical Library Association, 1996,84, 482–489.
24 Yeoman, A., Tyler, A., Urquhart, C. J. & Cooper, J. M. Themanagement of health library outreach services: evaluationand reflection on lessons learned on the VIVOS project.Journal of the Medical Library Association, 2003, 91, 426–433.
Received 23 December 2011; Accepted 15 May 2012
Appendix 1: Search Strategy
Sources Platform/Publisher
AMED (Allied and
Complementary Medicine)
NHS Evidence Healthcare
Databases Advanced
Search
BNI (British Nursing Index) NHS Evidence Healthcare
Databases Advanced
Search
Cinahl (Cumulative Index to
Nursing and Allied Health)
NHS Evidence Healthcare
Databases Advanced
Search
© 2012 The authors. Health Information and Libraries Journal © 2012 Health Libraries Group
Health Information and Libraries Journal
Do quality systems improve health libraries?, Hannah Gray et al.16
Table . (continued)
Sources Platform/Publisher
Cochrane Library (Database of
Systematic Reviews,
Controlled Trails Register,
Database of Abstracts of
Reviews of Effectiveness,
Methodology Database)
Wiley InterScience
Embase NHS Evidence Healthcare
Databases Advanced
Search
ERIC (Educational Resources
Information Centre)
Dialog
Google & Google Scholar Google
Health Information and Libraries
Journal (HILJ) (journal hand
search)
Wiley
HMIC (Health Management
Information Consortium)
NHS Evidence Healthcare
Databases Advanced
Search
Index to Theses Expert Information Ltd
Intute Mimas
LISA (Library and Information
Science Abstracts)
CSA (Cambridge Scientific
Abstracts)
LISTA (Library, Information and
Technology Science Abstracts)
EBSCOHost
Medline NHS Evidence Healthcare
Databases Advanced
Search
Performance Measurement and
Metrics (journal hand search)
Emerald
Proquest Library Science Proquest
Psycinfo NHS Evidence Healthcare
Databases Advanced
Search
Web of Knowledge ISI (Institute of Scientific
Information)
Email discussion lists
The following email discussion lists were [email protected]@[email protected]
Search terms
Terms used in database searches are listed below.Where possible, a wild card (* in the list of termsbelow) was used in individual database searchstrategies, to retrieve variations in the endings orspelling of terms. Database thesauri were utilisedwhere appropriate.‘evidence service*’ or ‘information service*’ or
‘knowledge service*’ or library or librariesANDdental or dentist* or health or healthcare or hos-
pital or medical or medicine or ‘National HealthService’ or NHS or nurse* or nursing or PCT or‘primary care’ or SHA or ‘Strategic HealthAuthority’(if applicable, i.e. do not use if searching a
health related database, e.g. CINAHL)ANDaccredit* or ‘balanced scorecard’ or benchmark*
or ‘codes of practice’ or helicon or improv* orindicator* or Libqual or LQAF or measur* ormonitor* or performance or quality or question-naire or Servqual or standard* or statistic* orsurveyANDassess* or evaluat* or effect* or affect* or
impact.
© 2012 The authors. Health Information and Libraries Journal © 2012 Health Libraries Group
Health Information and Libraries Journal
Do quality systems improve health libraries?, Hannah Gray et al. 17