do quality improvement systems improve health library services? a systematic review

Review Article

Do quality improvement systems improve healthlibrary services? A systematic reviewHannah Gray*, Gary Sutton† & Victoria Treadway‡*Corporate Nursing Warrington & Halton Hospitals NHS Foundation Trust, Warrington, Cheshire, UK, †Knowledge & Library

Service Warrington & Halton Hospitals NHS Foundation Trust, Warrington, Cheshire, UK, and ‡Integrated Library Service

(ILS) Wirral University Teaching Hospital NHS Foundation Trust, Wirral, Cheshire, UK

Abstract

Background: A turbulent financial and political climate requires health libraries to be more accountablethan ever. Quality improvement systems are widely considered a ‘good thing to do’, but do they produceuseful outcomes that can demonstrate value?Objectives: To undertake a systematic review to identify which aspects of health libraries are beingmeasured for quality, what tools are being used and what outcomes are reported following utilisation ofquality improvement systems.Results: Many health libraries utilise quality improvement systems without translating the data into serviceimprovements. Included studies demonstrate that quality improvement systems produce valuable outcomesincluding a positive impact on strategic planning, promotion, new and improved services and staffdevelopment. No impact of quality improvement systems on library users or patients is reported in theliterature.Conclusions: The literature in this area is sparse and requires updating. We recommend further primaryresearch is conducted in health libraries focusing upon the outcomes of utilising quality improvementsystems. An exploration of quality improvement systems in other library sectors may also providevaluable insight for health libraries.

Keywords: evaluation, health care, impact factor, libraries, qualitative, quantitative, review, systematic

Key Messages

Implication for Practice

• Many health libraries utilise quality improvement systems without translating the data into servicechanges. Health libraries should seek to translate resulting data into action planning and report onthe outcomes.

• Further research should be undertaken tomeasure the outcomes of utilising quality improvement systemsaswell as thebarriers encounteredby libraryand informationprofessionals to implementingsuchsystems.

• Health libraries should align quality improvement processes to the objectives of the wider organisa-tion to gain support and buy-in from senior management.Implications for Policy

• Strategic development of health library services should promote the importance of quality improve-ment outcomes (rather than processes) as the key to improving services.

• An holistic approach to quality improvement systems should be adopted by library managers toensure that valuable performance data is utilised effectively.

Correspondence: Gary Sutton, Knowledge & Library Service, Warrington & Halton Hospitals NHS Foundation Trust, Lovely Lane, Warrington, Cheshire,UK, WA5 1QG. E-mail: [email protected]

© 2012 The authors. Health Information and Libraries Journal © 2012 Health Libraries Group

Health Information and Libraries Journal 1

DOI: 10.1111/j.1471-1842.2012.00996.x

Introduction

Accepted wisdom dictates that measuring anddemonstrating quality is a ‘good thing to do’ inlibrary and information practice. The current politi-cal and financial climate also requires libraries tobe accountable and provide good value for money.Quality improvement systems are one way inwhich we can do this; they allow libraries togather data about their performance that may beused to demonstrate value to stakeholders, identifygaps in service provision and inform action plan-ning and service improvement.A wide range of quality improvement systems are

available for use in libraries: benchmarking, satis-faction questionnaires, accreditation criteria, perfor-mance indicators and ‘off the shelf’ products suchas LibQUAL+TM. Implementing a quality improve-ment system and gathering the resulting data is a rel-atively straightforward exercise. But what happensnext? How are libraries using the wealth of usageand performance data that are collected each year?Do quality improvement systems actually improvepractice or are they merely carried out as an habitualexercise? This review hopes to uncover ways inwhich health library services, their users and ulti-mately patients have benefitted from the use of qual-ity improvement systems. See Box 1 for a fulldescription of the review’s aim and objectives.

Current context

There is huge variety in the structure and deliveryof health library services in the UK. That com-bined with the assortment of quality improvementsystems available means that there is great diver-sity in the way that quality is approached. Anadditional challenge to the modern health libraryservice is to report on quality outcomes to demon-strate value and fitness for purpose. Projects haveemerged that explore ways to illustrate value andimpact, such as the Making Alignment a Priority(MAP) toolkit

1

that enables libraries to demonstratetheir alignment with strategic objectives. A stan-dardised approach has been taken nationally withthe implementation of the NHS Library QualityAssurance Framework (LQAF)

2

that requireslibraries to measure their compliance against a setof national standards.

Defining quality

Defining quality and what constitutes a good qual-ity library service has been a long running subjectof debate amongst library and information profes-sionals. Booth

3

observes that there are three per-spectives to defining the concept: a ‘top-down’,‘sideways’ and ‘bottom-up’ approach.

3

A top-downapproach implements standards from an organisa-tional level with an emphasis on structures andprocesses that does not necessarily take intoaccount local circumstances. A sideways approachenables benchmarking and peer comparison butmay concentrate too heavily on ‘what is done, notwhat ought to be done’.

3

A bottom-up perspectivemeasures customer satisfaction against customerexpectation but may be too subjective a measureand does not necessarily consider value for money.For the purposes of this review, we have defined aquality improvement system as an evaluation ofone or more facets of a library service that pro-duces data about library performance.This review hopes to explore some of the benefits

that libraries can obtain from implementing qualityimprovement systems. Does the literature reportpositive effects on the knowledge and developmentof library staff? Are there perceived benefits to thecustomer? What about the ultimate measure ofvalue: impact on patient care? In addition, is thereany evidence of positive outcomes on the homeorganisation, the NHS itself and how are thesebeing reported? There have been no systematicreviews published on this subject so it is hoped thatthis review will illuminate some of these issues.

Box 1: Aim and Objectives

AimTo identify the reported outcomes of utilising

quality improvement systems in health libraries.Objectives1 Identify aspects of library services that arebeing measured for quality.

2 Ascertain which tools are being used tomeasure quality.

3 Explore how quality improvement systemsaffect library staff, library users, patientsand the organisation.


Health Information and Libraries Journal

Do quality systems improve health libraries?, Hannah Gray et al.2

Methods

The review was carried out by three librariansworking across two NHS Trusts in the North Westof England. Owing to a lack of expertise amongstthe group in conducting systematic reviews, advicewas sought from health information professionalswith more experience in this area. These informa-tion professionals formed an advisory board fromwhich the review team could gain further input toensure a robust approach and effective methodol-ogy for the review. The advisory group consistedof the North West LIHNN (Library and Informa-tion Health Network North West) Quality Group,members of the North West Health Care LibrariesUnit (HCLU) and information professionals at theNational Institute for Health and Clinical Excel-lence (NICE). A review protocol was developedby the review group, and all members wereinvolved at every stage to ensure a standardisedapproach throughout the review and to enable allthree reviewers to gain experience of the fullreview process. The three reviewers began eachstage together to build confidence and to gain ashared understanding of the process (for example,application of the inclusion and exclusion criteria),and subsequently, the tasks were completed inpairs or individually.

Criteria for inclusion and exclusion

The key challenge for the review team in defininginclusion and exclusion criteria was to agree whattypes of quality improvement system would be thefocus of the review. Following much discussionbased on the literature and focussing specificallyon quality improvement systems utilised in healthlibraries, a decision was made to include qualityimprovement systems led by the library (Box 2).The group considered widening the criteria to

encompass library services in sectors beyondhealth but agreed that the literature would be tooextensive to conduct a thorough review with usefulconclusions and recommendations.To capture studies that measured the outcomes

of utilising quality improvement systems, thereview team included only those studies that evalu-ated the effect of quality improvement systems andreported outcomes relating to library staff, the

library service, library users, patients and the widerorganisation.

Search strategy

The search strategy was developed by the reviewteam using the inclusion and exclusion criteria asa framework for discussion and identification ofkey terms. Appropriate adaptations, including theuse of subject headings, were made depending onthe source used. The search was run in key data-bases in the fields of library and information man-agement, education, social sciences and business.The group also performed hand searches of theHealth and Information Libraries Journal (HILJ)1995–2010 and Performance Measurement andMetrics 2000–2010. References of the finalincluded studies were scanned for any furtherpapers that met the specified criteria. To identifygrey literature, searches were conducted inGoogle, Google Scholar, Index to Theses and In-tute and a request was made for relevant materialvia e-mail discussions lists including the CILIPHealth Libraries Group. All searches were runbetween June and September 2010. The fullsearch strategy and sources searched are availablein Appendix 1.

Filtering and selection

Figure 1 illustrates the flow of papers through thefiltering process, from the search stage to the iden-tification of papers to be included in the review.The review team filtered approximately 20% of theresults by title/abstract collectively to gain a sharedunderstanding of the inclusion and exclusion crite-ria. The remaining results were shared out and fil-tered individually by the review team members.Search terms were intentionally broad to ensurewe did not miss useful results; this meant that alarge number of results relating to libraries in othersectors were omitted at the filtering stage. The fulltext of papers was then obtained. Each study wasassessed by two review team members for confir-mation of exclusion or inclusion. Where there wasdisagreement between the two reviewers, the studywas discussed at a review meeting and consensusreached. In total, 16 studies were located for inclu-sion in the review.

4–20

These are summarised in the



Do quality systems improve health libraries?, Hannah Gray et al. 3

results table (Table 1) and inform the resultsdiscussion.

Assessment of study quality and data extraction

The review team completed the appraisal and dataextraction of three papers collectively to ensureconsistency of approach, the remaining paperswere then divided amongst the team and the pro-cess repeated individually. Two members of theteam then reviewed all the data (using the full textof the included papers) for accuracy and to iden-tify themes to be drawn out for the discussion andconclusions of the review.

Figure 1 Flow of literature through review process

Box 2: Inclusion and Exclusion Criteria

Inclusion criteria1 Studies featuring libraries that providelibrary and information services to practis-ing healthcare professionals.

2 Studies that feature any quality improve-ment system involving libraries, for exam-ple:• accreditation;• standards;• audit;• indicators;

• tools / systems to improve quality;• frameworks;• benchmarking.

3 Quality systems led by the library (ratherthan by the parent organisation).

4 Studies with an evaluation element of theeffects of the quality improvement systemon:• library staff;• library services;• library users;• patients;• the organisation.

5 English language studies.6 Published and unpublished items.Exclusion criteria1 Any organisation-led quality improvementsystems that do not involve libraries.

2 Discussion articles.3 Non-English language studies.

Results

The quality improvement systems used to measurehealth library services are diverse. Of the studiesthat employed a single method of measurement,seven

5,7,8,11,12,16,17

used questionnaires, three4,7,15

usedbenchmarking, one

10

used accreditation standardsand one

9

used LibQUAL+TM. Four studies13,18,19

used multiple methods including questionnaires,interviews, critical incident technique and statisti-cal data.The included studies measured different aspects

of the library service for quality. Eight stud-ies

4,6,7,9,12,13,17

measured multiple aspects of thelibrary service, three studies

8,18,20

measured outreachservices, two

16,19

measured searching services, one5

measured the library’s collection, one5

measured alibrary event and one

11

measured the Inter LibraryLoan service.The component of the library service that was

measured for quality was dependent on the aimsof the study. Only two studies

8,11

explicitly men-tioned service improvement as one of the aims oftheir study. The aims of other studies included themeasurement of library usage,

12,16

gathering dataabout the impact of the library service,

18,20

compar-




Tab

le1

Resultstable

Author

Yea

rMainaims

Methods

Library

staffoutcomes

Library

serviceoutcomes

Library

community

outcomes

Organ

isational

outcomes

Angel

4

1996

Ben

chmarkingofavarietyof

aspects

ofwomen

’s

hospital

library

services.

Ben

chmarking.

Nonestated

.Aben

efitofsurveyingthe

specialityhospitalswas

the

forum

itprovided

for

women

’shospital

librarian

s

tocommunicatewitheach

other

andlaygroundwork

forfuture

cooperation.

Theprototypic

ben

chmark

instrumen

tcould

be

adap

tedbyother

hospital

libraries

for

use

intheir

institutional

quality

andreen

gineering

initiatives.

Nonestated

.

Chen

g6

1996

Tooutlinehow

indicators

havebeendeveloped

to

allow

ben

chmarkingofall

aspects

ofservices

andthe

outcomes

ofthis

ben

chmarking.

Ben

chmarking.

‘Theprocess

itselfopen

sup

opportunitiesfor

improvemen

tan

dcantrigger

thecommitmen

tan

dthewill

todeliver

even

better

perform

ance’.

Foundthat

itisavaluab

letool

foreffectivecommunication

withthehospital

man

agem

entonlibrary

developmen

t’.

‘Wecanseethepositive

impactofourdevelopmen

t

strategiesin

less

than

3yearse.g.computer

searchinggrew

five

times’.

Nonestated

.Nonestated

.




Table1.

(con

tinued)

Author

Yea

rMainaims

Methods

Library

staffoutcomes

Library

serviceoutcomes

Library

community

outcomes

Organ

isational

outcomes

Cundari7

1995

Evaluationofalibrary

service

usingaTQM

approach.

Questionnaire.

Thisprocess

has

allowed

the

professional

library

staffto

iden

tify

customer

needs,

makechan

ges

to

accommodatethose

needs

andestablishthemindset

that

qualityim

provemen

tis

anongoingprocess.

Theservicewas

able

to

respondto

unsatisfied

customersan

dto

developan

actionplanforservice

improvemen

t.

Immed

iate

chan

ges

included

anew

rangeoflibrary

fact

sheets,curren

taw

aren

ess

servicean

dim

provemen

ts

mad

eto

internal

procedures

(ILLsservice).Thelongterm

planhas

yetto

be

implemen

tedbutwillinvo

lve

utilisingtheintran

etfor

raisingtheprofile

ofthe

library.

Nonestated

.Nonestated

.

Davis

2004

Evaluationofuserperception/

satisfactionwithchan

ges

tothehospital

library’s

collections.

Questionnaire.

Nonestated

.En

abledrecommen

dationsto

bemad

eforservice

improvemen

t,e.g.survey

responseshighlightedthat

copyrighted

ucationan

d

guidan

ceisrequired

.

Nonestated

.Nonestated

.




Table1.

(con

tinued)

Author

Yea

rMainaims

Methods

Library

staffoutcomes

Library

serviceoutcomes

Library

community

outcomes

Organ

isational

outcomes

Fama8

2005

Toiden

tify

featuresof

existinglibrary

outreach

servicemodelsto

enab

le

localim

provemen

t.

Survey

of

outreach

library

staff.

Nonestated

.En

abledthem

tomakedetailed

recommen

dationsforservice

improvemen

t.

Nonestated

.Nonestated

.

Foss

9

2006

Evaluationofalibrary

service

usingtheLibQUAL+

TM

survey.

LibQUAL+

TM.

Instigated

stafftraining.

Followingthesurvey

astaff

meetingwas

heldto

iden

tify

customer

service

improvemen

tsan

dthe

actionsiden

tified

werestated

ascompleted.

Thedatacollected

was

used

toad

aptexistingservices:

review

andmodifycurren

t

policiesan

dproceduresto

ensure

excellentcustomer

service,

retrainstaffon

aspects

ofcirculationduty,

cross-train

onresponsibilities,

attendlibrary

instruction

classes,

lookat

other

sources

toincrease

collectionbudget,

implemen

tationofnew

OPACsystem

,electronic

reserves,an

dhiringofnew

staff.Im

provedaccess

to

e-journals.

Nonestated

.Nonestated

.




Table1.

(con

tinued)

Author

Yea

rMainaims

Methods

Library

staffoutcomes

Library

serviceoutcomes

Library

community

outcomes

Organ

isational

outcomes

Gluck

10

2001

Toevaluatetheinclusionof

stan

dardsforhospital

libraries

inexternal

med

ical

educationaccred

itation

schem

es.

Accreditation

stan

dards.

Nonestated

Onelibrary

received

additional

fundingforstaffingan

d

betterlibrary

facilities.

Numeroushospitalshave

raised

theirstaffingor

budget

levelsas

aresult.

Onelibrary

had

afulltime

librarian

added

tothe

staffingquota.

Hospital

administrators

approvedan

additional

5h

per

weekforprofessional

library

staffto

meetthe

minim

um

stan

dards.

Onelibrary

was

awarded

fundingforelectronic

resources.

Theben

efits

ofhavinga

professional

librarian

onthe

accred

itationboardwere

realised

andthishas

been

mad

eaperman

entfixture.

Theweightthat

anexternal

accred

itationreport

can

carrywas

ackn

owledged

.

Thereview

canserveas

a

forum

inwhichthesm

ooth

andefficien

tleveloflibrary

perform

ance

isbroughtto

light,providinginvaluab

le

library

promotionto

the

physiciansan

dad

ministrators

whoareinvo

lved

inthe

hospital’s

CMEreview

team

.

Nonestated

.Becau

seofthetrust

established

duringthe

review

process,

administrationhas

beenmore

supportive

ofthelibrary

and

receptive

tomost

of

itsrequests’

Thehospital

took

CSM

Sreview

svery

seriouslyan

d

accred

itationby

outsideag

encies

was

promotedthroughthe

med

iaas

partoftheir

qualitycampaign.




Table1.

(con

tinued)

Author

Yea

rMainaims

Methods

Library

staffoutcomes

Library

serviceoutcomes

Library

community

outcomes

Organ

isational

outcomes

Had

ley1

1

2006

ToevaluateHealthLibraries

Weekwiththeaim

of

inform

ingfuture

direction

oftheinitiative.

Questionnaire.

Nonestated

.‘From

thedatawewereab

le

toproduce

asetof

recommen

dationsaround

new

,im

provedwaysof

workingforfuture

campaigns’.

Nonestated

.Nonestated

.

Hen

ry12

1993

Toevaluatetheuse

ofthe

library

servicebyhospital

administrators.

Questionnaire.

Nonestated

.Su

rvey

highlightedlack

of

awaren

essofman

agem

ent

materialin

collection.This

ledto

thelibrary

mountinga

publicitycampaignre

these

resources.

Reviewed

locationof

organ

isational

materialsto

increase

access.

Expan

ded

curren

taw

aren

ess

service.

Intendto

produce

new

sletter.

Thestructure

ofaCQI

program

provides

a

fram

ework

forthelibrary

staffto

exam

ineallaspects

ofthelibrary

operation.

TheCQIprocess

enab

lesthe

librarian

sto

iden

tify

underserved

usergroupsan

d

theirneeds,

tomarketthe

library

anditsservices

to

thesegroupsan

dto

make

chan

ges

toaccommodate

those

needs.

Nonestated

.Nonestated

.

Huntley

13

1970

Pilotofaqualitytoolto

evaluatevarious

componen

tsoflibrary

services.

Questionnaires,

interviewsan

d

statistical

inform

ation.

Nonestated

Observed

effects:

Library

usageincreased,services

broad

ened

,qualityprogram

introduced.

Nonestated

.Nonestated

.




Table1.

(con

tinued)

Author

Yea

rMainaims

Methods

Library

staffoutcomes

Library

serviceoutcomes

Library

community

outcomes

Organ

isational

outcomes

Leishman

15

1995

Ben

chmarkingofILLservices

provided

byaregional

consortium

oflibraries.

Ben

chmarking.

Nonestated

.So

meservices

havecompared

thedatawiththeirown

trackingstatistics

andhave

investigated

localprocedures

wherethey

did

notmeetthe

groupmed

ian.

Someservices

haveusedthe

datato

justifychan

ges

inILL

procedurese.g.from

phone

toe-mailrequests.

Datausedto

provideusers

witharealisticexpectation

regardingturnaroundtime.

Nonestated

.Nonestated

.

Lyon

16

1989

Evaluationofthe‘online

search

service’;to

assess

quality,

iden

tify

patternsof

use

andassess

interest

in

endusersearching.

Questionnaire.

Nonestated

.En

abledrecommen

dationsto

bemad

eforservice

improvemen

t.

Nonestated

.Nonestated

.

Mi17

2007

Needsassessmen

tofusers

of

oneservice;

gathering

baselinedataonclients’

needs,usagepatternsan

d

satisfactionwiththelibrary

service.

Questionnaire.

Nonestated

.Thequalitysystem

produced

eviden

ceforthelibrary

abouthow

itisad

dressing

theneedsofusers,provided

justificationforrequesting

additional

funding;inform

ed

decisionsonnew

resources,

library

trainingdesignan

d

IT.Overallitserved

asa

‘goodstartfortheongoing

process

oflibrary

service

improvemen

t’.

Thelibrary

mad

ealleB

ooks

accessible

viathelibrary

catalogue.

Nonestated

.Nonestated

.




Table1.

(con

tinued)

Author

Yea

rMainaims

Methods

Library

staffoutcomes

Library

serviceoutcomes

Library

community

outcomes

Organ

isational

outcomes

Robinson

18

2007

Toevaluatetheim

pactof

library

outreach

services.

Questionnaire,

interviews.

Nonestated

.A

series

ofrecommen

dations

forgoodpracticein

outreach

services

havebeen

developed

based

onthe

resultsofthisstudy,

plus

implicationsforpolicyan

d

practiceiden

tified

.

Nonestated

.Nonestated

.

Urquhart1

9

1996

Todeterminethevalueof

inform

ationprovided

by

NHSlibrary

services

to

clinical

decisionmaking(of

med

ics)

andto

developa

toolkitforau

ditingthis

aspectofinform

ation

services.

Surveys,

interviews,

critical

inciden

tstudy,

questionnaires.

Nonestated

.Developmen

tofatoolkitto

enab

leinform

ationservices

notonly

toassess

their

impactonclinical

decision

making,butalso

toiden

tify

waysin

whichthat

impact

could

been

han

cedthrough

improvedtargetingof

services.

Nonestated

.Nonestated

.

Yeo

man

20

2003

Developan

devaluate

methodsfordetermining

thevaluean

dim

pactof

library

outreach

services.

Questionnaires,

interviews,

critical

inciden

t

technique,

vignettes,

cost

study.

Project

staffap

preciated

closer

invo

lvem

entin

theoperation

ofaresearch

project

and

severalofthose

invo

lved

haveconsolidated

theirskills

innew

projects.

Oneoftheoutreach

service

man

agersnow

has

greater

responsibilities

ined

ucation

andtrainingforclinical

and

library

staff.

Librarian

scommen

tedthat

added

valuecamefrom

invo

lvem

entin

awider

study

that

raised

theprofile

and

hen

ceaw

aren

essoftheir

projectsat

localan

dregional

levels.

Evaluation:work

extended

toprimarycare

atonesite.

VIVOSeviden

cesupported

outreach

serviceman

agersin

justifyingtheirneedfor

trainersan

dforparticular

trainingap

proaches

aswell

asindicatingtheneedfora

24/7

approachto

support.

Methodologies

developed

inVIVOS

appliedto

evaluation

oftheNELH.

Tobedeveloped

asa

dissertationproject

forapost

graduate

studen

t.

Experience

inform

ed

thepreparationofa

research

proposalon

communityhealth

inform

ation.

Helped

guidefurther

developmen

tof

collaborative

research

fram

eworks.

Nonestated

.




ing data with other library services,4,6,15

measuringuser satisfaction,

5,17

or simply ‘evaluation’ of thelibrary service.

7,9

Whether service improvement isan implicit objective in these studies is a matterfor debate. What is noticeable in the literature is afocus on the methodology surrounding qualitymeasurement and the resulting data gathered,rather than the adoption of a holistic stance toquality improvement and the effects of such qual-ity measurement activities.In line with the objectives of this review, we

were interested in identifying reported outcomesrelating to library staff, the library service, thelibrary users, patients and the wider organisation.The most commonly reported outcomes were

related to the library service and how it is deliv-ered. Data gathered as a result of quality improve-ment systems can be used to justify funding foradditional services or resources. Gluck

10

recountshow involvement of the library in a medical edu-cation accreditation scheme provided ‘an excellentopportunity for an audit of library services andlevels of support, performed by outside profession-als’. The accreditation process enabled the libraryservices involved to negotiate extra working hoursfor professional library staff, improved IT facilitiesand an increased library budget.Several studies

5,7,8,11,16,18,19

reported that the qualitymeasurement process enabled the library service tomake recommendations for improved ways ofworking. Robinson

18

states that ‘a series of recom-mendations for good practice in outreach serviceshave been developed based on the results of thisstudy, plus implications for policy and practiceidentified’(p. 65). The potential of quality improve-ment systems to inform strategic action planning isacknowledged by a number of studies,

7,9,10,12,15,17

which reported tangible improvements to servicedelivery as a direct result of involvement in qualityinitiatives. As a result of a user questionnaire thathighlighted a lack of awareness amongst users aboutmanagement related material in the library’s collec-tion, Henry

12

mounted a promotional campaign topublicise these materials and reviewed the locationof its collection to improve access to these materi-als. Cundari

7

reports how the library service imple-mented a two-phase action plan for serviceimprovement as a result of a total quality manage-ment approach to service delivery. Immediate short-

term actions included a new range of library factsheets, current awareness service and improvementsmade to internal procedures. Long-term improve-ments involved utilising the intranet for raising theprofile of the library(p. 190). The use of LibQUAL+TM as a quality improvement tool enabled Foss

9

to fulfil a series of project recommendations withina 3-month period. Data collected from LibQUAL+TM were used to review and modify current poli-cies and procedures to ensure excellent customerservice, retrain staff on aspects of circulation duty,cross-train on responsibilities, attend library instruc-tion classes, look at other sources to increase collec-tion budget, implement a new OPAC system,introduce electronic reservations and hire newstaff(p. 13–14). All are examples that highlight thepotential for quality improvement systems to informshort-term and long-term service improvements.Many studies were excluded from the final review

because they failed to report on outcomes that werea result of utilising the quality improvement system.Implementing a quality improvement system withina health library may provide interesting usage andperformance data, but many studies did not translatethis knowledge into service changes and were,therefore, excluded from the review.An additional reported outcome was that the

quality improvement system provided the opportu-nity to raise the profile of the library service bothinternally and externally to the organisation.Librarians commented that as a result of involve-ment in an evaluation project

20

‘added value camefrom involvement in a wider study that raised theprofile and hence awareness of their projects atlocal and regional levels’(p. 431). Gluck

10

reportsthat externally to the organisation involvement inan accreditation programme ‘was promotedthrough the media as part of [the hospital’s] qual-ity campaign’(p. 274). A raised library profile withinthe organisation may also prompt discussions withsenior management; Cheng

6

states that undertakinga benchmarking activity proved to be ‘a valuabletool for effective communication with the hospitalmanagement on library development’(p. 6).Overall, reported outcomes of utilising quality

improvement systems were positive. Henry12

sug-gests that the structure of a total quality improve-ment system ‘provides a framework for the librarystaff to examine all aspects of the library’s opera-




tion’(p. 439). A quality improvement system may pro-vide an opportune pause for the library service toexamine service delivery. Mi

17

describes the imple-mentation of a quality improvement process as ‘agood start for the ongoing process of library serviceimprovement and for the continuous efforts of main-taining excellent customer satisfaction’(p. 51).Implementing quality improvement systems

undoubtedly requires a substantial degree of inputfrom library managers and library staff. An invest-ment of time and effort into quality improvementsystems can reap professional benefits for the staffinvolved. Studies reported various outcomes relat-ing to staff development, including expanding pro-fessional experiences and the acquisition of newskills. Yeoman

20

reports that ‘staff appreciated clo-ser involvement in the operation of a research pro-ject and several of those involved have consolidatedtheir skills in new projects’(p. 433). Cheng

6

recognis-es that staff involvement in quality improvementsystems encourages a team focus on serviceimprovement and that ‘the process itself opens upopportunities for improvement and can trigger thecommitment and the will to deliver even better per-formance’(p. 6). Cundari

7

agrees, stating that ‘thisprocess has allowed the professional library staff toidentify customer needs, make changes to accom-modate those needs and establish the mindset thatquality improvement is an ongoing process’(p. 191).Foss

9

illustrates practically how the quality measure-ment process enabled a stronger staff focus on cus-tomer service and describes how a staff meetingwas called ‘to communicate the results of the surveyand identify solutions to provide better customerservice while enforcing policies’(p. 13). So perhapsinvolvement in quality initiatives can not only allowlibrary and information professionals to expand thebreadth of their own experience and skills but alsoenable the library team to share a common directionin the improvement of service delivery.The effects of quality improvement systems on

library users are perhaps more difficult to quantify;none of the included studies reported any directimpact on customers. This may be for several rea-sons; it may be that direct impact of qualityimprovement systems on library users is difficult tomeasure. Perhaps positive effects on library usersare implicitly assumed within the notion of serviceimprovement. As Lyon

16

puts it, ‘readers are entitled

to the best possible service and, to ensure that theyreceive it, there is a constant need to evaluate andre-evaluate the quality of services on offer’(p. 3). Orperhaps library services concentrate too much onthe concept of quality as an internal mechanism anddo not adopt a customer-facing stance to improvingperformance. No effects on patients were reportedin any of the studies included in the review, possi-bly for the reasons outlined earlier.Only one study

10

reported any effects on thewider organisation. Following involvement in anaccreditation initiative the library service estab-lished a more positive working relationship withthe hospital administration that was consequently‘more supportive of the library and receptive tomost of its requests’(p. 275). Improved communica-tion between the library and senior managementabout the quality improvement process could per-haps enable the wider organisation to take a greaterinterest and assume more ownership in serviceimprovement within the library. To achieve thisultimately requires library managers to demonstratehow a library in pursuit of quality improvement isbeneficial to the wider health community by align-ing its objectives with those of the organisation.An effect that was reported in some studies

4,20

that was not anticipated at the beginning of thereview was the impact that quality improvementsystems can have on the wider health library com-munity. Yeoman

20

developed an evaluation method-ology that was later applied to other evaluationprojects(p. 432). The work undertaken also ‘helpedguide further development of collaborative researchframeworks’(p. 432). Angel

4

states that the bench-marking exercise undertaken provided a forum for‘hospital librarians to communicate with each otherand lay groundwork for future cooperation’(p. 584).Sharing any knowledge that is gained from under-taking quality improvement initiatives in healthlibraries can be valuable learning for other libraryand information professionals and can only serveto inform and improve good practice in this area.

Assessment of Quality

Study design characteristics

Half of the studies5,8–10,12,13,18,19

included in thisreview assumed a qualitative approach (n = 8),




encompassing surveys (n = 3), a mix of surveys,questionnaires and critical incident techniques,

13,19

LibQUAL+TM9

and accreditation standards.Five studies used a mixture of qualitative and

quantitative methodologies to approach theirstudy,

7,11,16,17,20

of which four used question-naires

7,11,16,17

and one used a combination of ques-tionnaires, interviews, critical incident technique,vignettes and cost study.

20

Two studies used a case study methodology,6,15

using benchmarking as their main method of datacollection.One study solely employed quantitative analy-

sis,4

looking at benchmarking within a women’shospital.

Assessment of quality

No obvious critical appraisal tool was available,which met our requirements sufficiently. Therefore,the review team formulated and developed a qual-ity checklist based upon three existing criticalappraisal tools.

21–24

The results table summarisesthe studies meeting these quality criterions.Nine studies had clearly stated aims and objec-

tives,4,7,8,10,13,15,17–19

compared with seven5,6,9,11,12,16,20

thatdid not.

Quality of reporting

Three studies’ methods were not clearlydescribed,

6,10,20

and an element of uncertainty totheir design was apparent. One study’s reportingof methods was unclear, as there was uncertaintyas to whether the study instrument, a question-naire, had been sent to all participants.

12

Theremaining studies’

4,5,7–9,11,13,15–19

(n = 12) methodswere clearly described, thus providing clarity andallowing for future replication.There were studies that exhibited good research

practice; Leishmann15

carried out two studies atdifferent times of the year to test the reliabilityagainst each other. Cundari

7

utilised both purpose-ful and random sampling techniques to ensure theyreached an appropriate mix of participants whilstminimising the risk of introducing selection biasinto the sample. Urquhart

19

made sure all recipientsreceived a follow-up survey, respondents whoreported that the information provided was of

‘actual’ or ‘potential value’ was invited to inter-view. A random sample was also utilised in thecritical incident aspect of the study, again minimis-ing selection bias. In contrast, Huntley

13

did notinform who the interviewees were, Fana

8

clearlydescribed their methods, but the geographicalrestrictions they placed upon their selections mayhave inhibited the responses they received. Henry

12

appeared to survey all hospital administrators, butit was unclear as to what their sample size actuallywas. Yeoman

20

utilised a mix of selection methodsover seven sites, which hampered the quality oftheir results. Uniformity of selection methodswould have produced higher quality results.The majority of studies

4,7–9,13,15–20

made a goodcase for the approach they decided to take.Huntley

13

argued statistics and data alone wouldnot support any judgements made and Urquhart

19

based their research methodology upon analysis ofprevious studies’ methods, and successful method-ologies were employed. The studies

5,6,10–12

where acase was not made either decided not to justifytheir methodology or partly provided reasons forsome of their methods. Examples of such rangefrom Foss’

9

decision to email surveys but not pro-vide print copies for participants who may nothave access to their email to Henry’s

12

adaption oftheir survey from other, non-related MedicalLibrary Association surveys.Seven of the studies neglected to make their

aims and objectives clear5,6,9,11,12,16,20

compared withnine studies

4,7,8,10,13,15,17–19

who had them plainlystated.The majority of the studies’

4,5,7,8,11–13,15,16,18,19

(n = 12) results addressed their original researchquestion. The other five studies

6,9,10,17,20

either devi-ated from their original hypothesis

6,10,17,20

orappeared to have not posed one originally.

9

The conclusions drawn from their results werein the main based upon an objective interpretationof the results.

5,7–9,12,13,15,17–19

Where appropriate, theresearchers offered caution because of thematiccoding drawn from the researchers’ interpretation

8

or admittance that no evaluation mechanism is per-fect.

13

Three studies4,11,20

did not appear to basetheir conclusions objectively, Hadley’s

11

conclusionwas unrelated to the survey and Angel

4

inferredwhy certain statistics were lower than others. Itwas unclear with two of the included studies

6,10




whether their conclusions were truly objective. InGluck’s

10

case, the results were anecdotal so it wasdifficult to assess.Related to this was the appropriateness of the

sample size to conclusions drawn from theirresults. Ten studies

5–7,11,12,16–19

made conclusions thatwere apt, in relation to the size of the sample.Angel’s

4

conclusions were not in relation to sam-ple size, and it was unclear in the following stud-ies: Fama,

8

Foss,9

Gluck,10

Huntley,13

Leishmann15

and Yeoman.20

Strengths and Limitations

A key challenge in conducting the review wasagreeing a definition of quality to inform the inclu-sion and exclusion criteria. Although the reviewteam spent much time deliberating over the inclu-sion criteria regarding quality, the different experi-ences of the team coupled with input from theadvisory board provided a comprehensiveapproach that arguably would not have beenachieved by a single reviewer. The decision tolimit the review to health libraries and English lan-guage omitted potentially valuable literature fromother sectors and countries; this was done to limitthe literature to a manageable volume. The reviewteam worked collaboratively where time and logis-tics allowed with all members commencing eachstage of the review together to ensure a consistentapproach. The full text was filtered by at least twomembers of the team; however, most criticalappraisal was conducted by a single reviewer.

Conclusions

The literature examining the outcomes of the useof quality improvement systems in health librariesis sparse and needs updating. There may be qualityimprovement work being undertaken on a moreinformal level within health libraries. To make thisknowledge more explicit and accessible to a wideraudience, we recommend further primary researchis conducted in health libraries focusing uponthe outcomes of utilising quality improvementsystems.Several tools were used in the studies to mea-

sure quality in a wide range of library services.The literature reveals a number of positive out-

comes of utilising quality improvement systemsrelating to the library service, library staff and thewider health organisation. However, no studieswere able to identify any effects on library usersor patients. This may be due to the fact thatimproved service to customers (and thus improvedpatient care) is implied following the implementa-tion of quality improvement systems. It may, how-ever, point to a need for health libraries to viewquality improvement as a customer-focused exer-cise rather than an internal mechanism. One out-come that was not anticipated at the beginning ofthe review was that quality improvement systemsalso have an effect on the wider library commu-nity, prompting health librarians to collaborate,network and share professional knowledge aroundthe subject.One of the objectives of the review was to

examine the outcomes of quality improvement sys-tems on library staff, library users, patients and theorganisation. However, none of the included stud-ies set out to do this as their primary aim butinstead described the quality improvement process.There is a tendency in the literature to focus onthe methodology surrounding quality improvementsystems rather than what prompts services to adoptquality improvement systems and the impact oftheir introduction. Many studies were excludedfrom the review because of lack of reporting onthe outcomes of utilising quality improvement sys-tems.Initially the review team had hoped to gain

some understanding of the barriers that preventlibrary and information professionals from imple-menting quality improvement systems. With hind-sight, however, an answer to this question isunlikely to be acquired from the literature and maybe best addressed through further primary researchin this area. An exploration of the outcomes ofutilising quality improvement systems in otherlibrary sectors may also provide valuable insightfor health libraries.The literature demonstrates how quality

improvement systems can result in improvementsto strategic planning, promotion, new andimproved services and staff training. However,many health libraries utilise quality improvementsystems without translating the data into serviceimprovements. Health libraries need to go one step




further by assessing and reporting on the outcomesof their quality improvement systems.

References

1 Booth, A. What is quality and how can we measure it? Shine2003. Available at: http://www.shinelib.org.uk/cpd/journal/volume_43/what_is_quality_and_how_can_we_measure_it(accessed 1 May 2012).

2 Strategic Health Authorities Leads. Library Quality Accredi-tation Framework. 2010. Available at: http://www.libraryser-vices.nhs.uk/forlibrarystaff/lqaf/ (accessed 1 May 2012).

3 Library and Information Health Network North West. Avail-able at: http://alignmentprojectnwlibrarians.pbworks.com/w/page/11046836/FrontPage (accessed 1 May 2012).

4 Koufogiannakis, D., Booth, A. & Brettle, A. ReLIANT:reader’s guide to the literature on Interventions Addressingthe Need for education and Training. Library and Informa-tion Research, 2006, 30, 44–51.

5 CriSTAL. Appraising a User Study. Key questions to helpyou make sense of a user study. Sheffield, UK: Universityof Sheffield, School of Health and Related Research(ScHARR). Available at: http://www.shef.ac.uk/scharr/eblib/use.htm (accessed August 2009).

6 Cawthorne, J. E. Leading from the Middle of the Organiza-tion: an examination of shared leadership in AcademicLibraries. The Journal of Academic Librarianship, 2010, 36,151–157.

7 Brettle, A., Maden-Jenkins, M., Anderson, L., McNally, R.,Pratchett, T., Tancock, J., Thornton, D. & Webb, A. Evalu-ating clinical librarian services: a systematic review. HealthInformation and Libraries Journal, 2011, 28, 3–22.

8 Angel, S. H. & Mackler, L. G. A benchmark instrumenttested in women’s hospital libraries. Bulletin of the MedicalLibrary Association, 1996, 84, 582–585.

9 Campbell, K., Davis, A., McGowan, J. & Shorr, R. A post-merger hospital library collection survey and data analysisindicated better resource allocation and user satisfaction.Journal of the Canadian Health Libraries Association,2004, 25, 29–38.

10 Cheng, G. Y. T. The use of benchmarking in improving thelibrary service quality of devolved management. Journal ofthe Hong Kong Library Association 1996, 18, 1–14.

11 Cundari, L. & Stutz, K. Enhancing library services: anexploration in meeting customer needs through total qualitymanagement.. Special Libraries, 1995, 86, 188–194.

12 Fama, J., Berryman, D., Harger, N., Julian, P., Peterson, N.,Spinner, M. & Varney, J. Inside outreach: a challenge forhealth sciences librarians. Journal of the Medical LibraryAssociation, 2005, 93, 327–337.

13 Foss, M., Buhler, A., Rhine, L. & Layton, B. HSCL Lib-QUAL+ 2004: from numbers and graphs to practical appli-cation. Medical Reference Services Quarterly, 2006, 25, 1–15.

14 Gluck, J. C. & Hassig, R. A. Raising the bar: the impor-tance of hospital library standards in the continuing medicaleducation accreditation process. Bulletin of the MedicalLibrary Association, 2001, 89, 272–276.

15 Hadley, S. Health libraries week 2005 – Evaluation usingan online survey tool. Health Information on the Internet,2006, 50, 9.

16 Henry, B. Continuous quality improvement in the hospitallibrary. Bulletin of the Medical Library Association, 1993,81, 437–439.

17 Huntley, J. L. & Orrok, P. The hospital library profile as anevaluation mechanism. Bulletin of the Medical Library Asso-ciation, 1970, 58, 403–414.

18 Lee, T. Exploring outcomes assessment: the AAHSL Lib-QUAL+ experience. Journal of Library Administration,2004, 40, 49–58.

19 Leishman, J. Improving interlibrary loan quality throughbench-marking: a case study from the Health Science Infor-mation Consortium of Toronto. Health Libraries Review,1995, 12, 215–218.

20 Lyon, E. The questionnaire – a quality control method foronline searching? Health Libraries Review, 1989, 6, 3–19.

21 Mi, M. & Gilbert, C. M. Needs assessment: prerequisite forservice excellence. Journal of Hospital Librarianship, 2007,7, 31–52.

22 Robinson, L. & Bawden, D. Evaluation of outreach servicesfor primary care and mental health; assessing the impact.Health Information and Libraries Journal, 2007, 24, 57–66.

23 Urquhart, C. J. & Hepworth, J. B. Comparing and usingassessment of the value of information to clinical decision-making. Bulletin of the Medical Library Association, 1996,84, 482–489.

24 Yeoman, A., Tyler, A., Urquhart, C. J. & Cooper, J. M. Themanagement of health library outreach services: evaluationand reflection on lessons learned on the VIVOS project.Journal of the Medical Library Association, 2003, 91, 426–433.

Received 23 December 2011; Accepted 15 May 2012

Appendix 1: Search Strategy

Sources Platform/Publisher

AMED (Allied and

Complementary Medicine)

NHS Evidence Healthcare

Databases Advanced

Search

BNI (British Nursing Index) NHS Evidence Healthcare

Databases Advanced

Search

Cinahl (Cumulative Index to

Nursing and Allied Health)


Databases Advanced

Search




Table . (continued)

Sources Platform/Publisher

Cochrane Library (Database of

Systematic Reviews,

Controlled Trails Register,

Database of Abstracts of

Reviews of Effectiveness,

Methodology Database)

Wiley InterScience

Embase NHS Evidence Healthcare

Databases Advanced

Search

ERIC (Educational Resources

Information Centre)

Dialog

Google & Google Scholar Google

Health Information and Libraries

Journal (HILJ) (journal hand

search)

Wiley

HMIC (Health Management

Information Consortium)


Databases Advanced

Search

Index to Theses Expert Information Ltd

Intute Mimas

LISA (Library and Information

Science Abstracts)

CSA (Cambridge Scientific

Abstracts)

LISTA (Library, Information and

Technology Science Abstracts)

EBSCOHost

Medline NHS Evidence Healthcare

Databases Advanced

Search

Performance Measurement and

Metrics (journal hand search)

Emerald

Proquest Library Science Proquest

Psycinfo NHS Evidence Healthcare

Databases Advanced

Search

Web of Knowledge ISI (Institute of Scientific

Information)

Email discussion lists

The following email discussion lists were [email protected]@[email protected]

Search terms

Terms used in database searches are listed below.Where possible, a wild card (* in the list of termsbelow) was used in individual database searchstrategies, to retrieve variations in the endings orspelling of terms. Database thesauri were utilisedwhere appropriate.‘evidence service*’ or ‘information service*’ or

‘knowledge service*’ or library or librariesANDdental or dentist* or health or healthcare or hos-

pital or medical or medicine or ‘National HealthService’ or NHS or nurse* or nursing or PCT or‘primary care’ or SHA or ‘Strategic HealthAuthority’(if applicable, i.e. do not use if searching a

health related database, e.g. CINAHL)ANDaccredit* or ‘balanced scorecard’ or benchmark*

or ‘codes of practice’ or helicon or improv* orindicator* or Libqual or LQAF or measur* ormonitor* or performance or quality or question-naire or Servqual or standard* or statistic* orsurveyANDassess* or evaluat* or effect* or affect* or

impact.




do quality improvement systems improve health library services? a systematic review

Documents