audit in the therapy professions: some constraints

Quality in Health Care 1996;5:206-214

Audit in the therapy professions: some constraintson progress

Sarah Robinson

Nursing ResearchUnit, King's College,London UniversitySarah Robinson, seniorresearch fellow

Correspondence to:Sarah Robinson,Nursing Research Unit,King's College, LondonUniversityAccepted for publication21 August 1996

Crown copyright reserved1995, published bypermission of the controllerof Her Majesty's StationeryOffice.

AbstractAims-To ascertain views about con-straints on the progress of auditexperienced by members of four of thetherapy professions: physiotherapy, occu-pational therapy, speech and languagetherapy, and clinical psychology.Methods-Interviews in six health servicesites with a history of audit in theseprofessions. 62 interviews were held withmembers of the four professions and 60with other personnel with relevantinvolvement. Five main themes emergedas the constraints on progress: resources;expertise; relations between groups;organisational structures; and overallplanning of audit activities.Results-Concerns about resources fo-cused on lack of time, insufficient finance,and lack of access to appropriate systemsof information technology. Insufficientexpertise was identified as a majorconstraint on progress. Guidance ondesigning instruments for collection ofdata was the main concern, but help withwriting proposals, specifying and keepingto objectives, analysing data, and writingreports was also required. Althoughsources of guidance were sometimesavailable, more commonly this was not thecase. Several aspects of relations betweengroups were reported as constraining theprogress of audit. These included supportand commitment, choice of audit topics,conflicts between staff, willingness to par-ticipate and change practice, andconcerns about confidentiality. Organisa-tional structures which constrained auditincluded weak links between heads ofprofessional services and managers ofprovider units, the inhibiting effect ofchange, the weakening of professionalcoherence when therapists were splitacross directorates, and the ethos ofregarding audit findings as businesssecrets. Lack of an overall plan for auditmeant that while some resources wereavailable, others equally necessary forsuccessful completion of projects werenot.Conclusion-Members of four of thetherapy professions identified a widerange of constraints on the progress ofaudit. If their commitment to audit is tobe maintained these constraints requireresolution. It is suggested that suchresolution not only requires additionalresources and greater availability of

expert advice, but also that these aredirected towards the particular needs ofthe four professions. Moreover, a forum isrequired within which all those with astake in therapy audit can acknowledgeand resolve the different agendas whichthey may have in the enterprise.(Quality in Health Care 1996;5:206-214)

Keywords: therapy professions, audit

IntroductionThis paper is concerned with the developmentof audit in some of the healthcare professionsin the United Kingdom. A considerableamount of time, effort, and indeed money hasbeen invested in medical audit. Much lessmoney, however, has been invested in thedevelopment of nursing audit, and even less inaudit in the therapy professions discussed inthis paper. Quality assurance activities in thetherapy professions began in the mid-1980swhen each of the professional bodies producednational standards for a wide range of aspectsof care.' Subsequently, individual practitionersinitiated audit of many of these standards.2"

The development of audit in the therapyprofessions as a whole was then givenconsiderable impetus in 1991 by theDepartment of Health's decision to fund a

AuditOMedical audit is a systematic criticalanalysis of the quality of medical care,including the procedures used fordiagnoses and treatment, the use ofresources, and the resulting outcome andquality of life for the patient

*Medical audit was initially motivated byprofessional concerns and promoted byindividual enthusiasts - managerialinvolvement increased alongside moves tomake the service more commerciallyoriented

*Audit in the nursing and therapyprofessions was similarly motivated byprofessional concerns and increasinglybecame the subject of managerialinvolvement

*Although uniprofessional activities con-tinue in medicine, nursing, and the therapyprofessions, much more emphasis is nowplaced on multiprofessional and clinicalaudit - the approach advocated in the1990s by the Department of Health

206

Audit in the therapy professions

three year programme to "enable thewidespread take up of audit in the nursing andtherapy professions".4 Although therapists, likemedical staff, originally conceived of audit as a

uniprofessional activity, they became increas-ingly involved in multiprofessional activitiesafter the promotion of clinical audit by theDepartment of Health.5The volume of research which now exists on

participation by medical staff in audit activitieshas identified a wide range of constraints whichmay hinder progress. These constraints includelack of time and guidance,6 authority to imple-ment findings,7 and concerns about the extentto which findings are regarded as confidential.8Comparable research for the therapyprofessions is scarce; it is however, importantto know what constraints may be experiencedby therapists if their quality assurance activitiesare to be sustained. Consequently, investiga-tion of constraints was one of the main aims ofa project funded by the Department of Health,commissioned to investigate audit activities inoccupational therapy, physiotherapy, speechand language therapy, and clinical psychology,and to make recommendations for future prac-

tice. This focus on constraints was also a

response to an earlier investigation of audit inthese professions which had concluded thatmore facilities and support for audit activitiesshould be provided for clinicians.9

MethodsThe research design was a qualitative studywith semistructured interviews held withhealth service staff in six sites. Interviews were

chosen rather than questionnaires to collectdata, as little previous research existed to guidethe formulation of a questionnaire, and discus-sions with key informants indicated a complexsituation not readily amenable to fixed choicequestions. Concentrating the interviews at a

few sites, rather than interviewing a sample ofstaff spread nationwide, meant that audit couldbe considered in the context in which itoccurred.

SELECTION OF SITESSix sites were selected; each was a unit oforganisation in the health service (usually a

trust) in which one or more of the four profes-sions were actively engaged in audit. The selec-tion was purposive to ensure a diversity ofthose criteria most likely to affect the nature ofaudit: client group, clinical setting, organisa-tional structures, and regional policies. Theprocess of selecting the six sites entailedsearching audit databases and published workand consultations with the project's advisorygroup, which was comprised primarily ofmembers of the four professions.

SELECTION OF INTERVIEWEES

At each site our aim was to interview all mem-bers of the four therapy professions involved inaudit activities, as well as other "stake holders"in these endeavours. The stake holdersincluded medical and nursing staff involved inmultiprofessional audit, managers of providerunits, purchasers, and quality advisors and

coordinators. A letter requesting agreement to

be interviewed was sent to each person identi-fied. This stressed the voluntary nature of par-ticipation, together with assurances ofconfidentiality and anonymity. Nearly all thoseapproached agreed to participate.

APPROACH TO INTERVIEWINGSix main topics (box) were explored with allparticipants using a semistructured interviewschedule. Identification of items within thetopic areas and associated probes drew on dis-cussions with key informants and on theprofessional and research literature on audit.The schedules were piloted in a seventh sitebefore the main field work to clarify terminol-ogy, assess relevance of items included, identifyomissions, and determine acceptability of topicsequencing and overall duration of interview.

The interviews were recorded on audiotape to:(a) have a verbatim record; (b) avoid breaks in theconversation to write notes; and (c) enable theresearcher to be free to detect cues such as

unease or hesitancy that indicated the need toprobe a subject further. Respondents were askedfor their views about audit in their professiongenerally, as well as drawing on their own

experience of participation in audit activities.The tapes were transcribed by the interviewer toprovide a full account for analysis. Reasons fornot tape recording included refusal of theinterviewee and the occasional technical mishap.Immediately after such an event, the interviewerwrote up a comprehensive account from notestaken during the interview to ensure as great a

degree of comparability as possible with the tran-scriptions of tape recorded interviews. Mostinterviews lasted about an hour. Many expressedsurprise that they had talked for so long; as one

interviewee said:

"I would never have thought I had so muchto say about audit".

ANALYSISThe aim of the analysis was to identify keyissues in the development of audit, and therange of experience and views in relation toeach; it was not to produce precisemeasurement of the kind "20 people said this,whereas 25 said that...". The content of thetranscripts was analysed with the frameworkapproach described by Ritchie and Spencer.'"Familiarity with content of the transcripts was

gained by repeated reading, enabling the main

Topics explored in the interviews:* Audit activities in the therapyprofessions, in progress at local andregional level* Initiation and management of audit* Own involvement in audit* Resources and support for undertakingaudit* The impact of audit on the delivery ofcare and on professional development* Future directions for the developmentof audit

207

Robinson

themes and subthemes within the data set as awhole to be identified. Some of these themesreflected the topics in the interview schedules,whereas others related to new issues raised byinterviewees. Each interview was then plottedon a matrix by summarising the responsesrelating to each of these themes.

ResultsA total of 122 people were interviewed. Sixtytwo were members of the therapy professions

heads of professional service as well as staffat each of the clinical grades. The other 60interviewees were the other stake holders inaudit. Taken together, the six sites in whichthese interviewees worked represented a diver-sity of client groups (elderly people, those withmental health problems, learning disabilities,physical disabilities, and rehabilitation needs);and of settings (hospitals of varying size, com-munity units, and services).Over 100 uniprofessional and multiprofes-

sional audit activities at varying stages of com-pletion were identified. The followingexamples illustrate the diversity of aspects ofcare being investigated: improving the qualityof record keeping; achieving equity of access tophysiotherapy services; reviewing case notes toidentify antecedents of incidents of self harm;improving information provided for strokepatients and their carers; assessing outcomes ofa rehabilitation programme for patients withhead injuries; developing a system to measure

quality of life outcomes for long term mentalhealth clients; ascertaining the relationbetween waiting time after referral for osteoar-thritis of the knee and subsequent outcome;and designing new treatment protocols andnew staff training packages for the care ofpatients with dysphagia.The therapists who were interviewed

regarded audit as responsible for some positiveimpacts on the delivery of care, and on themorale and careers of individual members ofthe four professions. (These findings havealready been reported in full by Kogan et al. 11)At the same time, however, they identifiedmany constraints to the progress of audit andthese are reported here. Five main themes ofconstraints emerged from the analysis of thetranscripts: resources; expertise and advice;relations between groups; organisational struc-tures; and the extent to which an overall planfor audit existed. The main focus of resultspresented here is the constraints experiencedby therapists; however, the extent to which theother groups of interviewees endorsed thetherapists' views or offered alternative perspec-tives is also included.

RESOURCESTime

Almost all interviewees identified lack of timeas the main constraint on progress; findingconvenient meeting times for all those involvedas well as time to undertake the work entailed.Meetings were especially problematic whenprojects involved several health professions or

were based in the community.

"It's more difficult to get everyone togethergeographically, plus many communityworkers are part timers and it's hard to finda time when everyone can get together. Itshouldn't put us off, but it does."

Few interviewees had been given a time allow-ance for audit activities and so these were

undertaken in clinical time or in their own

time. Audit being held in clinical time led toconcerns about compromising care to spendtime on audit. A junior member of staff said:

"I haven't got the time to do it and I feel Ihave to give patient care priority over thisaudit project. It's harder to give time tothings that are long term."

And a head of service commented:"It's hard for staff to take on extra work,especially when there is a waiting list. Theyhave to weigh it up do I spend time on

audit or on patient care and they choose thelatter -there's no contest as far as they are

concerned."

Long intervals between audit meetings andlack of time for the work itself, meant thatproject schedules slipped; this had become dis-heartening, particularly after an enthusiasticand committed start.The other interviewees endorsed the views

expressed by the therapists that lack of timewas the main constraint on audit activities. The

Constraints on the progress of audit(1) LACK OF RESOURCES* Time to undertake audit work and to

meet with colleagues involved in audit* Funding for equipment and secretarial

support* Appropriate systems of information tech-

nology(2) LACK OF EXPERTISE OR ACCESS TO

ADVICE IN

* Writing proposals* Specifying and keeping to objectives* Designing instruments* Analysing data* Writing reports(3) RELATIONS BETWEEN GROUPS* Lack of commitment by managers to use

findings of audit projects* Restrictions on choice of audit topic* Conflict between staff* Reluctance to participate and change

practice* Concerns about confidentiality(4) ORGANISATIONAL STRUCTURES* Weak organisational links between

professionals and managers of providerunits

* Change in organisational structure* Loss of professional confidence when

groups split across directorates* Insecurity of audit facilitators' posts* Restrictions on sharing audit findings in a

climate of competition for contracts(5) LACK OF AN OVERALL PLAN FOR AUDIT

208


quality coordinators thought that the clinicalcommitments of therapists were too heavy forthem to undertake audit. Some of the moresenior managers took the view that theproblem was their responsibility.

"If you're going to do it, you should do itproperly, and allow people time to do itproperly, particularly the heads of service."

Various suggestions to improve the situationwere made by therapists: increasing overallstaff numbers, appointing audit assistants tocollect and collate data, and establishing banksystems to provide cover for those involved.Suggested patterns of work included allocatinga proportion of everyone's working week toaudit, and allocating several consecutive daysto those involved in large scale projects as thisled to better progress, especially at the outset.

FundingSome projects were carried out within existingbudgets whereas others had small sumsallocated for specific items; some had receivedfunding part way through, whereas othersreceived funding of several thousand poundsfrom the outset. When funding wasinadequate, clinical managers said they could"lose" costs such as photocopying, stationery,and secretarial help, in other budgets.However, when this was combined with theamount of personal, unpaid time spent onaudit then, in the view of many, its true costwas not properly accounted.Lack of finance for audit was not disputed by

the other interviewees. The quality coordina-tors, in particular, saw this as a majorconstraint: firstly, there was little moneyavailable and it was not easy to obtain;secondly, once a project that was funded cameto an end there was a loss of impetus tocontinue with audit activities.

Information technologyAlthough all the therapists acknowledged thatinformation technology could contribute to theprogress of audit, only a few had access to suchsystems. Several projects had been undertakenwithout word processing or computerised datacapture facilities; all information was stored inhandwritten records and analysed manually. Asa head of service explained, this had oftenproved extremely time consuming.

"In many of our services, there is not anefficient system, many of us are still copingon manual systems. For the period of theproject we kept manual records, but it took26% of staff time, and was virtuallyunusable because data on one thing didn'tmatch up with others."

Others had been asked to enter data for auditprojects into systems that, in their view, wereeither not appropriate to the project's require-ments or were difficult to use.

"This database is the background for theproject, as well as being used for recordsgenerally. It's a new system, so there havebeen lots of hiccups and it's very slow to putthings in. It's also a problem to get

information out. When we kept manualrecords I could tell you at the end of everymonth how many patients, under whichconsultant, of which type we had treatedand details of treatment. Now it's all in acomputer which I can't access from here.The man in charge doesn't know what wewant and what we want tends to evolve any-way."

Considerable emphasis was placed by therapystaff on consultation about the kinds of systemsthey needed, rather than presenting them withpackages developed for other users. Moreover,they said that staff needed time and tuition tobecome competent with new systems.Although members of the other groups were

in agreement with the views about informationtechnology expressed by the therapists, someof the managers said that they had encounteredresistance when they had attempted tointroduce a new system. One described the fol-lowing reaction:

"I won't have anything to do with that, notat all. I want my own old word processor,nothing's going to get me away from that."

EXPERTISE AND GUIDANCEFive aspects of undertaking audit projects wereidentified as needing expertise: writing propos-als; specifying and keeping to objectives;designing instruments; analysing data; andwriting reports. Sources of guidance specifiedby interviewees included colleagues with auditexperience, holders of posts with a qualityassurance remit, staff of specialist audit orresearch units, published work, and auditinformation networks. Some heads ofprofessional services said that although courseson audit were available, their continuingeducation budget was limited, and staff choseto go on clinical courses rather than on auditcourses when faced with a choice. Intervieweesdiffered considerably in their experience ofaudit; a few had been involved for many years,others were just starting out on their firstproject. An indication of the lack of depth ofaudit experience, however, was provided bythose who said that having completed or evenjust started one project, they were then seen as"experts" and asked to advise others.

"While we were doing the project, and find-ing out how to do it, suddenly our nameswere being given to loads of people, and wewere being called in to advise them on theirprojects."

Without expert help in writing proposals, someinterviewees doubted whether their bid forregional or provider unit money would havebeen successful. Difficulties had beenexperienced in specifying the focus of an auditproject with sufficient precision to moveforward into effective collection of data. Othersthought that their group had lost its way byattempting to cover too many outcomes. A dis-cussion with someone more experienced, usu-ally an audit facilitator or coordinator, hadoften proved of great help in getting a projectback on track. Clarity of objectives was seen asimportant, not only because it increased the

209

Robinson

likelihood that the project would be effective,but also because commitment was lost whenpurposes became unclear.

"I'm in favour of the project but I am notsure what are the next best steps with it andthat's why I find it hard to find time for it."

The need for expert advice and guidance was

expressed most often over designing instru-ments for collection of data for example, a

semistructured interview to obtain patients'views, or a questionnaire to staff on their train-ing needs. Although examples were cited ofhelp given for this, a much more common

finding was that people wanted guidance, butdid not know from whom or from where itmight be obtained. Consequently, manyinstruments were designed without help and inignorance of the range of existing instruments.

"At the moment, information about how todo it is patchy. We have had to make all ourown contacts, we had to devise thequestionnaire ourselves - there's been verylittle guidance"."We've had very little advice on how to designforms and instruments. There probably isguidance, but it's not readily accessible."

Many of those who had considerableexperience of audit activities and of instrumentdesign expressed concern about people's lackof awareness of issues of validity and reliability."My concern was ... almost the naivety as tohow simple it is to undertake audit, and Ifelt people were going to waste an enormous

amount of effort and time drawing up halfbaked, non-scientific research withoutrealising how difficult it is to construct goodsimple audit to change things."

A common problem was one of getting boggeddown with excessive data; timely advice aboutfocusing the analysis had been the reason forsuccessful completion of several projects. Writ-ing up findings into an accessible form for dis-semination was difficult for those approachingthe task for the first time.

"Staffmay be willing to collect the data, butwe do need help to collate and interpret it."

Again, experienced input at this stage was seen

as essential if the effort of undertaking auditwas not to be wasted.Nursing staff were more likely than their

medical colleagues to express a need for guid-ance with audit activities. Concern about thevalidity of results when therapists were inexpe-rienced was expressed by some of themanagers and quality coordinators withresponsibility to initiate change. As one put it:

"We have considerable difficulty ininterpreting results."

And another talking about quality of question-naires:

"Some are very well thought out, but thereis some absolutely dire stuff- if work is tobe used then it has to be done with objectiv-ity."

RELATIONS BETWEEN GROUPSSeveral aspects of relations within and betweenprofessions and between professionals and

managers of provider units were reported asaffecting the progress of audit: these includedevidence of commitment, choice of audittopics, personal conflict, willingness to partici-pate and change practice, and concerns aboutconfidentiality.

Commitmentfrom managersSeveral heads of service said that although, intheir view, professional staff were unanimouslycommitted to the principles of audit, this wasnot always matched by support andcommitment from managers. Instances werecited of findings from audit projects that hadnot been used, or even considered, particularlythose concerned with delivery of care over longperiods (for example, for clients with continu-ing mental health problems). This was a disin-centive to further participation for thosealready stretched to capacity with clinical com-mitments.

"Junior staffwho have joined audit activitiesbecome dispirited if it's not clear that thework is being used properly."

The view that lack of interest by managersacted as a disincentive to further participationin audit, also emerged from interviews withmedical and nursing staff, purchasers, andquality coordinators. One quality coordinatorexpressed her views thus:

"...there is a desperate need for moresupport from senior managerial staff andacknowledgment for the work that is beingdone, rather than the occasional pat on theshoulder."

Managers themselves, however, offered twodifferent reasons for the lack of action on find-ings from audit projects. Several managers saidthat in their view audit data were not collectedsystematically and so they found it difficult tointerpret results and decide on a course ofaction. Secondly, some of the most seniormanagers interviewed, including a chief execu-tive, observed that the success of trusts isjudged to a disproportionate extent on the suc-cess of high profile medical work, and that nosystematic consideration is given to auditreports from those professions with a muchlower profile. Other managers stressed thatthey did act on findings, and alwaysacknowledged receipt of audit reports andthanked the staff involved.

Choice of topicThe fieldwork provided many instances ofaspects of care that had been of sufficient con-cern to motivate members of one or more ofthe four professions to undertake an auditproject. An increasing managerial input intothe choice of topics for audit projects was,however, identified as a constraint on attemptsto audit the more complex aspects of care.Managers emphasised cost efficiency issuesand topics that readily lent themselves tomeasurement. An often occurring example wasthe number of people seen within a givenperiod. Many of those interviewed, however,

210


thought that audit should be focusing on thequality of the encounter, not on its duration,and that the duration was an inappropriate useof staff time.

"My staff had to spend time on how longeach patient waited - the paper work wasenormous. It was pointless because wealready had a system in which no one waitedmore that half an hour- so I said, "Can wejust record exceptions?" - but they didn'tlike this. Much of the paperwork ismeaningless and it takes up staff time."

Many of those with long experience of auditthought that whereas the original motivationhad been to improve the quality of care, themanagerial agenda was moving to one ofcutting costs. Although not disputing thatdelivering high quality care as efficiently aspossible should be the aim of the service, con-cern was expressed that quality was accorded alower priority than hitherto. Not surprisingly,perhaps, the managers themselves did notdescribe the purpose of audit as one of reduc-ing costs; they emphasised improving servicesand using resources effectively.

Conflict between staffMost commonly interviewees described goodstaff relations which were conducive to theconduct of audit; instances of conflict were cited,however, which acted as a deterrent. One seniortherapist who also had a remit as an audit coordi-nator described her experience of this.

"The two groups, community and acute,don't talk to each other. I've had one hell ofa time getting them together to even talkabout possible assessment tools that theymight use...I've chaired their audit meetingscontinually to try to keep them on a happyfooting. If you've got conflict within theteam anyway, trying to get them to questioneach others' practice is very difficult..."

Reluctance to participate and to changeSome interviewees had experienced areluctance by colleagues to participate in auditbecause of fear that ensuing recommendationsmight mean an unwelcome change in practice,or a reduction in autonomy, whereas othersdescribed a reluctance to accept changes whichhad been indicated.

"It's so stupid because it's very personalisedas opposed to improving quality of care forpatients ... it's sad because the service knowsit has a problem, but just doesn't want tolook at it.""...You feel as if you are talking to a brickwall, and that all the findings in the worldyou produce aren't going to make themchange their practice, and they don't realisethat this is going to have an effect on the restof the team."

Several of the more experienced intervieweesstressed that audit projects, and their possibleimplications, should be discussed from theoutset with all of those on whom findingsmight have an impact, as this was far more

likely to achieve change than confronting peo-ple with a request for change out of the blue.Managers also attributed reluctance of profes-

sionals to participate in audit to concerns aboutthe changes which might result: attention wasdrawn, in particular, to a dislike of audit of theskill mix of teams. Other constraints onparticipation cited by managers included adislike of audit that led to greater client empow-erment and the view that audit wasadministrative, rather than professional work.

Concerns about confidentialityConfidentiality of audit findings was a cause ofconcern for some interviewees. They said thatimproving patient care through audit hadtraditionally depended on colleagues learningfrom each other by being willing to shareresults, both good and bad. Willingness to par-ticipate was based on guarantees that proceed-ings were absolutely confidential to thoseinvolved. They thought that multiprofessionalaudit in particular was now making it moredifficult to maintain confidentiality as severalgroups were involved; at some sitesmisunderstandings had already arisen as towho should share findings. Concern was alsoexpressed about managers having access toaudit findings as it was thought that they mightuse them to penalise staff in some way. Sometherapists said that they now selected "safe"topics that they could not subsequently be"whipped by", although they thought that theend result of this strategy would be a failure toassess care given "warts and all".None of the managers who were interviewed

held the view that audit findings should remainconfidential to the professionals involved;rather that the information should be availableto all with an interest in, or responsibility for,the service in question. Some, however, didacknowledge that audit was threatening toprofessional staff.

"They think it is their capabilities that arebeing audited, rather than providingobjective information on the servicesprovided."

None the less, views expressed by othermanagers indicated that in some instancesprofessional fears might not be withoutfoundation; for example, one manager said:

"We will want to use audit as a form ofsupervision and accountability."

ORGANISATIONAL STRUCTURESOrganisational links between heads ofprofessional services and managers of providerunits affected implementation of audit. In siteswith strong links, instances were cited of man-agers acting swiftly in response to auditfindings. When links were weak, however, auditfindings requiring managerial authority forimplementation were said to "get stuck", asthere was no route through which they couldbe channelled. Managers in these sitesconfirmed that they did not systematically

21

Robinson

consider reports on audit in the therapyprofessions; rather their attention was focusedon medical audit.Some sites were in the throes of considerable

organisational change at the time of thefieldwork and managers expressed the viewthat this had prevented audit activities fromgetting underway. The nature of these changesmeant that in some sites many intervieweeswere working in directorates other than the onein which their head of service was based. Someof the more senior therapy staff said that thisled to loss of professional coherence, which inturn undermined confidence to undertakeaudit.

Various audit databases and advice networkshad been established and provided informationabout methods used by others. This wasparticularly important for those relatively newto audit and lessened the likelihood of"reinventing the wheel". The quality assurancepostholders responsible for these services,however, said that they were "too thinlyspread" to meet all the demands on their time.Moreover, in some sites there was anxiety thattheir expertise might be lost altogether, aseither the future of the audit post itself wasuncertain, or the present incumbent hadbecome dissatisfied with a series of short termcontracts and was contemplating a move to amore secure job.A view emerging from the interviews with

therapists, although not from managers, wasthat the introduction of the quasi-internalmarket into health care and the separation ofproviders from purchasers had led to findingsfrom audit projects being regarded asconfidential to the trust. Such information wasincreasingly included in contract submissionsand so should not be shared with others withwhom the unit was potentially in competition.

"Some trusts are now banning sharinginformation about audit, because it'sbusiness secrets. This government has saidyou are in competition."

Some of the therapists thought that it wassomewhat ironic that just as involvement inaudit had become more widespread amongmembers of the four professions, the ethos ofcompetition between provider units might actas a deterrent to establishing a common bodyof knowledge and expertise.

AN OVERALL PLAN FOR AUDITIn all sites the interview data indicated thatalthough much audit activity was in progress,overall planning was lacking. Some projects,for example, had been well resourced withfacilities and expertise, but had failed to makean impact due to poor links with appropriatelevels of management. Other projects hadfailed to make progress despite initial commit-ment, because of time constraints of thoseinvolved. Some interviewees had been given atime allowance, but said that they failed tomake progress because of lack of advice as tohow to proceed at a particular stage. Some ofthe projects that had achieved an impact hadsucceeded due to the motivation and commit-

ment of staff involved although they had beenprovided with little in the way of resources.Future commitment, however, was said to bedependent on this resource deficit being madegood.

This lack of an overall plan was affirmed inthe views expressed by the other groups ofinterviewees. Some of the senior managersexpressed concern about the lack of coordina-tion of audit activities across the trust and sawthe development of an overall plan as one oftheir immediate priorities.

DiscussionThis study of audit in four of the therapy pro-fessions indicated that although much activitywas in progress, a wide range of constraints toprogress had been experienced. Theseconstraints (summarised in the box) deterredstaff from undertaking audit, lessened the like-lihood that projects would be completed, andrestricted the choice of aspects of care likely tobe selected for audit.

The main constraintsAvailability of time was the main constraint toprogress; other resource constraints identifiedwere lack of finance for equipment andsecretarial support, and inadequate orinappropriate information technology systems.Many interviewees did not have access to thekind of expertise that they thought wasnecessary, particularly in the design ofinstruments for collection of data and theinterpretation of results. Concerns wereexpressed about the validity and reliability ofresults in the absence of appropriate advice,and at the loss of such advice that was availableas some audit facilitators were seeking a moveto more secure employment.

Aspects of relations between groups whichserved as a deterrent to participation in auditincluded loss of enthusiasm when auditfindings were not used by managers, anxietiesabout the effects of recommendations on indi-vidual clinical practice, and fears thatincreasing loss of confidentiality meant thatmanagers could use audit findings critical ofpractice to penalise staff. The choice of topicsto be audited was restricted by increasing man-agement emphasis on cost efficiency and byprofessionals avoiding audit of the morecomplex aspects of care. Other constraints toprogress were attributed to changes inorganisation of the health service. The splittingof therapists across directorates led to a loss ofthe professional confidence necessary toundertake audit, lack of formal connectionsbetween heads of service and general managersmeant that audit findings were not alwaysacted on, and the climate of competitionbetween trusts for contracts was manifest in agrowing reluctance to share audit findings.Our sample was purposive rather than

representative. The relevance of the results tothose responsible for audit lies, therefore, notin the extent to which they can be generalised,but rather in the range of constraints identified,

212


as these may be impeding the progress of auditin their own sites.

Limitations of the studyThe constraints identified could all bedescribed as external barriers that is, theywere respondents' perceptions of whatprevented them from undertaking audit. It wasa limitation of the study that internal cognitivebarriers were not explored namely,individual doubts about the value of audit.Studies of doctors' views of audit suggest thatsuch doubts do exist," and further research isneeded to explore whether a comparable situa-tion exists among members of the therapy pro-

fessions. Although the present study showedthat therapists thought that a lack of resources

acted as a constraint to the progress of audit,precise measurement of additional resources

was not obtained, and further research isrequired.

Implications of the resultsIt has been suggested that time for audit shouldnot be a problem if it is integrated into theeveryday working practices of health profes-sionals.8 Although this may well be the case

when staff are completing patient records in a

format which makes them suitable for audit, itseems less feasible when, for example, they are

designing instruments for collection of data or

writing reports on findings. If clinical staff are

to be allowed sufficient time to undertake auditand the level of client care is to remain thesame, then additional resources are required.As suggested by respondents, these resources

could take the form of extra staff, bank staff,and audit assistants; certainly audit assistantshave been helpful to the progress of medicalaudit." 4A much more comprehensive approach than

that adopted to date is needed to clarify therole of information technology in audit in thetherapy professions, and could build on a pro-gramme developed by Matchet and Rose formedical audit.'5 This included a choice of soft-ware more appropriate to user's requirements,staff training programmes tailored toindividual needs, and the appointment of tech-nical support staff. It has to be recognised,however, that difficulties in choosing andinstalling appropriate information technologysystems have long been a feature of medical'5and clinical audit,'6 and it is unlikely to be dif-ferent for audit in the therapy professions.At the outset of an audit project an

assessment should be made by someone withexperience of what kind of expertise will berequired throughout its course and then, ifnecessary, links made with appropriate person-nel. It is surely unrealistic to expect peoplewithout relevant experience to design valid andreliable instruments; initially they need access

to those with appropriate experience. Certainlythe benefits of the help provided by auditfacilitators was acknowledged by many of theparticipants in this study, and similar findingshave emerged from studies of audit in otherprofessions for example, the study byCarroll et al of audit facilitators in general

practice.'7 Recognition is also required of theneeds of these audit facilitators themselves, ifthey are to develop and sustain audit initiatives.Concerns about motives for audit and

confidentiality of findings are unlikely to beallayed, because as Packwood et al have argued,audit has moved increasingly from a privateprofessional domain into the public interfacebetween purchasers and providers.'4 Moreover,organisational structures are unlikely to changein a way which mitigates the constraints whichrespondents thought that they imposed ontheir audit activities. The way forward is forthese issues to be acknowledged in a forumwhich represents all those involved, and inwhich resolution can be sought by means ofwhat Thomson and Barton refer to as matureand informed communication.8Such a forum should be a unit based audit

committee in which professionals, managers,and purchasers are represented with equalvoice. Such committees existed in some of thefieldwork sites and the Department of Healthhas recommended that they be established inall trusts.'8 The goal of the committee must beto develop clarity and unity of purpose aboutaudit, as it is hard for staff to remain enthusias-tic and committed when this is not the case. Ifthis goal is not achieved, then the researchreported here suggests that professionals mayrespond, albeit reluctantly, by steering theselection of audit towards what they regardas safe topics and away from those morecomplex aspects of care with a potential forunpredictable and possibly unwelcome results.The committee should also develop an over-

all plan for audit activities and have the author-ity to mandate appropriate resources for audit.As Thomson and Barton argue, this wouldhelp to ensure that all the elements necessaryfor the successful completion of audit pro-jects are in place.8 Considering constraintsattributed to inadequate resources and lack ofexpertise will of course have financial implica-tions, albeit that these may be offset in duecourse by the increased likelihood of thesuccessful completion of audit projects.

ConclusionAs Buxton has observed, audit is now seen bymany as an inherently good thing; an activitythat by definition deserves support andencouragement.'9 The research reported here,however, indicated that audit in the professionsof occupational therapy, physiotherapy, speechand language therapy, and clinical psychologywas taking place in an environment in whichvarious constraints had the potential tojeopardise progress. The extent to which com-mitment to audit is maintained in these andother professions may well depend on whetherthese constraints are resolved in the future.

This paper draws on a project funded by the Department ofHealth, and undertaken by a research team of which I was amember. The team directors were Maurice Kogan of the Cen-tre for Evaluation of Public Policy and Practice, Brunel Univer-sity and Sally Redfern of the Nursing Research Unit, King'sCollege, London; other members were Anemone Kober, IanNorman, and Tim Packwood. The research was a collaborativeeffort by the team; this paper reflects those aspects of the analy-

213

Robinson

sis and writing for which I took particular responsibility. Theviews expressed in this publication are my own and notnecessarily those of the Department of Health. Thanks are dueto Jenny Lynden and Geraldine Reast for administrative andsecretarial support.

1 Ellis R, Whittington D. Quality assurance in hicalthi caie: a

handbook. London: Edward Arnold, 1994.2 National Nursing and Therapy Audit Networks. Leeds:

Institute of Nursing, Leeds University, Newsletters,1992-3.

3 Kober A, Redfern S. The four professions. In: Kogan M,Redfern S, Kober A, Norman I, Robinson S, Packwood T.Clinical audit in four health professions. London: NursingResearch Unit, King's College, 1994.

4 Department of Health. Clinical audit in their nursing andtherapy professions. London: DoH, 1994.

5 Department of Health. Clinical audit: meeting and inipioving(Istandards in healthcare, London: DoH, 1993

6 Firth-Cozens J, Storer D. Registrars' and senior registrars'perceptions of their audit activities. Quality in Health Carc1992;l:161-4

7 Kerrison S, Packwood T, Buxton M. Medical audit: takingstock. London: King's Fund Centre, 1993. (Medical auditseries No 6.)

8 Thomson R G, Barton A. Is audit running out of steam.Quality in Health Care 1994;3:225-9

9 Normand C, Ditch J, Dockwell J, Finlay A, Gaskell G,Henderson G, Whittington D. Clinical audit inl professions

allied to medicine and related therapy projessions. Belfast:Health and Health Care Research Unit, Queen's Univer-sity, 1991.

10 Ritchie J. Spencer L. Qualitative data analysis for appliedpolicy research. In: Bryman A, Burgess B. Analsiing quali-tative data. London: Routledge, 1994.

11 Kogan M, Redfern S, Kober A, Norman I, Robinson S,Packwood T. Clinical audit in four health professions.London: Nursing Research Unit, King's College, 1994.

12 Black N, Thompson E. Obstacles to medical audit: Britishdoctors speak. Soc Sci Med 1993;36:849-56.

13 Hearnshaw H, Baker K, Robertson N. Multidisciplinaryaudit in primary healthcare teams: facilitation by auditsupport staff. Quality in Health Care 1994;3:164-8.

14 Packwood T, Kerrison S, Buxton M. The audit process andmedical organisation. Quality in Health Care 1992;1: 192-6.

15 Matchet A, Rose P. Medical audit in the quality process: apractical implementation. International Journal of HealthCare Quality Assurance 1992;5: 15-7.

16 Firth-Cozens J. Audit in mental health so cices. Hove:Lawrence Erlbaum Associates, 1993:8.

17 Carroll L, Thirlwell M, Wilson A. Medical audit and therole of the facilitator. International Yournal of Health CaseQuality Assurance 1994;7:8- 10

18 Mawhinney B, Nichol D. PurchasingJr health: afiranieworkfor action. London: National Health Service ManagementExecutive, Department of Health, 1993.

19 Buxton, M. Achievements of audit in the NHS. Quality illHealth Care 1994;3(suppl):35-6.

214

audit in the therapy professions: some constraints

Documents