measuring how well the nhs looks after its own staff: methodology of the first national clinical...

Measuring how well the NHS looks after its own staff:methodology of the first national clinical audits ofoccupational health services in the NHSjep_1574 283..289

Siân Williams MBBS MRCP MD FFOM,1 Caroline Rogers BSc (Hons) MRes MSc,2 Penny Peel BSc MSc,3

Samuel B. Harvey MBBS DCH MRCGP MRCPsych,4 Max Henderson MBBS MSc MRCP MRCPsych,5

Ira Madan MBBS (Hons) FRCP FFOM,6 Julia Smedley BMedSci (Hons) MBBS MD FFOM FRCP7 andRobert Grant BSc DipStat MSc8

1Clinical Director, 3Programme Manager, Health and Work Development Unit, Royal College of Physicians, London, UK2Programme Manager, Nuffield Council on Bioethics, London, UK4Clinical Lecturer and Honorary SpR in Psychiatry, 5Senior Lecturer in Epidemiological & Occupational Psychiatry and Honorary Consultant LiaisonPsychiatrist, Institute of Psychiatry, King’s College London, London, UK6Consultant and Senior Lecturer in Occupational Medicine, Guy’s and St Thomas’ NHS Foundation Trust, London, UK7Consultant and Senior Lecturer in Occupational Medicine, Southampton University Hospitals NHS Trust, Southampton General Hospital,Southampton, UK8Medical Statistician, Royal College of Physicians, London, UK

Keywords

clinical audit, methodology, national audit,National Health Service, occupational health

Correspondence

Dr Siân WilliamsHealth and Work Development UnitRoyal College of Physicians11 St Andrews PlaceRegents ParkLondon NW1 4LEUKE-mail: [email protected]

Further details about the audits and theresults of this paper can be found in:Occupational Health Clinical EffectivenessUnit. Depression screening andmanagement of staff on long-term sicknessabsence – Occupational health practice inthe NHS in England: A national clinical audit.London: RCP, 2009.Occupational Health Clinical EffectivenessUnit. Back pain management – Occupationalhealth practice in the NHS in England: Anational clinical audit. London: RCP, 2009.

Accepted for publication: 5 August 2010

doi:10.1111/j.1365-2753.2010.01574.x

AbstractRationale, aims and objectives Little is known about the quality of occupational healthcare provided to National Health Service (NHS) staff. We designed the first nationalclinical audits of occupational health care in England. We chose to audit depression andback pain as health care workers have high levels of both conditions compared with otheremployment sectors. The aim of the audits was to drive up quality of care for staff withthese conditions. The object of this paper is to describe how we developed an auditmethodology and overcame challenges presented by the organization and delivery ofoccupational health care for NHS staff.Methods We designed two retrospective case note audits which ran simultaneously. Sitessubmitted up to 40 cases for each audit. We used duplicate case entry to test inter-raterreliability and performed selection bias checks. Participants received their site’s auditresults, benchmarked against the national average, within 4 months of the end of the dataentry period. We used electronic voting at a results dissemination conference to informimplementation activities.Results Occupational Health departments providing services to 278 (83%) trusts inEngland participated in one or both audits. Median kappa scores were above 0.7 for bothpilot and full audits, indicating ‘good’ levels of inter-rater reliability.

In total, 79% of participants at a dissemination conference said that they had changedtheir clinical practice either during data collection (52%) or following receipt of their auditresults (27%).Conclusions Clinical audit can be conducted successfully in the occupational healthsetting. We obtained meaningful data that have stimulated local and national qualityimprovement activities. Our methodology would be transferable to occupational healthsettings outside the NHS and in other countries.

IntroductionNational clinical audit methodology is well established in the UK.It has been successful at measuring, and driving up, standards of

care in several medical specialties [1,2]. In this paper we describethe design of the first ever national clinical audits of occupationalhealth (OH) care for National Health Service (NHS) staff inEngland. Some of the methodology builds on that used in the more

Journal of Evaluation in Clinical Practice ISSN 1365-2753

© 2010 Blackwell Publishing Ltd, Journal of Evaluation in Clinical Practice 18 (2012) 283–289 283

established clinical specialties, such as stroke [3], and some wedeveloped specifically to address the complexities of OH servicedesign and case management.

The NHS in England is the largest employer in Europe withapproximately 1.4 million staff looking after a population of 56million. English health care professionals report more work-related health problems than most other professional groups [4–7].The NHS has high levels of health-related work impairment (pre-senteeism), high rates of sickness absence and extensive numbersof staff claiming long-term incapacity benefits [6–9]. NHSemployees take on average 10.7 days of sick leave per annum,compared to 9.7 days in the rest of the UK public sector and 6.4days amongst UK private sector employees [6]. The most commonapparent causes are psychiatric disorders and musculoskeletalproblems [6,9]. Studies of NHS acute trusts have shown associa-tions between staff ill health and low patient satisfaction,high rates of hospital acquired infections and poorer overallperformance [6,10,11].

To support the health of its staff the NHS provides OH services.Each NHS trust is responsible for either employing its own ‘in-house’ OH team or commissioning services from an external pro-vider. OH service provision varies across the country in terms ofstaff numbers, type, qualifications, services offered and fundinglevel. Advising staff and their managers on health problems thatare caused or made worse by work, and the impact of healthproblems on capacity for work, is an important core OH serviceactivity.

To drive up quality of care we designed two national clinicalaudits of OH care in the NHS in England. We chose to audit backpain management, and the detection of depression in staff onlong-term sickness absence. These conditions were chosenbecause they are common, account for a high proportion of sick-ness absence, and both have national evidence-based guidancefrom which to develop audit criteria [12,13].

The main aims of our audit projects were to: develop a meth-odology for national clinical audit of OH practice; assess variationin practice amongst OH clinicians providing services to NHS staffin England; benchmark current practice, both locally and nation-ally, against guideline standards; and provide baseline data forfurther quality improvement activities.

We describe here how we developed a methodology and over-came potential problems through the process of audit design,execution, data analysis, reporting and follow-up work with auditparticipants and their trusts. We present results on participation,selection bias, inter-rater reliability, typographic errors, data com-pleteness, uptake of dissemination activities, and qualitative feed-back from participants.

Methods

The team

The audits were run by the Occupational Health Clinical Effec-tiveness Unit (OHCEU). This collaboration between the RoyalCollege of Physicians of London and the Faculty of OccupationalMedicine was commissioned and funded by NHS Plus [14,15],with the aim of raising standards of OH care in the NHS and morewidely. Senior clinicians led on the audit design and developmentof the data collection tool, and were supported by a multidisci-

plinary audit development group. The OHCEU steering group,consisting of the key stakeholders, agreed the final audit processand reports.

Recruitment of participants

The organization of OH services within the NHS in England iscomplex. There are fewer OH providers to the NHS than there areNHS trusts. Many trusts with an ‘in-house’ OH service deliver aservice to several other local trusts under contract, and a few trustsuse more than one OH provider. In addition, some commercial OHproviders service several trusts across very wide geographicalareas.

We issued a unique identifying number to each pairing of a trustand OH service. We refer to these pairings as ‘sites’ and each sitewas asked to submit a separate set of case notes to the audit.

All NHS trusts in England were invited to participate throughletters to their human resources directors, clinical audit depart-ments, chief executives and OH providers. The audits were alsoadvertised through professional journals, newsletters, mail shotsand a free national conference for potential participants. Through-out the data entry period further reminders were sent to OH depart-ments using email and posted letters. Non-participants werecontacted by the OHCEU audit team and OH ‘champions’ atregional level, aiming to address any barriers to participation.

Audit design and timeline

The two audits ran simultaneously. Both were retrospective casenote audits. For each one, participating sites were asked to identifythe case notes of 40 consecutive staff who attended their OHdepartment between January and May 2008 and fulfilled the casedefinition shown in Box 1. Data were extracted from the case notesand entered locally in May, June and July 2008. Local results weresent to participating sites electronically in December 2008 andnational results were published in January 2009. We held anational dissemination conference in April 2009 and regionalimplementation workshops in June and July August 2009 (seeFig. 1).

Development of audit questions and tools

For each audit, the questions reflected recommendations in rel-evant national evidence-based guidance [12,13] plus questionsproposed by the audit clinical leads in consultation with the auditdevelopment group.

A web-based data collection tool was created using the tech-niques developed by the Royal College of Physicians’ ClinicalEffectiveness and Evaluation Unit in audit topics such as strokeand evidence-based prescribing [16]. The web tool routed the datacollector through a series of questions for each set of case notesthat was audited, making available only the applicable answers.Responses were checked by the computer against pre-defined rulesto avoid errors. The web tool allowed free text comments to beappended to each response. No patient-identifiable data wererequested.

The pilot

We piloted both online audit tools in March 2008. Eight sitesvolunteered to participate. All were acute trusts but differed in size

Occupational health audit methodology S. Williams et al.

© 2010 Blackwell Publishing Ltd284

and geographical location. Data from the pilot were analysed forcompleteness, coherence, and reliability. Participants were askedabout any difficulties in using the questionnaire/web tool, retriev-ing relevant information or interpreting the questions. Resultsfrom these data were examined by the project team and refine-ments made to the audit questions and the supporting help notesfor participants.

Full data collection

Once a participating trust had registered its details and its OHprovider’s details, a unique username and password were issued toallow confidential electronic data entry. The web site was open fordata collection for 3 months and the OHCEU provided a helpdeskthroughout this period to answer queries from participants.Responses to common queries were circulated amongst all partici-pants to improve accuracy of sampling, data extraction and entry.OHCEU was able to interrogate the web tool for informationon individual sites’ activity, so that reminders could be tailoredappropriately.

Patient confidentiality

Data extraction for national clinical audit is usually supported bytrust audit department staff locally. However, as OH patients are

staff of these trusts, we recommended that only OH doctors ornurses extracted the data (but where possible did not enter datafrom their own case note entries).

After the close of data collection, data were transported from theweb tool and held securely at the OHCEU where they werecleaned and analysed by the project team.

Inter-rater reliability

We used inter-rater reliability checks to test the extent to whichdifferent auditors would collect the same data about a givenpatient. Misunderstanding of the questions, errors in using the webtool, typographic errors and ambiguity in the case notes can alllead to disagreements.

We assessed inter-rater reliability quantitatively for the pilotstudy and again for the main data collection period. For eachsample submitted to the audit, the first five cases were re-enteredby a second, independent data collector. We calculated Cohen’skappa scores for questions with binary data, and percentage agree-ment for numerical data. The McNemar–Bowker test was used toconsider whether there was a systematic difference between earlierand later entries of the same data that might reflect availability ofinformation to the auditors.

We also compared the cases submitted with duplicates to thosesubmitted without duplication in terms of demographics and

Box 1 Inclusion criteria for audit cases

Depression screening and management during long-term sickness absence

A National Health Service staff member’s first consultation, between 1 January 2008 and 22 August 2008, with an occupational health doctor ornurse following 4 weeks of sickness absence for any health-related reason.Back pain

A National Health Service staff member’s first consultation for a new episode of back pain (separated from any previous episode by at least 4weeks) with an occupational health doctor or nurse between 1 January 2008 and 25 July 2008. We did not include consultations withphysiotherapists in this audit as these would need to be audited against other guidelines.Participants were asked to submit a sample of 40 consecutive eligible consultations for each audit.

Audit Delivery Timeline

Report writing ends

Local results sent

nuJguA JulMay Dec Jan

Site recruitment and case selection begins

Data collection begins

Data collection finishes

Statistical analysis and report writing begins

National results published

Regional implementation workshop programme begins

National conference for audit result dissemination

Jan Apr90028002

Regional workshop programme ends

Data collection and analysis Dissemination and implementation

Figure 1 Audit timeline.

S. Williams et al. Occupational health audit methodology

© 2010 Blackwell Publishing Ltd 285

length of time off work (which we regard as a proxy for thecomplexity of the patient’s OH notes) to make sure there was noselection bias.

Selection bias checks

Within each audit we checked for selection bias in those sitessubmitting fewer cases; this can occur inadvertently if simpler ormore accessible sets of case notes are entered first but the auditorthen omits to complete the remainder. We compared data fromsites that submitted fewer than the median number of cases withdata from those submitting more in terms of age, gender, occupa-tion, whether the person had been off work (in the back pain auditonly) and how long they had been off work.

When assessing selection bias, we did not conduct statisticaltests of significance because we are not able to distinguish sitesthat chose to enter few cases from those that only had very few andcould not have entered more; selection bias is only a real concernfor the former group. Also, there may be unknown characteristicsof the site and the OH service which confound the associationbetween having few data entered and demographics or time offwork, and we are not in a position to adjust for these. A thirdconsideration is that the great majority of OH services are repre-sented in the data and so inference to a population of sites is notmeaningful.

Reporting results to providers andcommissioners of services

Site level results were confidential and were provided only to theOH service and trust to which they related for comparison with thenational average results. Trusts that had entered a small number ofcases were warned to consider their own results from the auditwith great caution, but were reassured that they still contributedusefully to the national statistics.

Because only the local OH team will fully understand thecontext of the service organization, inference was not made in thereport in the form of confidence intervals or hypothesis tests.

National conference for audit participants

We held a national audit dissemination conference 4 months afterparticipants received their audit reports. We used anonymous elec-tronic voting to explore participants’ attitudes, behaviour andneeds around depression screening and back pain management.

Regional quality improvement workshops

We held nine regional workshops across England 6 months afterattendees had received their audit results. Working in small groups,participants used an adapted template developed by the NationalInstitute for Health and Clinical Excellence [17] to explore barriersexperienced locally in implementing guideline recommendationsand to design action plans. We identified common themes from thecompleted templates and sent these to audit participants to furtherinform their quality improvement activities.

Results

Pilot of the questionnaire

Seven of the eight volunteer sites piloted at least one of the audittools. For the back pain audit, six sites recorded 27 sets of patientnotes with second auditors duplicating 12 of these. For depressionscreening, seven sites audited 48 sets of notes and duplicated 18 ofthese. Kappa scores summarizing the inter-rater reliability hadmedian 0.78 in each audit but covered a wide range with 8/31 backpain questions scoring below 0.6 and 2/31 below 0.4. In thedepression screening data, 7/26 questions scored below 0.6 and2/26 below 0.4. Minor changes were only made to a few questions,based on kappa results and feedback from pilot sites.

Data cleaning

The web-tool design produced consistent and valid data thatrequired very little cleaning. In the free-text field of the data tool1616 comments were entered for the back pain data and 1051 forthe depression screening. Most of the comments did not influencedata analysis. In a few cases the participant explained that the webtool had required completion of a field despite the answer beingunknown. In these cases the response was deleted.

Response rate

The back pain audit collected 2959 cases from 261 sites. Thesesites provided services to 253/389 (65%) NHS trusts. Many siteswere unable to identify 40 eligible cases, with only 117 (46%) sitessubmitting 10 or more cases.

A similar pattern was seen in the depression screening auditalthough total numbers were higher; 6286 cases were audited by277 sites. These provided services to 267/389 (69%) trusts. A totalof 219 sites (79%) submitted 10 or more cases.

The OH departments providing services to 83% (278/389) ofNHS trusts in England participated in one or both of the audits.

Selection bias checks

When we compared data from sites submitting fewer than themedian number of cases with those submitting more within eachaudit, we found no persistent differences of a size that mightsuggest bias in case selection in terms of age, gender, occupationor time off work (Table 1).

For the depression screening audit, participants were asked toinclude staff seen after 4 weeks of sickness absence for any healthproblem. However, 9% of sites (26/277) entered only patients witha psychological diagnosis, accounting for 3% (186/6286) of casesentered nationally. In total, 4% of sites (11/277) entered onlypatients diagnosed with depression.

For the back pain audit, participants were asked to include allnew cases of back pain seen in OH, whether at work or off sick.However, 31% of sites (81/261) entered only patients who hadbeen off work with back pain. These accounted for 13% (372/2959) of cases. Data from these sites were compared to the othersin terms of time off work (as a proxy for severity), and no system-



atic difference was seen. The possibility of cross-confoundingbetween patients and site-level characteristics makes it unwise todo significance testing.

Inter-rater reliability

In addition to the data collected for the main audit analyses, 697cases from the back pain audit and 864 cases from depressionscreening had been independently re-audited and entered into the

web tool by the second auditor. These were checked foraccidental triplicates and the numbers reduced to 685 and853.

Cases selected for duplicate entry were not noticeably differentto those not selected in terms of demographics or the complexityof their notes, as measured by the length of time the person hadbeen off work (Table 2). Therefore there is no evidence here ofselection bias, and the duplicates should provide valid estimates ofinter-rater reliability.

Table 1 Checks for selection bias in sites selecting fewer cases

Back pain Depression screening

From sites submitting fewerthan median cases (1–8)

From sites submitting morethan median cases (>8)

From sites submitting fewerthan median cases (1–20)

From sites submitting morethan median cases (>20)

Age (years)<40 188 (35%) 932 (38%) 408 (26%) 1379 (29%)40–55 261 (49%) 1217 (50%) 879 (55%) 2504 (53%)>55 89 (17%) 361 (12%) 299 (19%) 817 (17%)

GenderMale 105 (20%) 423 (17%) 266 (17%) 746 (16%)Female 433 (80%) 1998 (83%) 1320 (83%) 3954 (84%)

OccupationAllied 65 (12%) 324 (13%) 212 (13%) 536 (11%)Ancillary 110 (20%) 406 (17%) 284 (18%) 859 (18%)Clerical 69 (13%) 323 (13%) 288 (18%) 731 (16%)Doctor 5 (0.9%) 48 (2%) 28 (2%) 87 (2%)Nurse 265 (49%) 1140 (47%) 675 (43%) 2157 (46%)Other 24 (4%) 173 (7%) 95 (6%) 313 (7%)

Off work? 424 (79%) 1615 (67%) Not applicable Not applicableWeeks off work

Median 5 weeks 4 weeks 9 weeks 9 weeksIQR 3–10 weeks 2–8 weeks 6–16 weeks 6–14 weeks

IQR, inter-quartile range.

Table 2 Checks for selection bias in inter-raterreliability duplicates

Back pain Depression screening

Not duplicated Duplicated Not duplicated Duplicated

Age (years)<40 868 (38%) 252 (37%) 1563 (29%) 224 (26%)40–55 1130 (50%) 348 (51%) 2919 (54%) 464 (54%)>55 274 (12%) 87 (13%) 951 (18%) 165 (19%)

GenderMale 400 (18%) 128 (19%) 873 (16%) 139 (16%)Female 1872 (82%) 559 (81%) 4560 (84%) 714 (84%)

OccupationAllied 318 (14%) 71 (10%) 643 (12%) 105 (12%)Ancillary 404 (18%) 112 (16%) 1016 (19%) 127 (15%)Clerical 304 (13%) 88 (13%) 873 (16%) 146 (17%)Doctor 34 (2%) 19 (3%) 103 (2%) 12 (1%)Nurse 1053 (46%) 352 (51%) 2434 (45%) 398 (47%)Other 153 (7%) 44 (6%) 347 (6%) 61 (7%)

Off work? 1545 (68%) 494 (72%) Not applicable Not applicableWeeks off work

Median 4 weeks 4 weeks 9 weeks 9 weeksIQR 2–8 weeks 2–8 weeks 6–14 weeks 6–15 weeks

IQR, inter-quartile range.



A total of 56% of the back pain data duplicates were enteredwithin 7 days of the initial case being entered, and 85% within 30days. For depression screening, the figures were very similar at55% and 84%, respectively. The median kappa score was 0.72 inback pain and 0.80 in depression screening. Only 3/30 categoricalquestions in back pain and 3/37 in depression screening gave akappa score below 0.6. None in either audit was below 0.4.

Matched responses from the initial and duplicate auditors werecompared by the McNemar test. Over both audits, 4/67 questionsshowed a significant difference; there was no coherent direction tothese differences. These are likely to have arisen by chance, as wewould expect 3/67 questions to be falsely significant with thestandard type I error rate of 5%.

Differences on numerical questions were consistent between thetwo audits: both had 3% of auditors disagreeing on age by morethan a year, 5% (back pain) and 6% (depression screening) dis-agreeing on appointment date by more than a week, and 3% and5%, respectively, disagreeing on the period of sickness absence bymore than 4 weeks. There is no obvious pattern to these disagree-ments that might explain them in terms of the later data entry of theduplicate finding a later appointment in the notes. Nor are longerwaits between initial and duplicate data entry linked with greaternumbers of errors. The only apparent pattern is for typographicerrors involving month, where differences by 30, 31 or 61 daysappear disproportionately.

Dissemination of results

The audit dissemination conference was attended by 183 OH stafffrom 60% of participating trusts. During electronic voting, 52% ofdelegates said that they had changed their clinical practice duringdata collection and a further 27% said they had done so followingreceipt of their audit results. Delegates who were actively involvedin data collection were more likely to have changed their practice(85%) than those who were not actively involved (63%).

The regional workshops were attended by 184 delegates, 120(65%) of whom had not attended our national conference. Alto-gether we reached 303 individuals through our conference andworkshops.

DiscussionWe have successfully completed the first two national clinicalaudits of OH care in England, and as far as we are aware, world-wide. We achieved an encouragingly high participation rate for thefirst round of a new clinical audit. Our participation rate of 72%compares favourably with rates from other first rounds, forexample 80% and 75% for national audits of stroke [18] andinflammatory bowel disease, respectively [19].

We faced several challenges when attempting to recruit partici-pants. Firstly, OH practitioners are not familiar with national clini-cal audit. Secondly, it was particularly important to preserveconfidentiality of the records of patients who were staff of theparticipating trust. In addition, it was challenging to capture ataudit the complexity of OH case management which involvescommunication with the employer as well as clinical interactionwith the patient and their health care providers. We were alsoaware that, in a competitive market place for OH services, pro-

spective audit participants in small OH units might be concernedthat scrutiny of their practice would threaten contracts with theircommissioning trust.

A major challenge was influencing many potential participantswho cited very high workload and staff shortages as the mainbarrier to participation.

We believe that we successfully overcame these challenges byusing a wide variety of communication modes to engage ouraudience. These included the launch conference where we dem-onstrated the audit tool and frequent contact with participantsthroughout the audit process.

The careful design and piloting of our audit tools and help notesproduced very good inter-rater reliability. These scores demon-strate the importance of using a rigorous pilot methodology. Thefinal kappa scores, and the clean data produced, suggest that thequestions in the audit tools were clear and extraction of data fromthe case notes was straightforward. Our kappa scores of 0.72 and0.80 were comparable with those achieved in the first round of thenational stroke audit which achieved a median kappa score of 0.70[20].

For both audits we found a small proportion of sites whoseresults suggested that the inclusion criteria for cases had beenmisinterpreted. In the depression screening audit, some sitesentered only cases off sick with a psychological diagnosisalthough physical diagnoses were allowed. This finding could bedue to misinterpretation of the instructions for case selection, ordue to chance particularly if few cases were entered. It is notpossible to assess this quantitatively as one would not expect aconstant depression diagnosis prevalence at all sites.

In the back pain audit some sites entered only cases that hadbeen off sick, when sickness absence was not a prerequisite forinclusion. Again this may be a misunderstanding of inclusioncriteria, particularly given that the audit was running simulta-neously with the depression screening audit where sicknessabsence was a pre-requisite for case inclusion. An alternativeexplanation is that some OH services may only see those staffwhere back pain has resulted in sickness absence, with those ableto work referred directly to the physiotherapist, whose consulta-tions were not included in this audit.

We concluded that these possible misunderstandings of theaudit criteria were infrequent, and so were unlikely to bias thedata. We will use these findings to inform improvements inthe clarity of our instructions on case inclusion criteria in futureaudit rounds.

Once participants had received their audit results we organizedboth national and regional events to maintain the momentum forquality improvement. While there was some overlap of attendees,the two approaches extended our audience considerably with over300 delegates from 155 English NHS trusts. These numberssuggest that the audit stimulated a high level of interest in qualityimprovement. This was supported by the results of the conferencevoting where 79% of delegates said that they had changed theirpractice as a result of participating in the audits. The finding thatmost of these delegates had changed practice even before the auditresults were available suggests that audit participation may be apowerful tool for changing behaviour amongst OH clinicians.Although our novel approach of using anonymous, electronicvoting has not been validated, we believe it is a useful tool to beginexploring beliefs and behaviours.



We have demonstrated that clinical audit can be conductedsuccessfully in the OH specialty. We obtained reliable data, withface validity, that is already stimulating local and national qualityimprovement activities. Future audits in the area will have baselinedata with which to compare and members of the OH professioncan build on the knowledge and skills they have acquired duringthis first round of audit.

Our audit methodology would be transferable to other audittopics, and to OH settings in other industries and other countries.Finally, the dissemination and implementation work of thenational conference and regional workshops demonstrates thatinteractive and outreach work is likely to have an important impacton continuing quality improvement.

AcknowledgementsThe audits were commissioned by NHS Plus. OHCEU is a part-nership between the Royal College of Physicians and the Facultyof Occupational Medicine. SBH and MH are supported by NIHRBiomedical Research Centre for Mental Health at the SouthLondon and Maudsley NHS Foundation Trust and Institute ofPsychiatry, King’s College London. IM was supported by NHSPlus and Guy’s and St Thomas’ NHS Foundation Trust.

References1. Intercollegiate Stroke Working Party (2009) National Sentinel Stroke

Audit Phase II (Clinical Audit) 2008: Report for England, Wales andNorthern Ireland. London: Royal College of Physicians of London.

2. National Audit of Myocardial Infarction Project (MINAP) SteeringGroup (2007) Management of Acute Coronary Syndromes in Englandand Wales: A Survey of Facilities in 2006. London: Royal College ofPhysicians of London.

3. Rudd, A., Lowe, D., Irwin, P., Rutledge, Z. & Pearson, M. (2001)National stroke audit: a tool for change? Qualitative Health Care, 10,141–151.

4. Jones, J. R., Hodgson, J. T. & Osman, J. (1997) Self-ReportedWorking Conditions in 1995: Results from a Household Survey.London: Health and Safety Executive.

5. Williams, S., Michie, S. & Pattani, S. (1998) Improving the Health ofthe NHS Workforce. London: Nuffield Trust.

6. Boorman, S. (2009) Health and Well-Being Review (Interim Report).London: Department of Health.

7. Harvey, S. B., Laird, B., Henderson, M. & Hotopf, M. (2009) TheMental Health of Health Care Professionals. London: Department ofHealth.

8. Ministerial Task Force for Health Safety and Productivity and TheCabinet Office (2004) Managing Sickness Absence in the PublicSector. London: Health and Safety Executive.

9. Ministerial Task Force for Health Safety and Productivity, T.C.Office(2005) One Year on Report. London: Health and Safety Executive.

10. Healthcare Commission (2005) Acute Hospital Portfolio Review.Ward Staffing. London: Healthcare Commission.

11. Taylor, C., Graham, J., Potts, H. W. W., Candy, J., Richards, M. A. &Ramirez, A. J. (2007) Impact of hospital consultants’ poor mentalhealth on patient care. British Journal of Psychiatry, 190, 268–269.

12. National Institute for Health and Clinical Excellence (2004) Depres-sion: Management of Depression in Primary and SecondaryCare (CG23). London: National Institute for Health and ClinicalExcellence.

13. Waddell, G. & Burton, K. (2000) Occupational Health Guidelinesfor the Management of Low Back Pain 2000: Evidence Reviewand Recommendations. London: Faculty of OccupationalMedicine.

14. Occupational Health Clinical Effectiveness Unit (2009) DepressionScreening and Management of Staff on Long-Term Sickness Absence– Occupational Health Practice in the NHS in England: A NationalClinical Audit. London: Royal College of Physicians.

15. Occupational Health Clinical Effectiveness Unit (2009) BackPain Management – Occupational Health Practice in the NHS inEngland: A National Clinical Audit. London: Royal College ofPhysicians.

16. Grant, R., Batty, G., Aggarwal, R., Lowe, D., Potter, J., Pearson, M.,Oborne, A. & Jackson, S. H. D. (2002) National sentinel clinical auditof evidence-based prescribing for older people: methodology anddevelopment. Journal of Evaluation in Clinical Practice, 8 (2), 189–198.

17. National Institute for Health and Clinical Excellence (2007) How toChange Practice: Understand, Identify and Overcome Barriers toChange. London: National Institute for Health and Clinical Excellence.

18. Irwin, P., Rutledge, Z. & Rudd, A. (2001) The feasibility of a nationalaudit of stroke. Journal of Clinical Governance, 6 (1), 27–33.

19. UK IBD Audit Steering Group (2007) UK IBD Audit 2006: NationalResults for the Organisation and Process of IBD Care in the UK.London: Royal College of Physicians.

20. Gompertz, P., Irwin, P., Morris, R., Lowe, D., Rutledge, Z., Rudd, A.& Pearson, G. (2001) Reliability and validity of the IntercollegiateStroke Audit Package. Journal of Evaluation in Clinical Practice, 7,1–11.



measuring how well the nhs looks after its own staff: methodology of the first national clinical...

Documents