an intro to audit

7/27/2019 An Intro to Audit

1/54

Audit for Registrars

Dr. Ramesh Mehay

Course Organiser

Bradford VTS

NOTE : Key points = core points to note for anysytematic approach to audit


2/54

Definition

Clinical audit is the systematic and criticalanalysis of the quality of clinical care.

This includes the procedures used for

diagnosis and treatment, the associateduse of resources and the effect of care onthe outcome and quality of life for thepatient.

Clinical Governance = improvingstandards


3/54

Crombie et al. defined

Audit as the process of reviewing thedelivery of health care to identify

deficiencies so that they may be remedied. Marinker (1990)

the attempt to improve quality of medical

care by measuring the performance inrelation to desired standards and byimproving on this performance


4/54

Definition less formal

Taking note of what we do

Learning from it

Changing it if necessary With the aim of improving care


5/54

Why do It?

Development of professional education and self regulation

Improvement of quality of patient care

Increasing accountability

Improvement of motivation and teamwork

Aiding in the assessment of needs

As a stimulus to research Clinical audit aims to lead to an improvement in the quality of service

providing:-

improved care of patients

enhanced professionalism of staff

efficient use of resources aid to continuing education

aid to administration

accountability to those outside the profession


6/54

Fundamental Principles

All about improving patient care

Should be seen as part of day to day

practice

Developing a critical eye on what we are

doing

Trying to improve things all the time


7/54

The Audit Cycle

What Should Be Happening

What Is Happening?

What changes are needed


8/54

The Audit Cycle


9/54

What Audit Is Not

Not about:

Performance Appraisal of Staff

Disciplinary Actions

Needs Assessment Research (which is usually about establishingnew knowledge)

Computers and Statistics

Competition between doctors Never judge good and bad professionals basedon audit it is about improving care


10/54

Audit vs Research

Research Audit

Defines Good Practice Assess extent to which good practice

is being obeyed/improvements

May involve allocating pts to random

treatment groups

Never involves this

May involve placebo Rx Never involves placebo Rx

Disturbs the pt beyond that required

for normal clinical management

Never does this

May involvea completely new

treatment

Never involves a completely new

treatment

One off process Ongoing

Other notes

Both audit and Research are concerned with clinical practice effectiveness

Audit can contribute to research

issues that need further exploration


11/54

When to Use What

Method When to use it Why

Research Good practice is not

defined andcomparisons are

needed

To define good

practice

Data Collection or

structured

observation

Practice patterns

unknown

To catalogue

prevailing practice

without makingjudgements

Audit Good practice is

defined but we want

to know how much

we are sticking to it

To improve current

performance


12/54


13/54

Making Audit Easier Avoid the

Blocks BEFORE YOU START

Time big audits can eat up time in an already busy schedule, so :

Keep it simple and small

Look at one or two criteria

Engage the whole team otherwise it will be difficult! Is the team

ready? (Enthusiasm, wanting to improve) WHEN YOU START

Delegate & Share the workload involve others

Make life easier use computers to do the laborious stuff (patientsearches)

Use protocols / standards already laid by others (why re-invent thewheel?)

Be careful of data collection choose a topic which does not entailtoo much data collection to the extent it becomes exhaustive withsubsequent loss of enthusiasm


14/54

Some Ideas

You can do an Audit of

Structure ie facilities being provided

Eg waiting times, availability of staff, record keeping (all patientrecords should have a summary card), equipment

Process ie what was done to the patient eg referrals, prescribing,

investigations Aspirin post MI, BP measurements 5 yearly in those aged 20-65

Outcome ie result for the patient

Eg patient satisfaction, patients with high BP aged between 20-35should have a diastolic below 90mmHg within the first year oftreatment

high risk practices (significant event audits) eg pneumococcalvaccines in splenectomised patients, are significant events beingacted upon?

The outcome is the ideal indicator for care but the most difficult tomeasure.


15/54

Choosing a Topic

Condition has an important impact on health or of great localconcern KEY POINT ie serious consequences otherwise

Condition affects a large number of people

Good reasons for believing current performance can be improved orimprovements are needed KEY POINT

Convincing evidence about appropriate care is available Data collection choose a topic which does not entail too much datacollection to the extent it becomes exhaustive with subsequent lossof enthusiasm (? A pilot??)

CHOOSE SOMETHING THAT REALLY INTERESTS YOU

NO POINT AUDITING SOMETHING YOU THINK THE PRACTICE

IS DOING REALLY WELL Then discuss with others are they interested too?


16/54

Choosing a Topic

Remember, topic should be important :

Chronic Disease Management eg referrals

or use of lab services (INRs in warfarin)

Preventative Care eg childhood imms,

Cervical Cytology

Prescribing eg aspirin post MI, PPIs (costissue)


17/54

Examples

Ways of spotting audit topics examples

Important clinical events admissions for asthma

Significant events patient died of MI norecord of smoking history or BP

Patients' complaints too long to get an

appointment Observation no system for ensuring bagdrugs up to date

Observations of staff patient on Warfarin not hadINR for 6 months

NICE subjects post-MI patients on aspirin


18/54

Criteria

= yardsticks

An audit criterion is a specific statement of whatshould be happening.

A statement which

A) defines a measurable item of health carewhich

B) can be used to assess quality

KEYPOINTCriteria should be explicit. You mustdemonstrate evidence for justifying them(literature search, Evidence Based!).


19/54

Criteria KEY POINTS

Ensure that the criterion is measurable

asthmatics should have had yearly PFs isdifficult to measure (how many years will you go back?);

asthmatics should have had a PF recorded in

the past year is more practical. Dont try to audit too many criteria at once one or two

will keep you busy enough.

Try filling in the gaps of the following phrase to set your

audit criterion: All patients with xxxxx should have had a xxxxx in the

last xxxxx.


20/54

Criteria

"All eligible women aged 25-65 should

have had a cervical smear in the last 5

years."

All asthmatics should have had a Peak

Flow recorded in the past year.

All drugs in our doctors bags should be

in-date.


21/54

Standards

An audit standard is a minimum level of

acceptable performance for that criterion.

Make sure the standard is directly related

to the criterion, also :-

Should include a suitable timeframe


22/54

Standards

Examples:

"At least 80% of eligible women aged 25-65 should havehad a cervical smear in the last 5 years."

At least 60% of asthmatics should have had a Peak

Flow recorded in the past year. 100% of drugs in our doctors bags should be in-date.

The standard should reflect the clinical and medico-legalimportance of the criterion.

in the example above, 80% of women should have had acervical smear,

But of those who've had an abnormalsmear, 100%should have had action taken.


23/54

Standards

How to set standards

Look at national guidelines

Literature (journals), textbooks

Local guidelines

Discussion with consultants/GPSIs

Discussion with trainer/partners

KEY POINT : Standards set should be realisticand attainable. Justifiable reasons for the

standard set should be made explicitly clear.


24/54

Standards

Some criteria are so important that they need 100%standard.

However, 100% standards are unusual patients orcircumstances usually conspire against perfection andthe standard needs to reflect that.

Your literature search should give you an idea of whatstandards others have managed to reach.

Your standard needs to follow on directly from your

criterion for example, Patients on thyroxine should have had TFTs done in thelast year; this should have happened in at least 90% ofpatients.


25/54

TYING IT ALL TOGETHER

Examples of Standards & Criteria

Criteria Standards

All children under 2 years should

be immunised against tetanus and

polio

90% of registered patients under

the age of 2 years should have

been immunised against polio andtetanus

All notes of those patients with an

allergy to penicillin should be

marked

95% of patients with an allergy to

penicillin should be clearly marked

All patients in the surgery should

wait no longer than 30 minutes

before a consultation

70% of patients in the surgery

should wait no longer than 30

minutes before a consultation


26/54

Preparation & Planning

Must show evidence of teamwork

otherwise you will fail


27/54

Data Collection (1)

You can collect information from:

computer registers

review of contents of medical records questionnaires patients, staff or GPs

data collection sheets


28/54

Data Collection

Be careful of data collection choose a topicwhich does not entail too much data collection tothe extent it becomes exhaustive withsubsequent loss of enthusiasm

? Sampling random or systemic

Only collect essential information

Use computers, ?data collection forms

Use other staff & delegatedont do all the workyourself

Set a deadline


29/54

Presenting the Results

Collect Results

Analyse Results

Summarise Results

Present Results to the team

Simple arithmetic calculations

Use percentages

Results of 2nd data collection presented inthe same way as the 1st


30/54

Discussion Data Collection 1


31/54

Comparing Results to Standards

Criterion Standard Observed Result

All patients should be

seen within 15

minutes of their

appointmenttimeMinimum

70% 45%

70%45%All diabetics

to have had HbA1C in

last 3

monthsMinimum

95% 90%

90%Drug allergies to

be marked as active

problem on

computerMinimum

100% 95%


32/54

Discussion Data Collection (1)

KEY POINT (Discussion of Data Collection

1) : You need to explain why you think the

practice didn't meet the standard that was

set.


33/54

Discussion why standards not

met Think: What reasons are there for practices not meeting audit

standards?

For example : reasons have included:

Practice reasons:

Results having been put down as free text on computer,rather than coded;

Opportunistic rather than formal recall system in use;

Doctor reasons:

Not all GPs were aware of the practice policy;

Not all partners agreed with the policy;

Patient reasons:

Patients refusing to have tests done;

Patients on holiday when tests due.


34/54

Implementing Changes

The most challenging stage

Audit can tell you whether changes are

needed, but it cant tell you what methods

to use


35/54

Implementing Change

The changes to be implemented should be

a team discussion and decision (?a

practice meeting)

What to do at the Pract ice Meeting :

Emphasise what has been achieved.

What are we proud of?

What are we not so proud of?

How can we correct any deficiencies?


36/54

Implementing Change

Changes mus t be pract ica l !

How are you actually going to make the changes?

Simply saying Weve got to do better wont result inchange

You need to think through in detail what needs to be done

whos going to do it

when

and how. If you get very low results, you may consider resetting

the standards to a more realistic level (but justify it)


37/54

Implementing Change

KEYPOINT

Just telling people to do things better won't result inchange. You need to write up in some detail how thechanges will take place.

FAIL Example:

"The GPs agreed to do a serum rhubarb on any patient that they seewho is on Viagra" - fail - this wouldn't be likely to pass, as there is nosystem to help them remember.

PASS Example :

(a) The GPs were given a prompt card that they could stick on theircomputer screen as a reminder to do a serum rhubarb on any

patient that they see who is on Viagra; (b) the secretary will search every three months for patients who are

overdue for their serum rhubarb, and flag it as an active problem onthe computer system" - pass - as it should result in change.


38/54

Closing the Loop

Ie repeat ing the cycle

Re-evaluate care to ensure that any

remedial action has been effective.

Audit is a continuous cycleif you didnt

meet the standard and youve planned

changes, youll need to repeat the audit to

make sure the changes have happened.


39/54

Conclusions from the Audit

Summary of main issues learned

KEYPOINTS:

Comment on any improvements that have resulted.

How well did your proposals for change work?

If you again didn't reach the standard that you set, whynot?

If you did, should you be aiming higher next time, or lookat something else e.g. whether abnormal serum

rhubarbs have actually been acted on? Where should the practice go from here


40/54

Useful Resources

MAAGs medical audit advisory groups

Clinical Governance Advisory Groups

National/Local Guidelines

RCGP database of simple tested audits forday-to-day use

Literature, Books

The WWW Consultants, GPSIs, Trainers, Partners


41/54

How To Fail

No justification for choice of audit No justification for criteria/standard settings

Not having explicit criteria/standards

Setting unreasonable standards

A general lack of evidence based literature or using material that is not peerreferenced

Not explicitly displaying teamwork in the method must give specificexamples

Numerical errors re: data collection

Presentation of data collection eg no graphs, no percentages (ie the readerhas to do the hardwork him/herself)

Not giving much thought to changes to be evaluated and not being specific

enough. Not delegating specific changes to specific people/persons. Poor conclusions and what the process has taught you

No inclusion for possible sources of bias

References not properly quoted


42/54

IF YOU DONT WANT TO FAIL

Go through the following online tutorial

http://www.mharris.eurobell.co.uk

Look at the Marking Schedule (yes, theyprovide you with an answer sheet!)

www.mharris.eurobell.co.uk/marking.htm

You must pass on all 8 criteria.
http://www.mharris.eurobell.co.uk/http://www.mharris.eurobell.co.uk/marking.htmhttp://www.mharris.eurobell.co.uk/marking.htmhttp://www.mharris.eurobell.co.uk/


43/54

SHOs doing Audit for Summative

Assessment

If you are doing the audit while an SHO, youneed to choose a topic that looks at the GP-hospital interface. Referrals or discharge lettersare possible areas for audit. Again, you need to

demonstrate that you've found a problem thatneeds to be investigated.

I suggest that you discuss your proposedaudit with your GP Scheme Organiser before

you go ahead - your hospital colleagues may notknow what's needed for Summative

Assessment.


44/54

Checking GPR Understanding

DISCUSS THE FOLLOWING STATEMENTS An example of the Audit of process is audit of referrals to hospitals.

Audit usually consumes an extensive amount of resources (of time, moneyetc.).

Rare conditions should be audited.

The higher the standard the practitioner starts with, the stronger is the

resulting audit. Maintaining clearly written notes of at least 20% of patients who are

sensitive to penicillin is an acceptable standard in general

practice.

The higher the amount of data the practitioner collects, the easier is thedecision making process in audit.

The most challenging stage in Audit is implementing change. In data collection all in the target population must be included.

The agreed standards can be reset at realistic percentages after the firstround of data collection.


45/54

Clinical Audit Association Ltd

Clinical Audit Association Ltd

Cleethorpes Centre

Jackson Place

Wilton RoadHunberton

Lincolnshire DN36 4AS

Tel: 01472 210 682http://www.the-caa-ltd.demon.co.uk

Clinical Governance Research
http://www.the-caa-ltd.demon.co.uk/http://www.the-caa-ltd.demon.co.uk/http://www.the-caa-ltd.demon.co.uk/http://www.the-caa-ltd.demon.co.uk/http://www.the-caa-ltd.demon.co.uk/http://www.the-caa-ltd.demon.co.uk/


46/54

Clinical Governance Researchand Development Unit

Dept of General Practice and Primary HealthCareUniversity of LeicesterLeicester General Hospital

Gwendolen RdLeicester LE5 4PW

Tel: 0116 258 4873Fax: 0116 258 4982

email: [email protected] http://www.le.ac.uk/cgrdu

C h D t b f
mailto:[email protected]://www.le.ac.uk/cgrduhttp://www.le.ac.uk/cgrduhttp://www.le.ac.uk/cgrduhttp://www.le.ac.uk/cgrdumailto:[email protected]


47/54

Cochrane Database of

Systematic Review

020 7383 6185

c/o

British Medical Association

BMA House

Tavistock Square

London WC1H 9JP

NICE


48/54

NICE

11 Strand

London

WC2N 5HR

Tel: 020 7766 9191

Fax: 020 7766 9123

http://www.nice.org.uk

RCGP Eff ti Cli i l P ti
http://www.nice.org.uk/http://www.nice.org.uk/


49/54

RCGP Effective Clinical Practice

Unit

School of Health and Related Research

Regent Court

30 Regent Street

Sheffield S1 4DA

Tel: 0114 222 5454

Fax: 0114 272 4095

Email: [email protected] http://www.shef.ac.uk/~scharr/
mailto:[email protected]://www.shef.ac.uk/~scharr/http://www.shef.ac.uk/~scharr/http://www.shef.ac.uk/~scharr/http://www.shef.ac.uk/~scharr/mailto:[email protected]


50/54

RCGP NE Scotland Faculty

The Primary Care Resource Centre

Foresterhill Road

Aberdeen AB25 2ZP

Tel: 01224 558 042

Fax: 01224 558 047

Email: [email protected]

http://www.rcgp.org.uk/rcgp/faculties/

nescot/index.asp
http://www.rcgp.org.uk/rcgp/facultieshttp://www.rcgp.org.uk/rcgp/facultieshttp://www.rcgp.org.uk/rcgp/facultieshttp://www.rcgp.org.uk/rcgp/facultieshttp://www.rcgp.org.uk/rcgp/facultieshttp://www.rcgp.org.uk/rcgp/facultieshttp://www.rcgp.org.uk/rcgp/faculties


51/54

UK Cochrane Centre

Dr Iain Chalmers, Director

NHS Research and Development

Programme

Summertown PavilionMiddle Way

Oxford OX2 7LG

Tel: 01865 516300

C h C ll b ti i th


52/54

Cochrane Collaboration in the

field of primary care For information concerning work by the

Cochrane Collaboration in the field of primary care,contact:Dr Lorne BeckerProfessor and Chair

Dept of Family MedicineSUNY Upstate Medical University475 Irving AvenueSyracuse, NY 13210USATel: +1 315 464 7010Fax: +1 315 464 6982E-mail: [email protected]://www.update-software.com/ccweb/

default.html

NHS C t f R i d
mailto:[email protected]://www.update-software.com/ccweb/http:/www.update-software.com/ccweb/http://www.update-software.com/ccweb/http:/www.update-software.com/ccweb/http://www.update-software.com/ccweb/http:/www.update-software.com/ccweb/http://www.update-software.com/ccweb/http:/www.update-software.com/ccweb/http://www.update-software.com/ccweb/http:/www.update-software.com/ccweb/http://www.update-software.com/ccweb/http:/www.update-software.com/ccweb/http://www.update-software.com/ccweb/http:/www.update-software.com/ccweb/mailto:[email protected]:[email protected]


53/54

NHS Centre for Reviews and

Dissemination

University of York

York, YO1 5DD

Tel: 01904 433 634

Fax: 01904 433 661Email: [email protected]

http://www.york.ac.uk/inst/crd
http://www.york.ac.uk/inst/crdhttp://www.york.ac.uk/inst/crdhttp://www.york.ac.uk/inst/crdhttp://www.york.ac.uk/inst/crd


54/54

Sources

This power point has been derived from :

http://www.mharris.eurobell.co.uk

http://kims.org.kw/bulletin/Issues/issue2/A

udit.pdf
http://www.mharris.eurobell.co.uk/http://kims.org.kw/bulletin/Issues/issue2/Audit.pdfhttp://kims.org.kw/bulletin/Issues/issue2/Audit.pdfhttp://kims.org.kw/bulletin/Issues/issue2/Audit.pdfhttp://kims.org.kw/bulletin/Issues/issue2/Audit.pdfhttp://www.mharris.eurobell.co.uk/

an intro to audit

Documents