“beyond data collection” or “what should we do now we’ve got the results?”

“Beyond Data Collection”or

“What Should We Do Now We’ve Got The Results?”

Dr John DerryMedical Director

Oxfordshire MAAG

PRIMIS Third National Conference Tuesday 1 April 2003Birmingham

Beyond data collectionor

“What should we do now we’ve got the results?”

Dr John DerryMedical Director

Oxfordshire MAAG


CHD Audit Resultsage & sex standardised prevalence of CVD

0

1

2

3

4

5

6

7

8

9

10

Practices

Ra

te p

er

10

0

Oxon average 4.72

PCT average 4.91


CHD Audit Resultsblood pressure recording & control

0

10

20

30

40

50

60

70

80

90

100

71

41

7

71

41

8

71

41

1

71

43

4

71

41

2

71

41

4

71

42

3

71

42

8

71

40

6

71

42

9

71

40

5

71

41

3

71

42

7

71

42

4

71

40

8

71

42

0

71

40

9

71

40

7

71

41

0

71

41

5

71

41

6

71

43

0

71

42

2

Practices

Pe

rce

nta

ge

s

BP not in target values <150sys <90 dia

BP in target values <150 sys <90 dia


But what should we do?

• See variation• What is significant?• Is it OK to be near

average?• When should we act?• Where should we start?• How should we act?


Use of “SPC”

• “Statistical Process Control”• Methods developed by Shewhart

and Deming (1930s – 1990s)• Cornerstone of quality improvement• Two different kinds of variation can

affect any process• Distinguish by statistical methods


“Statistical Process Control”

• “First and foremost, a way of thinking with some tools attached”

• “About the continual improvement of processes and outcomes”

• “About getting the most from your processes”

• Quotes from Don Wheeler in “Understanding Variation” SPC Press, 2000


Audience Participation!

• How long does it normally take you to get to work?

• Why does it vary?

• So you already know about the two kinds of variation!


Understanding variation

“Routine”• “common causes”• many factors, some

“unknowable”• “noise in the system”• affects process most

of the time• part of the process• variation is

predictable

“Exceptional”• “special causes”

• “assignable” causes

• usually few, not many

• can usually be identified

• not part of the process• intermittently apparent• unpredictable variation


What to do about variation

“Routine”• don’t react to

individual results• look at the average

and process limits• improve the whole

process if these not acceptable

• or continuously improve quality!

“Exceptional”• investigate each

point outside the limits

• look for the special cause and do something about it

• almost always something to find

• opportunities to learn


Two kinds of mistake

Mistake 1• Act as if there is a

special cause when there is only routine variation– Might make things

worse– Wasted effort

anyway

Mistake 2• Fail to spot a

special cause – assume there is just routine variation present– Missed opportunity

• reduce variation• improve quality• learn something


• Graphical method developed by Shewhart to help distinguish two kinds of variation– routine and exceptional

– predictable and unpredictable

– common and special cause

• “Process behaviour charts”(Don Wheeler)

Control charts


Anatomy of a control chart

Control Chart of Clinical Audit Data

0

5

10

15

20

25

0.00 20.00 40.00 60.00 80.00 100.00

SqRt number without criterion

Sq

Rt

nu

mb

er w

ith

cri

teri

on

Plus 3 SDs

Minus 3 SDs

Average

Routine variation

Exceptional variation

Exceptional variation


How to interpret the chart

Control Chart of Clinical Audit Data

0

5

10

15

20

25

0.00 20.00 40.00 60.00 80.00 100.00

SqRt number without criterion

Sq

Rt

nu

mb

er w

ith

cri

teri

on

Practices here cannot be distinguished from average

Practices here are

significantly different

from average

Increasing

difference

Practices here are

significantly different

from average

Increasing

difference


“Double square-root chart”

• Described recently by Mohammed et al (Lancet 2001; 357:463–467)

• Originally developed by Fisher, Tukey & Mosteller in 1940s

• Enable analysis of variation in “cross-sectional” data

• Based on binomial probability distribution for calculating SD


434

429

424420

417

416413

405406

407

409

0.00

5.00

10.00

15.00

20.00

25.00

0.00 20.00 40.00 60.00 80.00 100.00

SqRt number without CVD

Sq

Rt

nu

mb

er w

ith

CV

D

Standardised CVD Prevalence

Average = 4.6%

+/- 3SD Range = 3.6-5.7%


CHD Audit Resultsage & sex standardised prevalence of CVD

0

1

2

3

4

5

6

7

8

9

10

7142

9

7141

7

7140

6

7142

0

7143

0

7141

8

7142

8

7141

2

7141

0

7141

5

7142

2

7141

4

7142

3

7142

7

7140

8

7141

1

7140

9

7141

3

7140

7

7142

4

7141

6

7140

5

7143

4

Practices

Rat

e p

er 1

00

Oxon average 4.72

PCT average 4.91

293

57200115257

206146

43311

4

54 78 145 80


BP recorded

434

430

422417

416

411

412

0.00

5.00

10.00

15.00

20.00

0.00 2.00 4.00 6.00 8.00 10.00 12.00

SqRt number without BP record

Sq

Rt

nu

mb

er w

ith

BP

rec

ord

Average = 76%

+/- 3SD Range = 61-84%


CHD Audit Resultsblood pressure recording & control

0

10

20

30

40

50

60

70

80

90

1007

141

7

71

418

71

411

71

434

71

412

71

414

71

423

71

428

71

406

71

429

71

405

71

413

71

427

71

424

71

408

71

420

71

409

71

407

71

410

71

415

71

416

71

430

71

422

Practices

Per

cen

tag

es

BP not in target values <150sys <90 dia

BP in target values <150 sys <90 dia

1480

20085173257

54 28 43293

173235


Audit ResultsCardiovascular Disease Prevalence

511

512

516

519

521

6

10

14

18

22

40 50 60 70 80 90

SqRt without CVD

Sq

Rt

wit

h C

VD


Audit ResultsCVD Patients with cholesterol record

520

519

511508

5074

8

12

16

4 8 12 16

SqRt No Chol record

Sq

Rt

Ch

ol re

co

rde

d


Audit ResultsCholesterol levels in CVD Patients

521508

505

3

6

9

12

3 6 9 12

SqRt Chol above 5

Sq

Rt

Ch

ol b

elo

w 5


Audit ResultsStatin Rx for CVD Patients

521

519

512508

4

8

12

16

4 8 12 16

SqRt not Rx Statin

Sq

Rt

Rx S

tati

n


Control charts for clinical audit

• To answer the question– “What do we do now

we’ve got the results?”

• To identify where to target efforts

• To know when to act• To know what kind of

action to take

“beyond data collection” or “what should we do now we’ve got the results?”

Documents