tom marshall department of public health & epidemiology, university of birmingham...
TRANSCRIPT
![Page 1: Tom Marshall Department of Public Health & Epidemiology, University of Birmingham T.P.Marshall@bham.ac.uk Understanding Variation Using Registry data to](https://reader030.vdocuments.us/reader030/viewer/2022032414/56649ee15503460f94bf1f0e/html5/thumbnails/1.jpg)
Tom MarshallDepartment of Public Health & Epidemiology,
University of [email protected]
Understanding VariationUsing Registry data to drive improvement - what makes
clinicians take note of statistics
![Page 2: Tom Marshall Department of Public Health & Epidemiology, University of Birmingham T.P.Marshall@bham.ac.uk Understanding Variation Using Registry data to](https://reader030.vdocuments.us/reader030/viewer/2022032414/56649ee15503460f94bf1f0e/html5/thumbnails/2.jpg)
Acknowledgements
Mohammed A. Mohammed,
Department of Public Health & Epidemiology, University of Birmingham
![Page 3: Tom Marshall Department of Public Health & Epidemiology, University of Birmingham T.P.Marshall@bham.ac.uk Understanding Variation Using Registry data to](https://reader030.vdocuments.us/reader030/viewer/2022032414/56649ee15503460f94bf1f0e/html5/thumbnails/3.jpg)
Quality & Variation• Conventional tools
– Standard setting• Clinical Audit
– Ranking or League tables
– Hypothesis testing
• Effect– Pass or fail
– Action on those that fail
• Another way ..
![Page 4: Tom Marshall Department of Public Health & Epidemiology, University of Birmingham T.P.Marshall@bham.ac.uk Understanding Variation Using Registry data to](https://reader030.vdocuments.us/reader030/viewer/2022032414/56649ee15503460f94bf1f0e/html5/thumbnails/4.jpg)
Shewhart’s Concepts• Letter a
COMMON CAUSEACTION: PROCESS
SPECIAL CAUSEACTION:
FIND & ELIMINATE
PROCESS OF WRITING
![Page 5: Tom Marshall Department of Public Health & Epidemiology, University of Birmingham T.P.Marshall@bham.ac.uk Understanding Variation Using Registry data to](https://reader030.vdocuments.us/reader030/viewer/2022032414/56649ee15503460f94bf1f0e/html5/thumbnails/5.jpg)
![Page 6: Tom Marshall Department of Public Health & Epidemiology, University of Birmingham T.P.Marshall@bham.ac.uk Understanding Variation Using Registry data to](https://reader030.vdocuments.us/reader030/viewer/2022032414/56649ee15503460f94bf1f0e/html5/thumbnails/6.jpg)
Basis of Control Limits
• Tchebycheff’s theorem
X mean +/- t SD
P > 1 - 1/t2
• t=3
• Economic– common cause vs special cause
![Page 7: Tom Marshall Department of Public Health & Epidemiology, University of Birmingham T.P.Marshall@bham.ac.uk Understanding Variation Using Registry data to](https://reader030.vdocuments.us/reader030/viewer/2022032414/56649ee15503460f94bf1f0e/html5/thumbnails/7.jpg)
![Page 8: Tom Marshall Department of Public Health & Epidemiology, University of Birmingham T.P.Marshall@bham.ac.uk Understanding Variation Using Registry data to](https://reader030.vdocuments.us/reader030/viewer/2022032414/56649ee15503460f94bf1f0e/html5/thumbnails/8.jpg)
Special cause variation - action
1. Data • Accuracy• Definition of errors
2. Raw materials• Difficulty of tasks
3. Equipment, facilities, staffing• Typewriters, workload, lighting
4. Processes, procedures• How is the work organised?
5. People• Skill levels and techniques
![Page 9: Tom Marshall Department of Public Health & Epidemiology, University of Birmingham T.P.Marshall@bham.ac.uk Understanding Variation Using Registry data to](https://reader030.vdocuments.us/reader030/viewer/2022032414/56649ee15503460f94bf1f0e/html5/thumbnails/9.jpg)
![Page 10: Tom Marshall Department of Public Health & Epidemiology, University of Birmingham T.P.Marshall@bham.ac.uk Understanding Variation Using Registry data to](https://reader030.vdocuments.us/reader030/viewer/2022032414/56649ee15503460f94bf1f0e/html5/thumbnails/10.jpg)
Application to Health Care
• Case studies
![Page 11: Tom Marshall Department of Public Health & Epidemiology, University of Birmingham T.P.Marshall@bham.ac.uk Understanding Variation Using Registry data to](https://reader030.vdocuments.us/reader030/viewer/2022032414/56649ee15503460f94bf1f0e/html5/thumbnails/11.jpg)
Surgeon Variability• Surgeon Survived Died %• A 82 16 16• B 58 8 12• C 49 9 16• D 45 7 13• E 37 15 29• F 41 5 11• G 35 3 8• H 26 11 30• I 31 5 14• J 27 7 21• K 28 4 13• L 19 2 10• M 18 3 14
McArdle & Hole BMJ 1991;302:1501-5
Conclusions: “There were significant variations in patient outcome among surgeons after surgery forcolorectal cancer; such differences compromisesurvival. A considerable improvement in overallsurvival might be achieved if such surgery wereundertaken by surgeons with a special interest incolorectal surgery or surgical oncology.”
![Page 12: Tom Marshall Department of Public Health & Epidemiology, University of Birmingham T.P.Marshall@bham.ac.uk Understanding Variation Using Registry data to](https://reader030.vdocuments.us/reader030/viewer/2022032414/56649ee15503460f94bf1f0e/html5/thumbnails/12.jpg)
Surgeon Variability
McArdle & Hole BMJ 1991;302:1501-5
A
BC
D
E
F
G
H
IJ
K
LM
X Number alive
Y N
um
nber
die
d
..
Common cause variation
![Page 13: Tom Marshall Department of Public Health & Epidemiology, University of Birmingham T.P.Marshall@bham.ac.uk Understanding Variation Using Registry data to](https://reader030.vdocuments.us/reader030/viewer/2022032414/56649ee15503460f94bf1f0e/html5/thumbnails/13.jpg)
Fractured Hips
• 90 day mortality (N=580; 104 deaths 18%)– Hospital Mortality
• 1 19/79 24%
• 2 5/24 21%
• 3 16/79 20%
• 4 19/80 24%
• 5 12/80 15%
• 6 4/81 5%
• 7 14/79 18%
• 8 15/63 19%
Todd et al BMJ 1995;301:904-8
Conclusions: “Uncritical acceptance of the "advantages" of hospital 6 should, however, be avoided as random variationalmost certainly plays some part in these findings.”
![Page 14: Tom Marshall Department of Public Health & Epidemiology, University of Birmingham T.P.Marshall@bham.ac.uk Understanding Variation Using Registry data to](https://reader030.vdocuments.us/reader030/viewer/2022032414/56649ee15503460f94bf1f0e/html5/thumbnails/14.jpg)
Fractured Hips
X Number alive
Y N
umbe
r di
ed
Todd et al BMJ 1995;301:904-8
Special cause variation
![Page 15: Tom Marshall Department of Public Health & Epidemiology, University of Birmingham T.P.Marshall@bham.ac.uk Understanding Variation Using Registry data to](https://reader030.vdocuments.us/reader030/viewer/2022032414/56649ee15503460f94bf1f0e/html5/thumbnails/15.jpg)
Western Electric Company Rules
• Additional rules for detecting special causes– 1 data point >3 sigma from mean– 2 out of 3 data points >2 sigma from mean– 4 out of 5 data points >1 sigma from mean– 9 successive data points on one side of mean– Trend of 6 successive data points
![Page 16: Tom Marshall Department of Public Health & Epidemiology, University of Birmingham T.P.Marshall@bham.ac.uk Understanding Variation Using Registry data to](https://reader030.vdocuments.us/reader030/viewer/2022032414/56649ee15503460f94bf1f0e/html5/thumbnails/16.jpg)
Special cause variation: nine successive data points below the mean
![Page 17: Tom Marshall Department of Public Health & Epidemiology, University of Birmingham T.P.Marshall@bham.ac.uk Understanding Variation Using Registry data to](https://reader030.vdocuments.us/reader030/viewer/2022032414/56649ee15503460f94bf1f0e/html5/thumbnails/17.jpg)
Renal Registry Data
• Average haemoglobin per quarter
![Page 18: Tom Marshall Department of Public Health & Epidemiology, University of Birmingham T.P.Marshall@bham.ac.uk Understanding Variation Using Registry data to](https://reader030.vdocuments.us/reader030/viewer/2022032414/56649ee15503460f94bf1f0e/html5/thumbnails/18.jpg)
Run chart of quarterly mean Hb
9
10
11
12
13
14
9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48
Quarter
Qu
arte
rly
mea
n H
b
UK
Average
Run Chart – sequential data points + mean
Trend of 6 data points
![Page 19: Tom Marshall Department of Public Health & Epidemiology, University of Birmingham T.P.Marshall@bham.ac.uk Understanding Variation Using Registry data to](https://reader030.vdocuments.us/reader030/viewer/2022032414/56649ee15503460f94bf1f0e/html5/thumbnails/19.jpg)
Interpretation
• Rising trend in mean Hb nationally
• Difficult to interpret changing Hb in a single centre except in relation to rising trend nationally
![Page 20: Tom Marshall Department of Public Health & Epidemiology, University of Birmingham T.P.Marshall@bham.ac.uk Understanding Variation Using Registry data to](https://reader030.vdocuments.us/reader030/viewer/2022032414/56649ee15503460f94bf1f0e/html5/thumbnails/20.jpg)
Run chart of quarterly mean Hb
9
10
11
12
13
14
Jan-
97
May
-97
Sep-9
7
Jan-
98
May
-98
Sep-9
8
Jan-
99
May
-99
Sep-9
9
Jan-
00
May
-00
Sep-0
0
Jan-
01
May
-01
Sep-0
1
Jan-
02
May
-02
Sep-0
2
Jan-
03
May
-03
Sep-0
3
Jan-
04
May
-04
Sep-0
4
Jan-
05
May
-05
Sep-0
5
Jan-
06
May
-06
Sep-0
6
Quarter
Qu
arte
rly
mea
n H
b
Middlb
Average
![Page 21: Tom Marshall Department of Public Health & Epidemiology, University of Birmingham T.P.Marshall@bham.ac.uk Understanding Variation Using Registry data to](https://reader030.vdocuments.us/reader030/viewer/2022032414/56649ee15503460f94bf1f0e/html5/thumbnails/21.jpg)
XMR chart of difference between quarterly mean Hb and national average
-2.0
-1.5
-1.0
-0.5
0.0
0.5
1.0
1.5
2.0Ja
n-97
Jul-9
7
Jan-
98
Jul-9
8
Jan-
99
Jul-9
9
Jan-
00
Jul-0
0
Jan-
01
Jul-0
1
Jan-
02
Jul-0
2
Jan-
03
Jul-0
3
Jan-
04
Jul-0
4
Jan-
05
Jul-0
5
Jan-
06
Jul-0
6
Qu
arte
rly
mea
n H
b
Middlb
Average
+3 sig
-3 sig
Consistent with national average
9 data points above mean i.e. own long term average
Below national average
10 data points below mean
![Page 22: Tom Marshall Department of Public Health & Epidemiology, University of Birmingham T.P.Marshall@bham.ac.uk Understanding Variation Using Registry data to](https://reader030.vdocuments.us/reader030/viewer/2022032414/56649ee15503460f94bf1f0e/html5/thumbnails/22.jpg)
XMR chart of difference between quarterly mean Hb and national average
-2.0
-1.5
-1.0
-0.5
0.0
0.5
1.0
1.5
2.0Ja
n-97
Jul-9
7
Jan-
98
Jul-9
8
Jan-
99
Jul-9
9
Jan-
00
Jul-0
0
Jan-
01
Jul-0
1
Jan-
02
Jul-0
2
Jan-
03
Jul-0
3
Jan-
04
Jul-0
4
Jan-
05
Jul-0
5
Jan-
06
Jul-0
6
Qu
arte
rly
mea
n H
b
Middlb
Average
+3 sig
-3 sig
![Page 23: Tom Marshall Department of Public Health & Epidemiology, University of Birmingham T.P.Marshall@bham.ac.uk Understanding Variation Using Registry data to](https://reader030.vdocuments.us/reader030/viewer/2022032414/56649ee15503460f94bf1f0e/html5/thumbnails/23.jpg)
XMR chart of quarterly mean Hb
9
10
11
12
13
14
Jan-
97
May
-97
Sep-9
7
Jan-
98
May
-98
Sep-9
8
Jan-
99
May
-99
Sep-9
9
Jan-
00
May
-00
Sep-0
0
Jan-
01
May
-01
Sep-0
1
Jan-
02
May
-02
Sep-0
2
Jan-
03
May
-03
Sep-0
3
Jan-
04
May
-04
Sep-0
4
Jan-
05
May
-05
Sep-0
5
Jan-
06
May
-06
Sep-0
6
Quarter
Qu
arte
rly
mea
n H
b
Truro
Average
Run Chart – difference between this centre + UK Average
9 data points above mean
Average determined from first 8 data points
![Page 24: Tom Marshall Department of Public Health & Epidemiology, University of Birmingham T.P.Marshall@bham.ac.uk Understanding Variation Using Registry data to](https://reader030.vdocuments.us/reader030/viewer/2022032414/56649ee15503460f94bf1f0e/html5/thumbnails/24.jpg)
Control Chart – Run Chart + 3 sigma limits
XMR chart of quarterly mean Hb
9
10
11
12
13
14
Jan-
97
May
-97
Sep-9
7
Jan-
98
May
-98
Sep-9
8
Jan-
99
May
-99
Sep-9
9
Jan-
00
May
-00
Sep-0
0
Jan-
01
May
-01
Sep-0
1
Jan-
02
May
-02
Sep-0
2
Jan-
03
May
-03
Sep-0
3
Jan-
04
May
-04
Sep-0
4
Jan-
05
May
-05
Sep-0
5
Jan-
06
May
-06
Sep-0
6
Quarter
Qu
arte
rly
mea
n H
b
Truro
Average
+3 sig
-3 sig
![Page 25: Tom Marshall Department of Public Health & Epidemiology, University of Birmingham T.P.Marshall@bham.ac.uk Understanding Variation Using Registry data to](https://reader030.vdocuments.us/reader030/viewer/2022032414/56649ee15503460f94bf1f0e/html5/thumbnails/25.jpg)
XMR chart of quarterly mean Hb
9
10
11
12
13
14
Jan-
97
May
-97
Sep-9
7
Jan-
98
May
-98
Sep-9
8
Jan-
99
May
-99
Sep-9
9
Jan-
00
May
-00
Sep-0
0
Jan-
01
May
-01
Sep-0
1
Jan-
02
May
-02
Sep-0
2
Jan-
03
May
-03
Sep-0
3
Jan-
04
May
-04
Sep-0
4
Jan-
05
May
-05
Sep-0
5
Jan-
06
May
-06
Sep-0
6
Quarter
Qu
arte
rly
mea
n H
b
Truro
Average
+3 sig
-3 sigConsistent with two stable processes:
before Sept 04& after Sept 04
![Page 26: Tom Marshall Department of Public Health & Epidemiology, University of Birmingham T.P.Marshall@bham.ac.uk Understanding Variation Using Registry data to](https://reader030.vdocuments.us/reader030/viewer/2022032414/56649ee15503460f94bf1f0e/html5/thumbnails/26.jpg)
Special cause variation - action
1. Data (including definitions)
2. Raw materials (case-mix)
3. Equipment, facilities, staffing
4. Processes, procedures
5. People
![Page 27: Tom Marshall Department of Public Health & Epidemiology, University of Birmingham T.P.Marshall@bham.ac.uk Understanding Variation Using Registry data to](https://reader030.vdocuments.us/reader030/viewer/2022032414/56649ee15503460f94bf1f0e/html5/thumbnails/27.jpg)
Monitoring Many Centres
![Page 28: Tom Marshall Department of Public Health & Epidemiology, University of Birmingham T.P.Marshall@bham.ac.uk Understanding Variation Using Registry data to](https://reader030.vdocuments.us/reader030/viewer/2022032414/56649ee15503460f94bf1f0e/html5/thumbnails/28.jpg)
Walter A Shewhart 1931
![Page 29: Tom Marshall Department of Public Health & Epidemiology, University of Birmingham T.P.Marshall@bham.ac.uk Understanding Variation Using Registry data to](https://reader030.vdocuments.us/reader030/viewer/2022032414/56649ee15503460f94bf1f0e/html5/thumbnails/29.jpg)
“The central problem in management and leadership …is failure to understand the
information in variation”
William E Deming 1986 Out of the Crisis MIT pg 309
![Page 30: Tom Marshall Department of Public Health & Epidemiology, University of Birmingham T.P.Marshall@bham.ac.uk Understanding Variation Using Registry data to](https://reader030.vdocuments.us/reader030/viewer/2022032414/56649ee15503460f94bf1f0e/html5/thumbnails/30.jpg)
Summary• Shewhart’s concepts
– Understand variation– Simple & powerful– Guide action– Wide application
• Continual improvement• Clinical governance• Other implications ..
![Page 31: Tom Marshall Department of Public Health & Epidemiology, University of Birmingham T.P.Marshall@bham.ac.uk Understanding Variation Using Registry data to](https://reader030.vdocuments.us/reader030/viewer/2022032414/56649ee15503460f94bf1f0e/html5/thumbnails/31.jpg)
Implications• Prediction
– Limits of common cause variation– Statistical control
• Action– Common cause variation
• League tables, ranking, hypothesis testing all misleading• Improve process/system as a whole
– Special cause variation• Investigate and eliminate (or learn lessons)
• Data order important
![Page 32: Tom Marshall Department of Public Health & Epidemiology, University of Birmingham T.P.Marshall@bham.ac.uk Understanding Variation Using Registry data to](https://reader030.vdocuments.us/reader030/viewer/2022032414/56649ee15503460f94bf1f0e/html5/thumbnails/32.jpg)
![Page 33: Tom Marshall Department of Public Health & Epidemiology, University of Birmingham T.P.Marshall@bham.ac.uk Understanding Variation Using Registry data to](https://reader030.vdocuments.us/reader030/viewer/2022032414/56649ee15503460f94bf1f0e/html5/thumbnails/33.jpg)
How It Works In Industry
• Balanced set of measures
![Page 34: Tom Marshall Department of Public Health & Epidemiology, University of Birmingham T.P.Marshall@bham.ac.uk Understanding Variation Using Registry data to](https://reader030.vdocuments.us/reader030/viewer/2022032414/56649ee15503460f94bf1f0e/html5/thumbnails/34.jpg)
Balanced Set Of Measures
Customer
Financial
Internal ExternalAim
Four or five measures for each box
![Page 35: Tom Marshall Department of Public Health & Epidemiology, University of Birmingham T.P.Marshall@bham.ac.uk Understanding Variation Using Registry data to](https://reader030.vdocuments.us/reader030/viewer/2022032414/56649ee15503460f94bf1f0e/html5/thumbnails/35.jpg)
Balanced Set Of Measures
Patient Experience
Financial / Resources
Clinical effectiveness
Strategic Effectiveness
Aim
![Page 36: Tom Marshall Department of Public Health & Epidemiology, University of Birmingham T.P.Marshall@bham.ac.uk Understanding Variation Using Registry data to](https://reader030.vdocuments.us/reader030/viewer/2022032414/56649ee15503460f94bf1f0e/html5/thumbnails/36.jpg)
Special Causes
• Identify special causes in each domain
• Collate & prioritise for action– Low hanging fruit first