tutorial on risk adjusted p-chart farrokh alemi, ph.d
Post on 15-Dec-2015
217 Views
Preview:
TRANSCRIPT
Tutorial on Risk Adjusted P-chart
Farrokh Alemi, Ph.D.
Have Changes Led to Improvement?
Common cause variation (changes in outcomes because of chance) is everywhere.
Decision makers often mistakenly attribute positive outcomes to their own skills and
negative outcomes to others, while in reality both could be a chance outcome
Why Chart Data?
To discipline intuitions
To communicate data in vivid graphical ways
Purpose of Risk Adjustment
Purpose of p-chart Purpose of risk-adjusted p-chart
To detect if the process has improved beyond
historical levels. Assumes historically we have been serving the same type of
patients as now
To detect if the process has improved beyond what can be expected from patient conditions
Data Needed
Data collected over time
Risk (expected outcomes) for each patient
Outcomes for each patient
The purpose is to improve not to get so lost in measurement to loose sight of improvement.
What Is Risk?
A patient’s condition or characteristics that affects the expected outcomes for the patient A severity index used to predict patient
outcomesClinicians’ consensus regarding expected
outcomesPatient’s self rating of expected outcomes
Time 1 Time 2 Time 3 Time 4 Time 5 Time 6 Time 7 Time 8
# deaths 2 3 1 3 2 2 4 2
8 9 7 7 7 7 7 8
Case
1 0.18 0.97 0.85 0.27 0.12 0.07 0.96 0.05
2 0.88 0.88 0.61 0.71 0.44 0.05 0.05 0.96
3 0.33 0.04 0.27 0.07 0.18 0.93 0.75 0.96
4 0.29 0.29 0.28 0.74 0.67 0.24 0.04 0.14
5 0.14 0.03 0.8 0.08 0.51 0.14 0.96 0.05
6 0.24 0.19 0.71 0.04 0.62 0.58 0.71 0.58
7 0.15 0.14 0.85 0.76 0.67 0.05 0.15 0.07
8 0.04 0.74 0.16
9 0.07
Mortality Risks for Inidvidual Patients
MI Patients Over 8 Months in One Hospital
Observed mortality during this time period
Nu
mb
er o
f ca
ses
Expected probability of mortality for case 8 in time period 1. Estimated from severity indices or experts’ consensus.
Elements of a Control Chart
X axis shows timeY axis shows probability of adverse eventsObserved rates are plotted against time sequenceUpper control limit is a line drawn so that points above it are rare to be observed by mere chanceLower control limit is a line drawn so that points below it are rare to observe by mere chance Lets take
a look
An Example of P-chart
Example p-chart
0
0.2
0.4
0.6
0.8
0 1 2 3 4 5 6 7
Time
Pro
ba
bil
ity
of
ad
vers
e o
utc
om
eUpper control limit
Lower control limit
Observed rate
Steps in Creating P-chart for Mortality
1. Check assumptions
2. Calculate observed rates and plot them
3. Calculate expected rates and plot them
4. Calculate expected deviation
5. Calculate control limits and plot them
6. Interpret findings
7. Distribute chart and interpretation
Step One: Check Assumptions
We are examining discrete events that either happen or do not happen, e.g. mortality among MI patients, falls among nursing home patients, error in medical record entry, etc.The event is not rare, meaning the probability of it occurring exceeds 5% for each time period.Observed events are independent from each other. The probability of the event occurring does not change over time. This assumption is violated if one patient’s outcomes affects the outcomes for others, e.g. when dealing with infectious diseases.
Step Two: Calculate Observed Rates and Plot
Pi = Mortality rate in time period “i”
Oi = Mortality in period “i”
ni = Number of cases in time period “i”
Pi = Oi / ni
Number Observed Mortalityof cases Mortality rate
8 2 0.25
9 3 0.33
7 1 0.14
7 3 0.43
7 2 0.29
7 2 0.29
7 4 0.57
8 2 0.25
Observed Mortality Rates for All Time Periods
Plot of mortality rates
PeriodNumber of cases
Observed mortality
Expected mortality
Expected deviation
Upper limit
Lower limit
T1 8 0.25
T2 9 0.33
T3 7 0.14
T4 7 0.43
T5 7 0.29
T6 7 0.29
T7 7 0.57
T8 8 0.25
Plot of the Observed Rates
Time period 7 and 3 seem different but don’t rush to judgment.
Wait, until you see control limits of what could have been expected.
0.00
0.10
0.20
0.30
0.40
0.50
1 2 3 4 5 6 7 8
Time periodM
ort
ali
ty r
ate
Step Three: Calculate Expected Mortality
Eij = Expected mortality of case ‘j’ in time period “i”
Ei = Expected mortality for time period “i”
Ei = (j=1,…,ni Eij ) / ni
Sample calculation:
E1= (.18+.88+.33+.29 +.14+.24+.15+.04)/8
Expected Mortality Rates for All Time Periods
PeriodNumber of cases
Observed mortality
Expected mortality
Expected deviation
Upper limit
Lower limit
T1 8 0.25 0.28
T2 9 0.33 0.37
T3 7 0.14 0.62
T4 7 0.43 0.38
T5 7 0.29 0.46
T6 7 0.29 0.29
T7 7 0.57 0.52
T8 8 0.25 0.37
Plot of Expected Mortality
Plotting expected mortality helps interpret the observed rates but does not settle the question of whether differences are due to chance.
0.00
0.20
0.40
0.60
0.80
T1 T2 T3 T4 T5 T6 T7 T8P
rob
ab
ility
of
mo
rta
lity
Observed mortality Expected mortality
Step Four: Calculate Expected Deviation
Eij = Expected mortality of case ‘j’ in time period “i”
Di = Standard deviation of expected mortality in time period “i”, called by us as the Expected Deviation
Di = ( j=1,…,ni Eij (1-Eij))0.5 / ni
See sample calculation
Expected Deviation for Time Period 1
Case
Expected mortality E1j E1j *(1-E1j)
1 0.18 0.15
2 0.88 0.11
3 0.33 0.22
4 0.29 0.21
5 0.14 0.12
6 0.24 0.18
7 0.15 0.13
8 0.04 0.04
1.15
1.07
0.13Divided by # of casesSquare root of sum
SumExpected deviation
A
BC
D
Results: Expected Deviations for All Time
Periods
PeriodNumber of cases
Observed mortality
Expected mortality
Expected deviation
Upper limit
Lower limit
T1 8 0.25 0.28 0.13
T2 9 0.33 0.37 0.11
T3 7 0.14 0.62 0.16
T4 7 0.43 0.38 0.14
T5 7 0.29 0.46 0.17
T6 7 0.29 0.29 0.13
T7 7 0.57 0.52 0.05
T8 8 0.25 0.37 0.11
Step Five: Calculate Control Limits
UCLi = Upper control limit for time period “i”
LCLi = Lower control limit for time period “i”
t = Constant based on t-student distribution
UCLi = Ei + t * Di
LCLi = Ei - t * Di
Where for 95% confidence intervals:Degrees of
freedom
95% student t
value
7 2.37
8 2.31
9 2.26
10 2.23
Calculation of Control Limits for Time Period 1
UCL = .28 + 2.37 * .13 = .59
LCL = .28 –2.37 * .13 = -.03
t-value
Negative limits are set to zero as negative probabilities are not possible
Results: Control Limits for All Time Periods
PeriodNumber of cases
Observed mortality
Expected mortality
Expected deviation
Upper limit
Lower limit
T1 8 0.25 0.28 0.13 0.60 0.00
T2 9 0.33 0.37 0.11 0.62 0.12
T3 7 0.14 0.62 0.16 1.02 0.23
T4 7 0.43 0.38 0.14 0.72 0.04
T5 7 0.29 0.46 0.17 0.88 0.04
T6 7 0.29 0.29 0.13 0.60 0.00
T7 7 0.57 0.52 0.05 0.64 0.39
T8 8 0.25 0.37 0.11 0.63 0.12
Plot Control Limits
UCL
LCL
Step Six: Interpret Findings
There are no points above UCL. There is one point below LCL.In time period 3, mortality is lower than what can be expected from patient’s conditions.All other time periods are within expectations, even time period 7 with its high mortality rate is within expectation.
Step Seven: Distribute Control Chart
Include in the information:How was severity measured and expected
mortality anticipated?Why are assumptions met?What does the control chart look like?What is the interpretation of the findings?
Index of ContentClick on the Slide You Wish to Review
1. Check assumptions
2. Calculate and plot observed mortality
3. Calculate expected mortality
4. Calculate expected deviation
5. Calculate and plot control limits
6. Interpret findings
7. Distribute control chart
top related