Process Improvement in Healthcare: Volunteer Clinic Case Study

Nonparametric Statistics

ISE 491 Fall 2009

Dr. Joan BurtnerAssociate Professor, Department of Industrial Engineering and Industrial Management

Fall 2009 ISE 491 Dr. Burtner ~ Clinic Case Study Slide 2

Case Study Description

Volunteer Clinic Utilization Study Methods

Prediction of Future Demand (Forecasting) Interviews with Key Clinical Personnel On-site Observations (Time Studies) Review of Policies and Procedures Process Mapping Retrospective Data Analysis

Fall 2009 ISE 491 Dr. Burtner ~ Clinic Case Study Slide 3

Does Volunteerism Vary By Month?

Factor: Month (3 levels, A B C) Sample: Random selection of 9 physicians per

month Balanced design (3x9=27 observations) Outcome: Patient contact hours Interval level data With an assumption that the underlying

distribution for each month is normal, the appropriate hypothesis test is a one-way ANOVA

Statistical package: Minitab 14 or 15

Fall 2009 ISE 491 Dr. Burtner ~ Clinic Case Study Slide 4

Raw Data and Minitab Format

Demo DemoNumber Month

70 A30 A26 A60 A34 A26 A57 A39 A44 A53 B39 B27 B29 B23 B28 B25 B23 B22 B36 C23 C29 C34 C16 C21 C23 C25 C20 C

MonthA

70

30

26

60

34

26

57

39

44

MonthB

53

39

27

29

23

28

25

23

22

MonthC

36

23

29

34

16

21

23

25

20

Fall 2009 ISE 491 Dr. Burtner ~ Clinic Case Study Slide 5

One Way ANOVA - Stacked

One-way ANOVA: DemoNumber versus DemoMonth

Source DF SS MS F PDemoMonth 2 1509 754 5.63 0.010Error 24 3217 134Total 26 4726

S = 11.58 R-Sq = 31.92% R-Sq(adj) = 26.25%

Individual 95% CIs For Mean Based on Pooled StDevLevel N Mean StDev ---+---------+---------+---------+------A 9 42.89 16.04 (-------*-------)B 9 29.89 10.07 (-------*-------)C 9 25.22 6.59 (-------*-------) ---+---------+---------+---------+------ 20 30 40 50

Pooled StDev = 11.58

Fall 2009 ISE 491 Dr. Burtner ~ Clinic Case Study Slide 6

Simultaneous Confidence Intervals

Tukey 95% Simultaneous Confidence IntervalsAll Pairwise Comparisons among Levels of DemoMonth

Individual confidence level = 98.02%

DemoMonth = A subtracted from:

DemoMonth Lower Center Upper -+---------+---------+---------+--------B -26.62 -13.00 0.62 (--------*--------)C -31.29 -17.67 -4.04 (--------*--------) -+---------+---------+---------+-------- -30 -15 0 15

DemoMonth = B subtracted from:

DemoMonth Lower Center Upper -+---------+---------+---------+--------C -18.29 -4.67 8.96 (--------*--------) -+---------+---------+---------+-------- -30 -15 0 15

Fall 2009 ISE 491 Dr. Burtner ~ Clinic Case Study Slide 7

Initial interpretation

Null hypothesis: The mean number of patient contact hours (in the population) is the same for each month

Alternate hypothesis: The mean number of patient hours (in the population) differs for at least one month

Assumed significance level = 0.05 P-value reported = 0.010 Decision: Reject null hypothesis Conclusion: Physician volunteer hours vary by month

Fall 2009 ISE 491 Dr. Burtner ~ Clinic Case Study Slide 8

Interpretation of Tukey CIs

The means for Month A and B are not significantly different

The means for Month A and C are significantly different; Month A mean contact hours is significantly greater than Month C mean contact hours

The means for Month B and C are not significantly different

Fall 2009 ISE 491 Dr. Burtner ~ Clinic Case Study Slide 9

Validation of Assumptions

Residual

Perc

ent

30150-15-30

99

90

50

10

1

Fitted Value

Resi

dual

4540353025

20

10

0

-10

-20

Residual

Fre

quency

20100-10

8

6

4

2

0

Observation Order

Resi

dual

2624222018161412108642

20

10

0

-10

-20

Normal Probability Plot of the Residuals Residuals Versus the Fitted Values

Histogram of the Residuals Residuals Versus the Order of the Data

Residual Plots for DemoNumber

Fall 2009 ISE 491 Dr. Burtner ~ Clinic Case Study Slide 10

Does Volunteerism Vary By Month? Part 2

Outcome: Total number of patient contact hours Factor: Month (3 levels, A B C) Random selection of 9 physicians per month Balanced design Interval level data No assumption that the underlying distribution for

each month is normal Appropriate analysis is a Kruskal-Wallis or Mood

Median Test Statistical package: Minitab 14 or 15

Fall 2009 ISE 491 Dr. Burtner ~ Clinic Case Study Slide 11

Mood Median Results

Mood Median Test: DemoNumber versus DemoMonth

Mood median test for DemoNumber

Chi-Square = 4.75 DF = 2 P = 0.093

Individual 95.0% CIs

DemoMonth N<= N> Median Q3-Q1 ---+---------+---------+---------+---

A 2 7 39.0 30.5 (----------*---------------)

B 6 3 27.0 11.0 (---*-------)

C 6 3 23.0 11.0 (-*-------)

---+---------+---------+---------+---

24 36 48 60

Overall median = 28.0

Fall 2009 ISE 491 Dr. Burtner ~ Clinic Case Study Slide 12

Source: Minitab Help Guide – Mood’s Median Test

Stat > Nonparametrics > Mood's Median Test Mood's median test can be used to test the equality of medians

from two or more populations and, like the Kruskal-Wallis Test, provides an nonparametric alternative to the one-way analysis of variance. Mood's median test is sometimes called a median test or sign scores test. Mood's median test tests:

H0: the population medians are all equal versus H1: the medians are not all equal

An assumption of Mood's median test is that the data from each population are independent random samples and the population distributions have the same shape. Mood's median test is robust against outliers and errors in data and is particularly appropriate in the preliminary stages of analysis. Mood's median test is more robust than is the Kruskal-Wallis test against outliers, but is less powerful for data from many distributions, including the normal.

Fall 2009 ISE 491 Dr. Burtner ~ Clinic Case Study Slide 13

Kruskal – Wallis Results

Kruskal-Wallis Test: DemoNumber versus DemoMonth

Kruskal-Wallis Test on DemoNumber

DemoMonth N Median Ave Rank ZA 9 39.00 20.1 2.83B 9 27.00 12.7 -0.59C 9 23.00 9.2 -2.24Overall 27 14.0

H = 8.91 DF = 2 P = 0.012H = 8.95 DF = 2 P = 0.011 (adjusted for ties)

Fall 2009 ISE 491 Dr. Burtner ~ Clinic Case Study Slide 14

Source: Minitab Help Guide – Kruskal-Wallis

You can perform a Kruskal-Wallis test of the equality of medians for two or more populations.

This test is a generalization of the procedure used by the Mann-Whitney test and, like Mood's Median test, offers a nonparametric alternative to the one-way analysis of variance. The Kruskal-Wallis hypotheses are:

H0: the population medians are all equal versus H1: the medians are not all equal

An assumption for this test is that the samples from the different populations are independent random samples from continuous distributions, with the distributions having the same shape. The Kruskal-Wallis test is more powerful than Mood's median test for data from many distributions, including data from the normal distribution, but is less robust against outliers.

Fall 2009 ISE 491 Dr. Burtner ~ Clinic Case Study Slide 15

Modified Design and Analysis

The experimenter did not assume that the number of volunteer hours follows a normal distribution

The experimenter collected data for the same nine physicians for three different 31-day months

Since the design includes three groups (months) blocked by physicians, the appropriate hypothesis test would be the Friedman

Response: Load (Hours) Treatment: Month31 Blocks: Physician

Load Month31Physician

70 January Allen 30 January Brown 26 January Cook 60 January Dodd 34 January Ellis 26 January Frank 57 January Grey 39 January

Howard 44 January Ingle 53 May Allen 39 May Brown 27 May Cook 29 May Dodd 23 May Ellis 28 May Frank 25 May Grey 23 May

Howard 22 May Ingle 36 July Allen 23 July Brown 29 July Cook 34 July Dodd 16 July Ellis 21 July Frank 23 July Grey 25 July

Howard 20 July Ingle

Fall 2009 ISE 491 Dr. Burtner ~ Clinic Case Study Slide 16

Source: Minitab Help Guide – Friedman Test

Stat > Nonparametrics > Friedman Friedman test is a nonparametric analysis of a

randomized block experiment, and thus provides an alternative to the Two-way analysis of variance. The hypotheses are:

H0: all treatment effects are zero versus H1: not all treatment effects are zero

Randomized block experiments are a generalization of paired experiments, and the Friedman test is a generalization of the paired sign test.

Fall 2009 ISE 491 Dr. Burtner ~ Clinic Case Study Slide 17

Friedman Test Results

Friedman Test: Load versus Month31 blocked by Physician

S = 5.56 DF = 2 P = 0.062

Sum Est ofMonth31 N Median RanksJanuary 9 41.00 23.0July 9 23.00 13.0May 9 25.00 18.0

Grand median = 29.67

Fall 2009 ISE 491 Dr. Burtner ~ Clinic Case Study Slide 18

Interpretation of Friedman

P = 0.062 For an assumed alpha level of 0.05, there is

insufficient evidence to reject H0 because the p-value is greater than the alpha level.

Therefore we conclude that the data do not support the hypothesis that any of the treatment effects are different from zero.

Physician volunteerism, in terms of patient contact hours, does not vary significantly by month.