if you are qualified, fill out make-up exam form by 5pm ...xuanyaoh/stat350/xymar23lec22.pdf · if...
TRANSCRIPT
If you are qualified, Fill out make-up exam form by 5pm this Friday, attach necessary documents.
Check your calendar…
3/23/12 Lecture 20 1
Interpreting the ANOVA results
Chapter 9
3/23/12 2 Lecture 20
Assumptions (prior to ANOVA)
• Two important assumptions for ANOVA 1. Constant variance: The variances of the k
populations are the same. – Check this with the ratio of the largest and smallest
standard deviations, the ratio must be < 2
2. Each of the k populations follows a normal distribution.
– Check this by looking at QQplots for each group
3/23/12 3 Lecture 19
• ANOVA is a good example of a situation where often a nonsignificant test is actually useful. – Suppose we are comparing a new drug to several standard
drugs already used – Suppose also that the new drug is less expensive to produce – In this case, mostly what we’d like to show is that the new
drug is at least effective as the other standard drugs used – So in this situation, a non-significant ANOVA is a great result!
• Remember: statistical significance ≠ practical significance
Quick sidenote
3/23/12 4 Lecture 19
9.3 Interpreting ANOVA results
• If the results are significant in ANOVA, we’d like to know explicitly which means are different
• Remark: If insignificant in ANOVA, we don’t have to try further steps…
• Two benefits of ANOVA 1. Single test with single chance (α) of type I error 2. Better estimation of error among all groups
• By comparing all the groups simultaneously, we get a better picture of the overall error among groups.
3/23/12 5 Lecture 20
How do we know which means are different?
• We need some "Supplementary Analysis" to tell that. – One way is to do a check visually, using the "effect plots"
• Scatter plot of means • Side-By-Side Boxplots
– Another way is to perform multiple comparison of means
• Tukey's method • Dunnett's method • many more: LSD, Scheffe, Bonferroni, FDR, etc.
Actually, We can compare the means pairwise and keep the two benefits of ANOVA, we do this by adjusting the T value we use to compare the two means.
3/23/12 6 Lecture 20
Visual Check - Boxplots
3/23/12 Lecture 20 7
Visual Check – Scatterplot of Means
3/23/12 Lecture 20 8
Multiple comparisons (General Concepts)
• How far apart do two means need to be to be statistically significant? – This value can be calculated directly similar to
what we did with confidence intervals • It functions like a t from a t-test
– Generally, the critical value (t for example) is modified to “correct” the inflated type I error rate to keep it at the desired α level (like 0.05)
• So instead of an α error rate for each test, we get an “family” α error rate—one rate for the entire comparison
3/23/12 9 Lecture 20
Tukey’s Method • Controls the type I error rate directly by modifying
the T value
or if ni = nj • qα comes from the studentized range tables
– Table IX on pages 577-578 – Df of qα is (k, n – k) for single-factor ANOVA
• If the difference in two means is greater than this critical value, we say those two means are statistically significantly different
)11(2 ji nnMSEqT += α
inMSEqT α=
3/23/12 10 Lecture 20
More simply
• Calculate T, calculate the differences
• If < T, means are not significantly different
• If > T, means ARE significantly different • Remark: Tukey’s method is conservative,
sometimes its conclusion will be inconsistent with that by using ANOVA test results.
If we took a less stringent alpha level we might see some of the significant differences.
ji xx −
ji xx −
ji xx −
3/23/12 11 Lecture 20
Example—generic
• Let’s say we are comparing 4 means with equal sample sizes of ni = 5 for all i. With an MSE of 10.
• Looking at Table IX, we have k = 4, and Error df = n – k = 16 – qα = 4.05
• So,
• Any difference of means more than 5.73 apart would be significant different
73.551005.4 ===
inMSEqT α
3/23/12 12 Lecture 20
13
Example—generic
• Suppose that you have four treatment groups and the treatment means are:
• TRT 1: 52 • TRT 2: 63 • TRT 3: 58 • TRT 4: 54
• Which pairs are significantly different?
3/23/12 Lecture 20
Example (from Monday’s Class) • For the cereal example, let’s use Tukey’s method using α = 0.01
• The means are (arranged in descending order):
• Note, group 3’s sample size, what effect will that have on the comparisons?
4.13 ,56.14 ,55.19 ,42.27 ,5
22
11
33
44
==
==
==
==
xnxnxnxn
3/23/12 14 Lecture 20
Another Method: Using SAS code
proc glm data=cereal alpha=0.01; class design; model cases = design; means design / tukey cldiff; run;
3/23/12 15 Lecture 20
Example (cont) using SAS • Notice it stars the pairs
that are significantly different.
• So the only pairs that are
significantly different are: 1 and 4 2 and 4
3/23/12 16 Lecture 20
Alternate SAS code
proc glm data=cereal alpha=0.01;
class design; model cases = design; means design / tukey lines;
run;
3/23/12 17 Lecture 20
Example (cont) using SAS
• Same information as before, differences are:
1 and 4 2 and 4
• Notice the nice “groupings” though
3/23/12 18 Lecture 20
Dunnett’s Multiple Comparison
3/23/12 19 Lecture 20
3/23/12 Lecture 20 20
Multiple Comparison – Dunnett’s Method
3/23/12 Lecture 20 21
Dunnett’s Method • Functions like a Tukey, just uses a different T
• tα comes from the Dunnett’s t table – Table X on page 579 – Only use when one of the groups is a control group
– Only interested in comparing the “other” groups to the control group
• Again we take pairwise differences,
)11(),1(Ci nn
MSEknktT +−−= α
Ci xx −
3/23/12 22 Lecture 20
Example (Cereal Design) using SAS
• Dunnett’s SAS code, pretend design 1 was the regular design already used
proc glm data=cereal alpha=0.01; class design; model cases = design; means design / dunnett(“1”) cldiff; run;
Note: lines doesn’t work with Dunnett’s
3/23/12 23 Lecture 20
Example (cont) using SAS
Dunnett's t Tests for cases NOTE: This test controls the Type I experimentwise error for
comparisons of all treatments against a control. Alpha 0.01 Error Degrees of Freedom 15 Error Mean Square 10.54667 Critical Value of Dunnett's t 3.43026 Comparisons significant at the 0.01 level are indicated by ***. Difference design Between Simultaneous 99% Comparison Means Confidence Limits 4 - 1 12.600 5.554 19.646 *** 3 - 1 4.900 -2.573 12.373 2 - 1 -1.200 -8.246 5.846
• So if group1 was the control, only group 4 is significantly different
3/23/12 24 Lecture 20
Summary about ANOVA and Multiple Comparison
• ANOVA (Analysis of Variances) – Check the assumptions (constant variance/normality) – Be able to do most of ANOVA by hand or by SAS both
• Lots of hand calculations • Be able to read and interpret SAS output
– For Hw, do it either way you like, but for the exam be prepared to do both!
• Multiple Comparison methods (ONLY when ANOVA result is significant) – are useful in other situations, but they all involve calculating a T
value and using it to compare pairs of means – Tukey’s is approprirate if there’s no control group; try Dunnett’s
if there is any control(s)
3/23/12 25 Lecture 20
After Class
• Hw#8, due by 5pm next Monday
• Start review of Exam 2 (Ch.7, 8 and Monday’s complete notes) – Practice Test 2 – Hw5-8, Lab 3 and 4
3/23/12 Lecture 20 26