a nova sumner 2016
TRANSCRIPT
-
7/25/2019 A Nova Sumner 2016
1/23
ANOVA
Jeremy SumnerMaths and Physics, University of Tasmania
KMA711, June 2016
-
7/25/2019 A Nova Sumner 2016
2/23
Useful resources
http://rtutorialseries.blogspot.com.au/
http://www.r-tutor.com/elementary-statistics/analysis-variance
http://www.statmethods.net/stats/anova.html
http://www.r-bloggers.com/one-way-analysis-of-variance-anova/
http://www.stat.columbia.edu/martin/W2024/R3.pdf
-
7/25/2019 A Nova Sumner 2016
3/23
Time to coagulation vs. diet
-
7/25/2019 A Nova Sumner 2016
4/23
When is ANOVA a useful tool?
Categorical explanatory variable(s)DATA:
Continuousresponse variable
e.g. Time to blood coagulation (response variable) under different diets(explanatory variable). . .
Analysis goals: Is the response variable significantly affected bythe different levels (A,B,C,D) of the explanatory variable?
i.e. Does diet affect time to coagulation?
-
7/25/2019 A Nova Sumner 2016
5/23
ANOVA basics and assumptions
ANOVA is pretty much the same idea as regression exceptexplanatory variables are categorical/factors.
Coagulate TimeDiet exampleDATAis Time =yij, for Dieti=A, B, C, Dand observationj
NULL: different groups/levels for the categorical/factorvariables make no difference to the response.
ASSUMPTION:ANOVA assumes the response variable is normally dis-tributed with identical variance about the group means.
The idea is to compare the withingroups to the betweengroup variation.
Simple ANOVA model: yij=i+ ij, where errors areindependent and N(0, 2).
-
7/25/2019 A Nova Sumner 2016
6/23
ANOVA in a nutshell
Question: How can we statistically quantify what we see in
the box plots?Answer:
1 Use data to compute summary statistic F2 Assuming NULL (no effect), useF to compute a p-value
-
7/25/2019 A Nova Sumner 2016
7/23
What is a p-value again?
A p-value is the probability of observing the data (or
more extreme) if the NULL hypothesis is true
i.e. What is the probability of seeing box plots like this if dietmakes no difference?
-
7/25/2019 A Nova Sumner 2016
8/23
ANOVA basics
Time to coagulation by Diet
NULL: A = B= C= D
Groups normal about their mean
ALT: i= jBoxes: withingroup variation
Averages: betweengroup variation
Model: yij=i+ random
-
7/25/2019 A Nova Sumner 2016
9/23
ANOVA in a nutshell PART 2
withinvariation less than between variation means the
diet choice probably matters
this implies small p-value, so NULL hypothesis is probablyfalse
-
7/25/2019 A Nova Sumner 2016
10/23
Summary statistics
DATA assumptions
yij=i+ ij= + i+ ij
yij isjth sample fromith group
is the grand meani= +iare group means
ij N(0, 2)
Group Data Dist1 y11, y12, . . . , y1N1 N(1, 2)
2 y21, y22, . . . , y2N2 N(2, 2)
3 y31, y32, . . . , y3N3 N(3, 2)
......
...
k yk1, yk2, . . . , ykNk N(k, 2)
ANOVA summary statistics
Sample means: yi= 1Ni
Nij=1yij
Sample variances: s2i = 1Ni1
Nij=1(yij yi)2
Size Mean Variance
N1 y1 s2
1N2 y2 s
22
N3 y3 s23
......
...Nk yk s
2k
-
7/25/2019 A Nova Sumner 2016
11/23
ANOVA in a nutshell PART 2
withinvariation MSEshould roughly equal 2
between variation MS Groupsshould roughly equal 2
F statistic: F = MSGroupsMSE
= 1?
p-value is probability of observing F >1
-
7/25/2019 A Nova Sumner 2016
12/23
Some equations and the F-test
Uses2i to obtain a pooled estimate of2:
MSE = SSE
k 1= s
2p =
(N1 1)s21 + (N2 1)s22 +. . .
+ (Nk 1)s2ki(Ni 1)
=
ki=1
Nij=1
yij yi
2
N k=
total variation around group means
# data points # of means computed
Under NULL yi
N(, 2/Ni), so independentestimate of2:
MS Groups =SS Groups
k 1=
k
i=1
(yi y)2
k 1=
variation of group means about grand mean
# groups 1
Total variation: SST =
i,j(yij y)2
MIRACLE OF ANOVA: SST= SS Groups + SSE
Under NULL, the ratio F = MSGroupsMSE
should be F-distributed about 1with numerator DF (k 1) and denominator DF (N k).
SS Groups is what is explained by the separate group means and
SSE is what is left over.
-
7/25/2019 A Nova Sumner 2016
13/23
Simple ANOVA outputs
Generic tableSource df SS MS F
Between k 1 SS Groups = iNi(yi y)2 SS Groupsk1 F = MSGroupsMSEWithin N
k SSE = i(Ni
1) s2iSSENk
Total n 1 SST =i,j(yij y)2
R output for the Diet data
Df Sum Sq Mean Sq F value Pr(>F)Diet 3 228 76.0 13.57 4.66e-05 ***Residuals 20 112 5.6
Signif. codes: 0 *** 0.001 ** 0.01 * 0.05 . 0.1 1
-
7/25/2019 A Nova Sumner 2016
14/23
ANOVA as regression
Consider regression model on 4 diets:
time =0(diet A)+1(diet B)+2(diet C)+3(diet D)(diet A) is the indicator function: = 0 or 1
In R: 0 =A, 1 = (B A), 2 = (C A), 3 = (D A)
Regression style outputs
Coefficients:Estimate Std. Error t value Pr(>|t|)
(Intercept) 6.100e+01 1.183e+00 51.554 < 2e-16 ***DietB 5.000e+00 1.528e+00 3.273 0.003803 **DietC 7.000e+00 1.528e+00 4.583 0.000181 ***DietD -3.333e-15 1.449e+00 0.000 1.000000
Signif. codes: 0 *** 0.001 ** 0.01 * 0.05 . 0.1 1
-
7/25/2019 A Nova Sumner 2016
15/23
Plot checks for ANOVA assumptions
Visual checks for heteroscedascity, non-linearity, normality, and leverage:
-
7/25/2019 A Nova Sumner 2016
16/23
Test for ANOVA assumptions
Bartlett Test for equal variance across groups:
bartlett.test(Time
Diet,data=d)
Essentially simultaneously compares ratio of pooled varianceto variance for each group
This test is sensitive to departures from normality.
-
7/25/2019 A Nova Sumner 2016
17/23
Pairwise comparisions: A vs. B? A vs. C? B vs. D? . . .
Which diet is best A,B,C,or D?Imagine doing multiple pairwise comparisons i=jusing at-test at 95% = 100 (1 )% confidence levelType I error: Chance of rejecting null hypo when we shouldnthave, ie. = 0.05
We are guaranteed to stuff up 5% of the time!
Under multiple tests, we will eventually make a Type I error
Bonferroni: multiply p-value by # of tests k
Conservative. Chance of observing at least one ofkevents isless than the sum of probs for each event, ie.
-
7/25/2019 A Nova Sumner 2016
18/23
Contrast choice and data dredging
Standard ANOVA null hypothesis: 1=2=. . .= k.Contrasts: 1 2= 0 and 1 12 (2+ 3) = 0.Often the treatment structurewill suggest useful contrasts.
Snail tissue.
{LL,LH,HL,HH
}: humidity L/H, and temp L/H.
Contrast Comparison12 (LH+ HH) 12 (HL+ LL) Temperature matters12 (HL+ HH) 12 (LH+ LL) Humidity matters12 (LL+ HH) 12 (LH+ HL) Same matters
Be VERY careful if using data to suggest a contrast!In the lab we look at planned and un-plannedcomparisons.
-
7/25/2019 A Nova Sumner 2016
19/23
What to do when data is not normal?
Is it possible to apply a transformation to the responsevariable?
yij ory2ij or log(yij)?
Box-Cox :
y()ij =
(yij 1)/ = 0
log(yij) = 0
And worry a bit more to find best sensible using ML
Use when Bartlett test says unequal variances.
Welch s method: essentially a series of t-tests but doesntpool variances across groups
In R: pairwise.t.test(d$Time ,d$Diet,pool.sd=F),adjusted p-values
-
7/25/2019 A Nova Sumner 2016
20/23
What to do when data is not normal?
To the rescue! Non-parametric testsCareful! Valid for a wider range of distributions but lose power
Skewed data or extreme outliers?
Analysis of Medians:
Kruskal-Wallis : rank based, average rank for each group,variation in these rank-averages in analysed
Moods Median test: Contingency table, greater than grandmedian? less than? 2 test
Both assume groups have same shaped distribution, andKruskal-Wallis is more powerful than Mood
pairwise Wilcoxon: assumes roughly symmetric distribution,rank based, Holm adjusted p-values
-
7/25/2019 A Nova Sumner 2016
21/23
Why not always use a non-parametric test?
Its easier to reject NULL using tests with strong assumptions.
Type I error, : rejected NULL and shouldnt haveType II error, : kept NULL and should have rejected
power = (1 ), ie. power to reject.
Null hypothesis Weak assumptions Strong assumptions
(or model) (eg. non-parametric) (eg. Simple ANOVA)# parameters many (!?) few
p-values large small
power low high
fit good bad
Type I rate low high
Type II rate high low
Sample size large small
Bias low high
Variance high low
-
7/25/2019 A Nova Sumner 2016
22/23
Factorial ANOVA
Effect of sleeping tablets AND alcohol.
More than the sum of the parts= interaction
Model: ResponseSleep Tab + Alcohol + Sleep Tab:AlcoholFactorial design: All treatments in all combinations.
Eg. 5 people given nothing, 5 given sleep tabs, 5 givenalcohol, and 5 given sleep tabs & alcohol (ST:A).
In factorial ANOVA, the main effect is the effect of eachvariable separately, but now also have an interaction effect.
General model: yijk= + i+ j+ ij+ ijk with ijkindependent and N(0, 2).
ij is the interaction term.
-
7/25/2019 A Nova Sumner 2016
23/23
Take home messages
You should always double check that your data satisfies theassumptions of the method you are applying.
The more you can assume the better, as you can use a morepowerful test and hence reduce Type II error
For ANOVA there is a sequence of assumptions across groups:
normal with identical variance. . . normal without identicalvariance . . . not normal but same shape. . . completely nuts
The equal means null hypothesis is a good start, but if itsfalse we always want to know more this is where contrasts
come in.Multiple tests lead to increased chance of Type I error
Contrasts are great, but p-values must be corrected formultiple testing, AND dont use the data to suggest contrasts.