![Page 1: Multiple comparisons - multiple pairwise tests - orthogonal contrasts - independent tests - labelling conventions](https://reader030.vdocuments.us/reader030/viewer/2022032702/56649cae5503460f94972225/html5/thumbnails/1.jpg)
Multiple comparisons
- multiple pairwise tests
- orthogonal contrasts
- independent tests
- labelling conventions
![Page 2: Multiple comparisons - multiple pairwise tests - orthogonal contrasts - independent tests - labelling conventions](https://reader030.vdocuments.us/reader030/viewer/2022032702/56649cae5503460f94972225/html5/thumbnails/2.jpg)
Card example number 1
![Page 3: Multiple comparisons - multiple pairwise tests - orthogonal contrasts - independent tests - labelling conventions](https://reader030.vdocuments.us/reader030/viewer/2022032702/56649cae5503460f94972225/html5/thumbnails/3.jpg)
Multiple tests
Problem:
Because we examine the same data in multiple comparisons, the result of the first comparison affects our expectation of the next comparison.
![Page 4: Multiple comparisons - multiple pairwise tests - orthogonal contrasts - independent tests - labelling conventions](https://reader030.vdocuments.us/reader030/viewer/2022032702/56649cae5503460f94972225/html5/thumbnails/4.jpg)
Multiple tests
ANOVA shows at least one different, but which one(s)?
significant
Not significant
significant
•T-tests of all pairwise combinations
![Page 5: Multiple comparisons - multiple pairwise tests - orthogonal contrasts - independent tests - labelling conventions](https://reader030.vdocuments.us/reader030/viewer/2022032702/56649cae5503460f94972225/html5/thumbnails/5.jpg)
Multiple tests
T-test: <5% chance that this difference was a fluke…
affects likelihood of finding a difference in this pair!
![Page 6: Multiple comparisons - multiple pairwise tests - orthogonal contrasts - independent tests - labelling conventions](https://reader030.vdocuments.us/reader030/viewer/2022032702/56649cae5503460f94972225/html5/thumbnails/6.jpg)
Multiple tests
Solution:Make alpha your overall “experiment-wise” error rate
affects likelihood (alpha) of finding a difference in this pair!
T-test: <5% chance that this difference was a fluke…
![Page 7: Multiple comparisons - multiple pairwise tests - orthogonal contrasts - independent tests - labelling conventions](https://reader030.vdocuments.us/reader030/viewer/2022032702/56649cae5503460f94972225/html5/thumbnails/7.jpg)
Multiple tests
Solution:Make alpha your overall “experiment-wise” error rate
e.g. simple Bonferroni:Divide alpha by number of tests
Alpha / 3 = 0.0167
Alpha / 3 =0.0167
Alpha / 3 = 0.0167
![Page 8: Multiple comparisons - multiple pairwise tests - orthogonal contrasts - independent tests - labelling conventions](https://reader030.vdocuments.us/reader030/viewer/2022032702/56649cae5503460f94972225/html5/thumbnails/8.jpg)
Card example 2
![Page 9: Multiple comparisons - multiple pairwise tests - orthogonal contrasts - independent tests - labelling conventions](https://reader030.vdocuments.us/reader030/viewer/2022032702/56649cae5503460f94972225/html5/thumbnails/9.jpg)
Orthogonal contrastsOrthogonal = perpendicular = independent
Contrast = comparison
Example. We compare the growth of three types of plants: Legumes, graminoids, and asters.
These 2 contrasts are orthogonal:
1. Legumes vs. non-legumes (graminoids, asters) 2. Graminoids vs. asters
![Page 10: Multiple comparisons - multiple pairwise tests - orthogonal contrasts - independent tests - labelling conventions](https://reader030.vdocuments.us/reader030/viewer/2022032702/56649cae5503460f94972225/html5/thumbnails/10.jpg)
Trick for determining if contrasts are orthogonal:
1. In the first contrast, label all treatments in one group with “+” and all treatments in the other group with “-” (doesn’t matter which way round).
Legumes Graminoids Asters + - -
![Page 11: Multiple comparisons - multiple pairwise tests - orthogonal contrasts - independent tests - labelling conventions](https://reader030.vdocuments.us/reader030/viewer/2022032702/56649cae5503460f94972225/html5/thumbnails/11.jpg)
Trick for determining if contrasts are orthogonal:
1. In the first contrast, label all treatments in one group with “+” and all treatments in the other group with “-” (doesn’t matter which way round).
2. In each group composed of t treatments, put the number 1/t as the coefficient. If treatment not in contrast, give it the value “0”.
Legumes Graminoids Asters +1 - 1/2 -1/2
![Page 12: Multiple comparisons - multiple pairwise tests - orthogonal contrasts - independent tests - labelling conventions](https://reader030.vdocuments.us/reader030/viewer/2022032702/56649cae5503460f94972225/html5/thumbnails/12.jpg)
Trick for determining if contrasts are orthogonal:
1. In the first contrast, label all treatments in one group with “+” and all treatments in the other group with “-” (doesn’t matter which way round).
2. In each group composed of t treatments, put the number 1/t as the coefficient. If treatment not in contrast, give it the value “0”.
3. Repeat for all other contrasts.
Legumes Graminoids Asters +1 - 1/2 -1/2 0 +1 -1
![Page 13: Multiple comparisons - multiple pairwise tests - orthogonal contrasts - independent tests - labelling conventions](https://reader030.vdocuments.us/reader030/viewer/2022032702/56649cae5503460f94972225/html5/thumbnails/13.jpg)
Trick for determining if contrasts are orthogonal:
4. Multiply each column, then sum these products.
Legumes Graminoids Asters +1 - 1/2 -1/2 0 +1 -1
0 - 1/2 +1/2
Sum of products = 0
![Page 14: Multiple comparisons - multiple pairwise tests - orthogonal contrasts - independent tests - labelling conventions](https://reader030.vdocuments.us/reader030/viewer/2022032702/56649cae5503460f94972225/html5/thumbnails/14.jpg)
Trick for determining if contrasts are orthogonal:
4. Multiply each column, then sum these products.
5. If this sum = 0 then the contrasts were orthogonal!
Legumes Graminoids Asters +1 - 1/2 -1/2 0 +1 -1
0 - 1/2 +1/2
Sum of products = 0
![Page 15: Multiple comparisons - multiple pairwise tests - orthogonal contrasts - independent tests - labelling conventions](https://reader030.vdocuments.us/reader030/viewer/2022032702/56649cae5503460f94972225/html5/thumbnails/15.jpg)
What about these contrasts?
1. Monocots (graminoids) vs. dicots (legumes and asters).
2. Legumes vs. non-legumes
![Page 16: Multiple comparisons - multiple pairwise tests - orthogonal contrasts - independent tests - labelling conventions](https://reader030.vdocuments.us/reader030/viewer/2022032702/56649cae5503460f94972225/html5/thumbnails/16.jpg)
Important!
You need to assess orthogonality in each pairwise combination of contrasts.
So if 4 contrasts:
Contrast 1 and 2, 1 and 3, 1 and 4, 2 and 3, 2 and 4, 3 and 4.
![Page 17: Multiple comparisons - multiple pairwise tests - orthogonal contrasts - independent tests - labelling conventions](https://reader030.vdocuments.us/reader030/viewer/2022032702/56649cae5503460f94972225/html5/thumbnails/17.jpg)
How do you program contrasts in JMP (etc.)?
Treatment SS
}Contrast 2
}Contrast 1
![Page 18: Multiple comparisons - multiple pairwise tests - orthogonal contrasts - independent tests - labelling conventions](https://reader030.vdocuments.us/reader030/viewer/2022032702/56649cae5503460f94972225/html5/thumbnails/18.jpg)
How do you program contrasts in JMP (etc.)?
Normal treatments
Legume 1 1Legume 1 1Graminoid 2 2Graminoid 2 2Aster 3 2Aster 3 2
SStreat 122 67Df treat 2 1MStreat 60
MSerror 10Df error 20
Legumesvs. non-legumes “There was a significant
treatment effect (F…). About 53% of the variation between treatments was due to differences between legumes and non-legumes (F1,20 = 6.7).”
F1,20 = (67)/1 = 6.7 10
From full model!
![Page 19: Multiple comparisons - multiple pairwise tests - orthogonal contrasts - independent tests - labelling conventions](https://reader030.vdocuments.us/reader030/viewer/2022032702/56649cae5503460f94972225/html5/thumbnails/19.jpg)
Even different statistical tests may not be independent !
Example. We examined effects of fertilizer on growth of dandelions in a pasture using an ANOVA. We then repeated the test for growth of grass in the same plots.
Problem?
![Page 20: Multiple comparisons - multiple pairwise tests - orthogonal contrasts - independent tests - labelling conventions](https://reader030.vdocuments.us/reader030/viewer/2022032702/56649cae5503460f94972225/html5/thumbnails/20.jpg)
Multiple tests
Not significantsignificant
Not significant
a a,b
bConvention:Treatments with a common letter are not significantly different
![Page 21: Multiple comparisons - multiple pairwise tests - orthogonal contrasts - independent tests - labelling conventions](https://reader030.vdocuments.us/reader030/viewer/2022032702/56649cae5503460f94972225/html5/thumbnails/21.jpg)
![Page 22: Multiple comparisons - multiple pairwise tests - orthogonal contrasts - independent tests - labelling conventions](https://reader030.vdocuments.us/reader030/viewer/2022032702/56649cae5503460f94972225/html5/thumbnails/22.jpg)
0
5
10
15
20
25
30
35
Farm fa
llow
Ebouf
ek co
mplete
Ebogo
comple
te
Ebogo
mec
hanic
al
Ebogo
man
ual
Ebouf
ek un
clear
ed
Ebogo
uncle
ared
Bilik u
nclea
red
Mea
n no
. but
terf
lies/
5 m
in.w
alk
a a
abbc bc
dd
cd