1
Handout Eight: Two-Way Between-Subjects Design
with Interaction- Assumptions, & Analyses
EPSE 592Experimental Designs and Analysis in
Educational ResearchInstructor: Dr. Amery Wu
Today, we will introduce the analysis and assumptions of a two-way between-Subjects designs.This design explains/predicts continuous data using two independent variables, each with two or more levels (i.e., groups) where subjects are independent across the groups.
Where We Are TodayTwo-way Between-Subjects Factorial Design
with InteractionMeasurement of Data
Continuous Categorical
Type ofthe
Inference
DescriptiveA B
InferentialC D
2
3
The Design A two-way between-subjects ANOVA is an
inferential statistical method includes two factors (independent variables), each factor has two or more levels (groups or
categories) subjects are independent across the levels for each
factor.Data Assumptions
Data assumptions are the same as those of one-way ANOVA, except that assumptions are made about the RxC sub-samples.
R is the # of levels of the first factor the C is the # of levels of the second factor.
Review of Two-way Between-Subjects Factorial Design
4
Two-way Between-Subjects Factorial DesignPartitioning the Total Sum of Squares by the 2
Factors
SSw-g
SSb-gA
SSb-gBSSb-gA*B
2 Factors
5
Statistical Model of Two-way Factorial ANOVAIn a two-way ANOVA, two independent variables, A and B,are involved, which are termed “factors” in ANOVAHowever, the data (DV) is being modeled (explained orpredicted) by three terms: A, B, and their interaction effect A*BData = Model + residual
Y = intercept + aA + bB + cA*B + residual
Questions: What is the “model” for a one-way ANOVA? Write the equation.
6
Variable A is treated as the primary cause, of which the effect on the Y (DV) is the major research interest. An analogy is the effect of a switch on the light.
Variable B is interpreted as a moderator that adjusts the effect of A on Y. An analogy is a dimmer adjusts the effect of the switch on the light. Namely, the effect of A on Y is dependent on the level (or amount) of B
The choice of which is the primary cause should be guided by the research question and the substantive theory. Interaction Effect
Interpreting the Results- Randomized Experimental Data
Two-way Between-Subjects Factorial ANOVA
7
The relationship between A and Y (DV) is dependent on the level (or amount) of B.
Variable B moderates the relationship between A and Y.
The choice of whether A or B should be considered as the moderator depends on the research question and the substantive theory.
Interpreting the Results - Observational Data
Two-way Between-Subjects Factorial ANOVA
Interaction Relationship
Question: what are the differences between this figure and the one on the last slide?
8
One Way ANOVA (One Independent Variable A)SStot = SSbgA + SSwg, or simplySStot = SSA + SSr
Two Way ANOVA (Two independent variables A & B)SStot = SSbgA + SSbgB + SsbgA*bgB + SSwg or simplySStot = SSA + SSB + SSAB+ SSr
Comparing One-Way and Two-Way Factorial ANOVAUsing Data = Model + Residual
Interpreting a Two-way Factorial ANOVA-- When the Interaction Effect is Non-significant
ANOVA TableSS df MS F p
Instruction 35 2 17.50 9.55 <0.05Sex 7.5 1 7.50 4.09 >0.05Instruction*Sex 5 2 2.50 1.36 >0.05Error (within) 44 24 1.83Total 91.5 29
When the interaction effect is not significant, run the post hoc paired-comparisons for the main effect(s) that are significantIf both main effects are significant; hence, more multiplepaired comparisons are involved, a more conservative post-hoc t-test should be considered to protect from Type-I error. (see the Word document: GLM post hoc comparison.doc)
Instruction by Gender
4
5
6
7
8
9
10
A B C
Instruction Method
Perfo
rman
ceFemale
Male
An example Plot for No Interaction Effect
When the two separate lines (here representing the moderator) are parallel, it signals that there is no interaction effect.However, one must NOT use such a plot to infer about thepopulation interaction effect as many textbooks suggest because it is descriptive and does not test whether the parallelism occurred only by chance.
Lab Activity: Post-hoc Comparison for Main effects
SPSS Output: Post-hoc Comparison for Main effects
Multiple Comparisons
Dependent Variable: performanceTukey HSD
-2.00* .606 .008 -3.51 -.49-2.50* .606 .001 -4.01 -.992.00* .606 .008 .49 3.51-.50 .606 .691 -2.01 1.01
2.50* .606 .001 .99 4.01.50 .606 .691 -1.01 2.01
(J) instruction231312
(I) instruction1
2
3
MeanDifference
(I-J) Std. Error Sig. Lower Bound Upper Bound95% Confidence Interval
Based on observed means.The mean difference is significant at the .05 level.*.
MeansInstruction Method
A B C Marsex
Sex Male 6 7 8 7Female 6 9 9 8
Marinstruction 6 8 8.5 7.5
Number of SubjectInstruction Method
A B C Marsex
Sex Male 15 15 15 45Female 15 15 15 45
Marinstruction 30 30 30 90
Using the same research context (IV: instruction and gender;DV: performance), all the means of the main effect and interaction cells are the same as before. However, number of subjects increased from 5 to 15 for each cell.
14
Students’ performance was rated by their clinical supervisors on a scale of 0-10. Mean ratings of the
groupsare tabulated below.
An Example to Contextualize Learning
Two-way Between-Subjects Factorial ANOVA
Research Questions:1.Does the instruction method have an effect on
students’ performance?2.Is there a sex difference in students’
performance?3.Does the effect of instruction method on the
students’ performance depend on the sex of the students?
Means Instruction Method A B C Marsex
Sex Male 6 7 8 7 Female 6 9 9 8
Marinstruction 6 8 8.5 7.5
15
Computing The F Statistic and the F TestTwo-way Between-Subjects Factorial ANOVA
Lab Activity: See Excel File “Two Way ANOVA with Interaction.xls.”
16
Calculating the F Statistic - SS of Total
Calculating Total Sum of Squares
Subject ID Sex Instruction performance Deviation2
1 female 1 4 12.25
2 female 1 5 6.25
3 female 1 6 2.25
4 female 1 7 0.25⁞ ⁞ ⁞ ⁞ ⁞
87 male 3 7 0.25
88 male 3 8 0.25
89 male 3 9 2.25
90 male 3 10 6.25
SS Total 274.5
df Total N-1 89
17
Calculating the F Statistic - SS for the Main effects of Instruction Method
Means Instruction Method A B C Marsex
Sex Male 6 7 8 7 Female 6 9 9 8
Marinstruction 6 8 8.5 7.5
SS instruction 30*(6-7.5)(6-7.5)+30*(8-7.5)(8-7.5)+30*(8.5-7.5)(8.5-7.5) 105df = C-1 2
18
Calculating the F Statistic - SS for the Main effects of Sex
Means Instruction Method A B C Marsex
Sex Male 6 7 8 7 Female 6 9 9 8
Marinstruction 6 8 8.5 7.5
SS Sex 45*(7-7.5)*(7-7.5)+45*(8-7.5)*(8-7.5) 22.5df = R-1 1
19
Testing the interaction effect is, in essence, testing whether there is a true difference in the difference.
Calculating the F Statistic- SS for “instruction by sex” interaction effect
Instruction Method A B C
Sex Male 6 7 8 Female 6 9 9
sex difference 0 -2 -1
Sex diffInstruction A 0Instruction B -2Instruction C -1Mean -1
SS difference=15*(0-(-1))*(0-(-1))+15*(-2-(-1))*(-2-(-1))+15*(-1-(-1))*(-1-(-1)) 30SS interaction = Half SS difference 15df = (R-1)*(C-1) 2
20
Calculating the F Statistic- SS for within Groups
Calculating SS Within Groups
Subject ID Sex Instruction Performance Deviation2
1 female 1 4 42 female 1 5 13 female 1 6 04 female 1 7 15 female 1 8 46 female 1 4 47 female 1 5 18 female 1 6 09 female 1 7 110 female 1 8 411 female 1 4 412 female 1 5 113 female 1 6 014 female 1 7 115 female 1 8 4
mf1 6 30 SSf1
16 male 1 4 417 male 1 5 118 male 1 6 019 male 1 7 120 male 1 8 421 male 1 4 422 male 1 5 123 male 1 6 024 male 1 7 125 male 1 8 426 male 1 4 427 male 1 5 128 male 1 6 029 male 1 7 130 male 1 8 4
mm1 6 30 mm1
31 female 2 8 132 female 2 9 033 female 2 9 034 female 2 9 035 female 2 10 136 female 2 8 137 female 2 9 038 female 2 9 039 female 2 9 040 female 2 10 141 female 2 8 142 female 2 9 043 female 2 9 044 female 2 9 045 female 2 10 1
mf2 9 6 SSf2
46 male 2 5 447 male 2 6 148 male 2 7 049 male 2 8 150 male 2 9 451 male 2 5 452 male 2 6 153 male 2 7 054 male 2 8 155 male 2 9 456 male 2 5 457 male 2 6 158 male 2 7 059 male 2 8 160 male 2 9 4
mm2 7 30 SSm2
61 female 3 8 162 female 3 9 063 female 3 9 064 female 3 9 065 female 3 10 166 female 3 8 167 female 3 9 068 female 3 9 069 female 3 9 070 female 3 10 171 female 3 8 172 female 3 9 073 female 3 9 074 female 3 9 075 female 3 10 1
mf3 9 6 SSf3
76 male 3 6 477 male 3 7 178 male 3 8 079 male 3 9 180 male 3 10 481 male 3 6 482 male 3 7 183 male 3 8 084 male 3 9 185 male 3 10 486 male 3 6 487 male 3 7 188 male 3 8 089 male 3 9 190 male 3 10 4
mm3 8 30 SSm3
SS Within 132
dfTotal N- # of cell groups 84
21
The ANOVA Summary TableTwo-way Between-Subjects Factorial ANOVA
ANOVA Summary F Table SS df MS F p
Instruction 105.0 2 52.50 33.41 <0.001Sex 22.5 1 22.50 14.32 <0.001Instruction*Sex 15.0 2 7.50 4.77 0.011Error (within) 132.0 84 1.57
Total 274.5 29
22
Lab Activity: Two-Way Between-Subjects Factorial ANOVA in SPSS
23
Lab Activity: SPSS Outputs for Two-Way Between-Subjects Factorial ANOVA in SPSS
Tests of Between-Subjects Effects
Dependent Variable: performance
142.500a 5 28.500 18.136 .0005062.500 1 5062.500 3221.591 .000
105.000 2 52.500 33.409 .00022.500 1 22.500 14.318 .00015.000 2 7.500 4.773 .011
132.000 84 1.5715337.000 90
274.500 89
SourceCorrected ModelInterceptinstructionsexinstruction * sexErrorTotalCorrected Total
Type III Sumof Squares df Mean Square F Sig.
R Squared = .519 (Adjusted R Squared = .491)a.
Interpreting a Two-way Factorial ANOVA-- When the Interaction Effect is SignificantIncreasing the sample size increased the power, the interaction effect is now significant even though all the cell means were the same.
Interpreting a Two-way Factorial ANOVA- When the Interaction Effect is Significant
When SPSS “General Linear Model” (GLM) menu is used: If the interaction effect is significant, interpret the
interaction effect ONLY (even if the main effects are significant).
One should then conduct the post hoc paired comparisons of the cell mean differences - simple main effects. The simple main effect is the mean difference between the groups of factor B (i.e., sex) for each given group of factor A (i.e., instruction method).
Using SPSS GLM, interpreting the main effects is unnecessary because, the main effects are included in the estimate of cell means. Test of the simple main effects actually have incorporated the test of main effects.
Lab Activity: SPSS Syntax for Testing Simple Main Effects
UNIANOVA performance BY instruction sex /METHOD = SSTYPE(3) /INTERCEPT = INCLUDE /CRITERIA = ALPHA(.05) /DESIGN = instruction sex instruction*sex /EMMEANS = TABLES (instruction*sex) compare (sex) adj (Bonferroni).
Note that for post hoc simple main effect t-tests, only the methods of BONFERRONI, LSD, SIDAK tests are available in SPSS.
SPSS drop-down menu dose not provide function of testing simple main effects. Instead, you should use the following syntax of EMMEANS to test them.
For females, there was significant difference between instruction methods A and B, methods A and C (p= 0.005). However there is no difference between methods B and C, p= 1.000.
For males, there were not significant differences between any instruction methods.
Lab Activity: SPSS Output for Testing Simple Main Effects
Protecting from Type- I error
Primary IV
Moderator IV
Remember to hit “Add”
Lab Activity: Plotting Interaction Effect in SPSS
Lab Activity: SPSS Plot for Interaction Effect
321
instruction
9
8
7
6Est
imat
ed M
argi
nal M
eans
malefemale
sex
Estimated Marginal Means of performance
The difference between female and male students’ performance is different for each instruction method group. Namely, the gender difference in performance is dependent on (moderated by) which instruction method they receive.
The effect of instruction method on students’ performance is different for males and females. Namely, the effect of the instruction method is dependent on ( moderated by) sex.
30
Interpreting Models with More Than Three Factors
In general, for any ANOVA models, if the higher-order interaction effect is statistically significant, we do not interpret the lower-order effects directly, including the main effects or the interaction effects.
Lower-order effects will be interpreted in the context of the higher level interaction effect.
For example, if the 3-way interaction of an ANOVA including three factors is statistically significant, the 2-way interaction effects are interpreted in the context of the 3-way interaction. If a 2-way interaction effect is significant, its corresponding one-way main effects are interpreted in the context of the 2-way interaction effect.
31
Observed Effect SizeEta-Squared vs. Partial Eta-Squared
Eta-squared and partial eta-squared are effect size measures for the effect/difference of the factors included in a ANOVA.
Lab Activity:Calculate the eta-squared and partial-eta squared using the ANOVA table in the Excel file: Two Way ANOVA with Interaction.xls.Questions:Is there a difference between eta-squared and partial eta-squared for one-way ANOVA?Will partial eta-squared always be smaller or greater than the eta-squared?
Two common used effect size measures for ANOVA:h2 (eta-squared) = SSeffect / SStotalPartial h2 = SSeffect / (SSeffect + SSerror) or
= SSeffect / (Sstotal – SSother effects)
Given that the numerators of the two effect size measures are the same, Partial h2 will yield a bigger effect size than does the h2 because the denominator for partial h2 is smaller (by removing the SS other than the given effect.)
• However, h2 is conceptually easier to interpret; indicating the percentage of the total sum of square that is attributable to a given effect. Also, h2 can be used to compare the relative effects because all the effects have a common denominator.
Observed Effect SizeEta-Squared vs. Partial Eta-
Squared
Lab Activity: Effect Sizes for Two-way ANOVA in SPSS
Tests of Between-Subjects Effects
Dependent Variable: performance
142.500a 5 28.500 18.136 .000 .5195062.500 1 5062.500 3221.591 .000 .975
105.000 2 52.500 33.409 .000 .44322.500 1 22.500 14.318 .000 .14615.000 2 7.500 4.773 .011 .102
132.000 84 1.5715337.000 90
274.500 89
SourceCorrected ModelInterceptinstructionsexinstruction * sexErrorTotalCorrected Total
Type III Sumof Squares df Mean Square F Sig.
Partial EtaSquared
R Squared = .519 (Adjusted R Squared = .491)a.