manova - analysis
TRANSCRIPT
-
7/29/2019 MANOVA - Analysis
1/33
MANOVA: MultivariateAnalysis of Variance
-
7/29/2019 MANOVA - Analysis
2/33
Review of ANOVA: UnivariateAnalysis of Variance
An univariate analysis of variance looks for the causalimpact of a nominal level independent variable (factor) on asingle, interval or better level dependent variable
The basic question you seek to answer is whether or notthere is a difference in scores on the dependent variable
attributable to membership in one or the other category ofthe independent variable Analysis of Variance (ANOVA): Required when there are three
or more levels or conditions of the independent variable (butcan be done when there are only two) What is the impact of ethnicity (IV) (Hispanic, African-
American, Asian-Pacific Islander, Caucasian, etc) on annualsalary (DV)?
What is the impact of three different methods of meeting apotential mate (IV) (online dating service; speed dating;setup by friends) on likelihood of a second date (DV)
-
7/29/2019 MANOVA - Analysis
3/33
Basic Analysis of Variance Concepts
We are going to make two estimates of the common populationvariance, 2
The first estimate of the common variance 2 is called the between(or among) estimate and it involves the variance of the IV categorymeans about the grand mean
The second is called the within estimate, which will be a weighted
average of the variances within each of the IV categories. This is anunbiased estimate of2
The ANOVA test, called the Ftest, involves comparing the betweenestimate to the within estimate
If the null hypothesis, that the population means on the DV for thelevels of the IV are equal to one another, is true, then the ratio ofthe between to the within estimate of2 should be equal to one(that is, the between and within estimates should be the same)
If the null hypothesis is false, and the population means are notequal, then the Fratio will be significantly greater than unity(one).
-
7/29/2019 MANOVA - Analysis
4/33
Basic ANOVA Output
Tests of Between-Subjects Effects
Dependent Variable: Respondent Socioeconomic Index
29791.484b 4 7447.871 22.332 .000 .072 89.329 1.000
1006433.085 1 1006433.085 3017.774 .000 .724 3017.774 1.000
29791.484 4 7447.871 22.332 .000 .072 89.329 1.000
382860.051 1148 333.5023073446.860 1153
412651.535 1152
Source
Corrected Model
Intercept
PADEG
ErrorTotal
Corrected Tota l
Type III Sum
of Squares df Mean Square F Sig.
Partial Eta
Squared
Noncent.
Parameter
Observed
Powera
Computed using alpha = .05a.
R Squared = .072 (Adjusted R Squared = .069)b.
Some of the things that we learned to look for on the ANOVA output:A. The value of the F ratio (same line as the IV or factor)
B. The significance of that F ratio (same line)C. The partial eta squared (an estimate of the amount of the effectsize attributable to between-group differences (differences in levels ofthe IV (ranges from 0 to 1 where 1 is strongest)D. The power to detect the effect (ranges from 0 to 1 where 1 isstrongest)
The IV,fathershighestdegree
A B C D
-
7/29/2019 MANOVA - Analysis
5/33
More Review of ANOVA Even if we have obtained a significant value ofFand the overall
difference of means is significant, the Fstatistic isnt telling usanything about how the mean scores varied among the levels ofthe IV.
We can do some pairwisetests after the fact in which we comparethe means of the levels of the IV
The type of test we do depends on whether or not the variances ofthe groups (conditions or levels of the IV) are equal
We test this using the Levene statistic. If it is significant at p < .05 (group variances are significantly different)
we use an alternative post-hoc test like Tamhane If it is not significant (groups variances are not significantly different)
we can use the Sheff or similar test
In this example, variances are not significantly different (p > .05) sowe use the Sheff testTest of Homogeneity of Variances
Self-disclosure
.000 2 9 1.000
Levene
Statistic df1 df2 Sig.
-
7/29/2019 MANOVA - Analysis
6/33
Review of Factorial ANOVA Two-way ANOVA is applied to a situation in which you
have two independent nominal-level variables and oneinterval or better dependent variable
Each of the independent variables may have any numberof levels or conditions (e.g., Treatment 1, Treatment 2,
Treatment 3 No Treatment) In a two-way ANOVA you will obtain 3 F ratios
One of these will tell you if your first independentvariable has a significant main effecton the DV
A second will tell you if your second independentvariable has a significant main effecton the DV
The third will tell you if the interaction of the twoindependent variables has a significant effect on theDV, that is, if the impact of one IV depends on thelevel of the other
-
7/29/2019 MANOVA - Analysis
7/33
Review: Factorial ANOVAExample
Tests of Hypotheses:
(1) There is no significant main effect for education level (F(2, 58) = 1.685, p =.194, partial eta squared = .055) (red dots)
(2) There is no significant main effect for marital status (F(1, 58) = .441, p =.509, partial eta squared = .008)(green dots)
(3) There is a significant interaction effect of marital status and education level (F(2, 58) = 3.586, p = .034, partial eta squared = .110) (blue dots)
-
7/29/2019 MANOVA - Analysis
8/33
Plots of Interaction Effects
Estimated Marginal Means of TIMENET
CollegeorNot
CollegeorMoreSomePostHighHighSchool
9
8
7
6
5
4
3
2
MarriedorNot
Married/Partner
NotMarried/Partner
Education Level is plottedalong the horizontal axis andhours spent on the net isplotted along the verticalaxis. The red and green
lines show how maritalstatus interacts witheducation level. Here wenote that spending time onthe Internet is strongestamong the Post High Schoolgroup for single people, butlowest among this group formarried people
-
7/29/2019 MANOVA - Analysis
9/33
MANOVA: What Kinds ofHypotheses Can it Test?
A MANOVA or multivariate analysis of variance is away to test the hypothesis that one or moreindependent variables, or factors, have an effect on aset of two or more dependent variables For example, you might wish to test the hypothesis
that sex and ethnicity interact to influence a set ofjob-related outcomes including attitudes toward co-workers, attitudes toward supervisors, feelings ofbelonging in the work environment, and identificationwith the corporate culture
As another example, you might want to test the
hypothesis that three different methods of teachingwriting result in significant differences in ratings ofstudent creativity, student acquisition of grammar,and assessments of writing quality by an independentpanel of judges
-
7/29/2019 MANOVA - Analysis
10/33
Why Should You Do a MANOVA?
You do a MANOVA instead of a series of one-at-a-timeANOVAs for two main reasons Supposedly to reduce the experiment-wise level of Type I
error (8 Ftests at .05 each means the experiment-wiseprobability of making a Type I error (rejecting the nullhypothesis when it is in fact true) is 40%! The so-calledoverall test or omnibus test protects against this inflatederror probability only when the null hypothesis is true. Ifyou follow up a significant multivariate test with a bunch ofANOVAs on the individual variables without adjusting theerror rates for the individual tests, theres no protection
Another reasons to do MANOVA. None of the individualANOVAs may produce a significant main effect on the DV,but in combination they might, which suggests that thevariables are more meaningful taken together thanconsidered separately
MANOVA takes into account the intercorrelations amongthe DVs
-
7/29/2019 MANOVA - Analysis
11/33
Assumptions of MANOVA 1. Multivariate normality
All of the DVs must be distributed normally (can visualizethis with histograms; tests are available for checking thisout)
Any linear combination of the DVs must be distributed
normally Check out pairwise relationships among the DVs fornonlinear relationships using scatter plots
All subsets of the variables must have a multivariatenormal distribution These requirements are rarely if ever tested in practice MANOVA is assumed to be a robust test that can stand up to
departures from multivariate normality in terms of Type I errorrate Statistical power (power to detect a main or interaction effect)
may be reduced when distributions are very plateau-like(platykurtic)
-
7/29/2019 MANOVA - Analysis
12/33
Assumptions of MANOVA, contd 2. Homogeneity of the covariance matrices
In ANOVA we talked about the need for the variances ofthe dependent variable to be equal across levels of theindependent variable In MANOVA, the univariate requirement of equal variances
has to hold for each one of the dependent variables
In MANOVA we extend this concept and require that thecovariance matrices be homogeneous Computations in MANOVA require the use of matrix
algebra, and each persons score on the dependentvariables is actually a vector of scores on DV1, DV2, DV3,. DVn
The matrices of the covariances-the variance sharedbetween any two variables-have to be equal across alllevels of the independent variable
-
7/29/2019 MANOVA - Analysis
13/33
Assumptions of MANOVA, contd
This homogeneity assumption is tested with a test that is similar toLevenes test for the ANOVA case. It is called Boxs M, and itworks the same way: it tests the hypothesis that the covariancematrices of the dependent variables are significantly differentacross levels of the independent variable Putting this in English, what you dont want is the case where if
your IV, was, for example, ethnicity, all the people in the other
category had scores on their 6 dependent variables clustered verytightly around their mean, whereas people in the white categoryhad scores on the vector of 6 dependent variables clustered veryloosely around the mean. You dont want a leptokurtic set ofdistributions for one level of the IV and a platykurtic set foranother level
If Boxs Mis significant, it means you have violated an assumptionof MANOVA. This is not much of a problem if you have equal cell
sizes and large N; it is a much bigger issue with small samplesizes and/or unequal cell sizes (in factorial anova if there areunequal cell sizes the sums of squares for the three sources (twomain effects and interaction effect) wont add up to the Total SS)
-
7/29/2019 MANOVA - Analysis
14/33
Assumptions of MANOVA, contd
3. Independence of observations Subjects scores on the dependent measures should not be
influenced by or related to scores of other subjects in thecondition or level
Can be tested with an intraclass correlation coefficient iflack of independence of observations is suspected
-
7/29/2019 MANOVA - Analysis
15/33
MANOVA Example Lets test the hypothesis that region of the
country (IV) has a significant impact onthree DVs, Percent of people who areChristian adherents, Divorces per 1000population, and Abortions per 1000populations. The hypothesis is that there
will be a significant multivariate main effectfor region. Another way to put this is thatthe vectors of means for the three DVs aredifferent among regions of the country
This is done with the General Linear Model/Multivariate procedure in SPSS (we will lookfirst at an example where the analysis hasalready been done)
Computations are done using matrix algebrato find the ratio of the variability ofB(Between-Groups sums of squares andcross-products (SSCP) matrix) to that of theW (Within-Groups SSCP matrix)
MY1
MY2
My3
Vectors of meanson the three DVs(Y1, Y2, Y3) forRegions South andMidwest
MY1
MY2
My3
South Midwest
-
7/29/2019 MANOVA - Analysis
16/33
MANOVA test of Our Hypothesis
First we will look at the overall Ftest (over all three dependent variables). What weare most interested in is a statistic called Wilks lambda (), and the Fvalueassociated with that. Lambda is a measure of the percent of variance in the DVsthat is *not explained* by differences in the level of the independent variable.Lambda varies between 1 and zero, and we want it to be near zero (e.g, no variancethat is not explained by the IV). In the case of our IV, REGION, Wilks lambda is.465, and has an associated Fof 3.90, which is significant at p.
-
7/29/2019 MANOVA - Analysis
17/33
MANOVA Test of our Hypothesis,contd
Continuing to examine our
output, we find that the partialeta squared associated with themain effect of region is .225 andthe power to detect the maineffect is .964. These are verygood results!
We would write this up in the following way:
A one-way MANOVA revealed a significantmultivariate main effect for region, Wilks = .465, F(9, 95.066) = 3.9, p
-
7/29/2019 MANOVA - Analysis
18/33
Boxs Test of Equality of CovarianceMatrices
Box's Test of Equality of Covariance Matricesa
60.311
2.881
18
4805.078
.000
Box's M
F
df1
df2
Sig.
Tests the null hypothesis that the observed covariance
matrices of the dependent variables are equal across groups.
Design: Intercept+REGIONa.
Checking out the Boxs Mtest we find that the test is significant (which means that there aresignificant differences among the regions in the covariance matrices). If we had low power
that might be a problem, but we dont have low power. However, when Boxs test finds thatthe covariance matrices are significantly different across levels of the IV that may indicate anincreased possibility of Type I error, so you might want to make a smaller error region. If youredid the analysis with a confidence level of .001, you would still get a significant result, soits probably OK. You should report the results of the Boxs M, though.
-
7/29/2019 MANOVA - Analysis
19/33
Looking at the IndividualDependent Variables
If the overall Ftest is significant, then its commonpractice to go ahead and look at the individualdependent variables with separate ANOVA tests The experimentwise alpha protection provided by the
overall or omnibus Ftest does not extend to the
univariate tests. You should divide your confidencelevels by the number of tests you intend to perform,so in this case if you expect to look at Ftests for thethree dependent variables you should require that p< .017 (.05/3)
This procedure ignores the fact the variables may be
intercorrelated and that the separate ANOVAS do nottake these intercorrelations into account You could get three significant Fratios but if the
variables are highly correlated youre basicallygetting the same result over and over
-
7/29/2019 MANOVA - Analysis
20/33
Univariate ANOVA tests ofThree Dependent Variables
Above is a portion of the output table reporting the ANOVA tests on the three
dependent variables, abortions per 1000, divorces per 1000, and % Christianadherents. Note that only the Fvalues for %Christian adherents and Divorces per1000 population are significant at your criterion of .017. (Note: the MANOVAprocedure doesnt seem to let you set differentp levels for the overall test and theunivariate tests, so the power here is higher than it would be if you did these testsseparately in a ANOVA procedure and setp to .017 before you did the tests.)
-
7/29/2019 MANOVA - Analysis
21/33
Writing up More of Your Results
So far you have written the following: A one-way MANOVA revealed a significant multivariate
main effect for region, Wilks = .465, F(9, 95.066) =3.9, p
-
7/29/2019 MANOVA - Analysis
22/33
Finally, Post-hoc Comparisons with Sheff Testfor the DVs that had Significant UnivariateANOVAs
Leve ne's Test of Equality of Error Variancesa
1.068 3 41 .373
1.015 3 41 .396
1.641 3 41 .195
Aborti ons per 1,00 0
women
Percent of pop who are
Christian a dherents
Divorces per 1,000 pop
F df1 df2 Sig.
Tests the null hypothesis that the error varian ce of the dependent variable
equal across groups.Design: Intercept+REGIONa.
The Levenes statistics for the two DVs that had significantunivariate ANOVAs are all non-significant, meaning thatthe group variances were equal, so you can use the Shefftests for comparing pairwise group means, e.g., do the
South and the West differ significantly on % of Christianadherents and number of divorces.2. Census region
23.333 3.188 16.895 29.772
14.136 2.884 8.312 19.960
17.229 2.556 12.066 22.391
18.118 2.884 12.294 23.942
53.389 3.907 45.498 61.28060.182 3.534 53.044 67.320
55.921 3.133 49.594 62.248
43.718 3.534 36.580 50.856
3.600 .353 2.887 4.313
3.745 .319 3.101 4.390
4.964 .283 4.393 5.536
5.591 .319 4.946 6.236
Census region
Northeast
Midwest
South
West
NortheastMidwest
South
West
Northeast
Midwest
South
West
Dependent Variabl e
Abortio ns per 1,000
women
Percent of pop who areChristian adherents
Divorces per 1,000 pop
M ea n St d. Error L owe r Bou nd Up pe r Bou nd
95% Confidence Interval
-
7/29/2019 MANOVA - Analysis
23/33
Significant Pairwise Regional Differenceson the Two Significant DVs
You might wantto set yourconfidencelevel cutoffeven lower
since you aregoing to bedoing 12 testshere (4(3)/2)for eachvariable
-
7/29/2019 MANOVA - Analysis
24/33
Writing up All of Your MANOVAResults
Your final paragraph will look like this A one-way MANOVA revealed a significant multivariate main effect
for region, Wilks = .465, F(9, 95.066) = 3.9,p
-
7/29/2019 MANOVA - Analysis
25/33
Now You Try It!
Go here to download the filestatelevelmodified.sav
Lets test the hypothesis that region of thecountry and availability of an educated
workforce have an impact on threedependent variables: % union members,per capita income, and unemployment rate
Although a test will be performed for aninteraction between region and workforceeducation level, no specific effect ishypothesized
Go to SPSS Data Editor
http://www-rcf.usc.edu/~mmclaugh/550x/DataFiles/statelevelmodified.savhttp://www-rcf.usc.edu/~mmclaugh/550x/DataFiles/statelevelmodified.savhttp://www-rcf.usc.edu/~mmclaugh/550x/DataFiles/statelevelmodified.sav -
7/29/2019 MANOVA - Analysis
26/33
Running a MANOVA in SPSS Go to Analyze/General Linear Model/ Multivariate Move Census Region and HS Educ into the Fixed Factors
category (this is where the IVs go) Move per capita income, unemployment rate, and % of
workers who are union members into the Dependent
Variables category Under Plots, create four plots, one for each of the two main
effects (region, HS educ) and two for their interaction. Usethe Add button to add each new plot Move region into the horizontal axis window and click the Add
button Move hscat4 (HS educ) into the horizontal axis window and
click the Add button Move region into the horizontal axis window and hscat 4 into
the separate lines window and click Add Move hscat4 (HS educ) into the horizontal axis window and
region into the separate lines window and click Add, then clickContinue
-
7/29/2019 MANOVA - Analysis
27/33
Setting up MANOVA in SPSS
Under Options, move all of the factors includingthe interactions into the Display Means for window
Select descriptive statistics, estimates ofeffect size, observed power, andhomogeneity tests
Set the confidence level to .05 and clickcontinue
Click OK Compare your output to the next several
slides
-
7/29/2019 MANOVA - Analysis
28/33
ox's Test of Equality of Cova riance Matricesa
55.398
2.191
18
704.185.003
Box's M
F
df1
df2Sig.
Tests the nul l hypothesis that the observed covariance
matrices of the dependent variables are equal across groups.
Design: Intercept+REGION+HSCAT4+REGION *
HSCAT4
a.
MANOVA Main and InteractionEffects
Note that there are significant main effects for both region (green) and hscat4(red) but not for their interaction (blue). Note the values of Wilks lambda;only .237 of the variance is unexplained by region. Thats a very good result.Boxs Mis significant which is not so good but we do have high power. If youredid the analysis with a lower significance level you would lose hscat4
-
7/29/2019 MANOVA - Analysis
29/33
Univariate Tests: ANOVAs on each ofthe Three DVs for Region, HS Educ
Since we have obtained a significant multivariate main effect for eachfactor, we can go ahead and do the univariate F tests where we look ateach DV in turn to see if the two IVS have a significant impact on them
separately. Since we are doing six tests here we are going to reguire anexperiment-wise alpha rate of .05, so we will divide it by six to get anacceptable confidence level for each of the six tests, so we will set thealpha level to p < .008. By that criterion, the only significant univariateresult is for the effect of region on unemployment rate. With a morelenient criterion of .05 (and a greater probability of Type I error), three
other univariate tests would have been significant
-
7/29/2019 MANOVA - Analysis
30/33
Pairwise Comparisons on theSignificant Univariate Tests
We found that the only significant univariate main effect was forthe effect of region on unemployment rate. Now lets ask thequestion, what are the differences between regions inunemployment rate, considered two at a time?
What does the Levenes statistic say about the kind of post-hoctest we can do with respect to the region variable?
According to the output, the group variances on unemploymentrate are not significantly different, so we can do a Sheff test
Levene's Test of Equality of Error Variancesa
2.645 12 37 .012
1.281 12 37 .270
2.573 12 37 .014
Percent of workers who
are union membersUnemployment rate
percap i ncome
F df1 df2 Sig.
Tests the null hypothesis that the error variance of the dependent variable i
equal across groups.
Design: Intercept+REGION+HSCAT4+REGION * HSCAT4a.
-
7/29/2019 MANOVA - Analysis
31/33
Pairwise Difference of Means
Since we are doing 6 significance tests (K(k-1)/2) looking at the pairwisetests comparing the employment rate by region, we can use the smallerconfidence level again to protect against inflated alpha error, so lets dividethe .05 by 6 and set .008 as our error level. By this standard, the Southand Midwest and the West and Midwest are significantly different inunemployment rate.
-
7/29/2019 MANOVA - Analysis
32/33
Reporting the Differences
2. Census region
16.254 1.851 12.504 20.004
14.454a 1.656 11.098 17.810
9.447a 1.612 6.182 12.713
13.861a 1.584 10.651 17.070
5.108 .334 4.433 5.784
3.917a .299 3.312 4.521
5.076a .290 4.488 5.665
6.294a .285 5.716 6.872
23822.583 1010.336 21775.447 25869.719
20624.016a 904.224 18791.884 22456.147
21051.631a 879.836 19268.915 22834.348
21386.655a 864.768 19634.468 23138.841
Census region
Northeast
Midwest
South
West
Northeast
Midwest
South
West
Northeast
Midwest
South
West
Dependent Variable
Percent of workers who
are union members
Unemployment rate
percap income
M ea n Std. Error L owe r Boun d Uppe r Boun d
95% Confidence Interval
Based on modified population marginal mean.a.
Significant mean differences in unemployment rate wereobtained between the Midwest (M = 3.917) and the West(6.294) and Midwest and the South (M = 5.076)
-
7/29/2019 MANOVA - Analysis
33/33
Lab # 9
Duplicate the preceding data analysis in SPSS. Writeup the results (the tests of the hypothesis about themain effects of region and HS Educ on the threedependent variables of per capita income,unemployment rate, and % union members, as if youwere writing for publication. Put your paragraph in aWord document, and illustrate your results with tablesfrom the output as appropriate (for example, theoverall multivariate Ftable and the table of meanscores broken down by regions). You can also use
plots to illustrate significant effects