02 ml - testing models in spss · 5 9 step 2 null random intercept model theoretical: yij= 00 + 0j...
TRANSCRIPT
1
MULTILEVEL MODELS
Multilevel-analysis in SPSS - step by step
Dimitri MortelmansCentre for Longitudinal and Life Course Studies (CLLS)University of Antwerp
2
Overview of a strategy1. Data preparation (centering and standardizing)
2. Null random intercept model3. Random intercept with Level-1 predictors4. Adding Level-2 predictors to step 3
5. Testing random slopes level 16. Testing significant random slopes Level 17. Testing random slopes with level-2 predictors8. Reducing parameters in var-covar
9. Testing cross-level interactions10. Change estimation method and compare models11. Standardizing
2
3
The basic formula
• General multilevel-model
Yij= 00 + 10 Xij + 01 Zj + 11 Zj Xij + u1j Xij + 0j + rij
All fixed parameters are in this part
= FIXED PART
All random parameters are in this part
= RANDOM PART
4
Overview of ML models
Model
0RI Null random intercept / Intercept only / Unconditional means
RI Random intercept
FR Fully Random
3
PHASE 1: PREPARATION1. Data preparation
6
STEP 1 Data preparation
Actions:
• Look at the distribution of variables• Check outliers
• Data centering and standardizing• Check your centering and standardizing
(see slides data management)
4
PHASE 2: VARIANCE COMPONENT MODELS2. Null random intercept model3. Random intercept with Level-1 predictors4. Adding Level-2 predictors to step 3
8
PHASE 2
2. Null random intercept model• PURPOSE: estimate a basic model to compare
other models with
3. Random intercept with level-1 variables• PURPOSE: how much variance can be explained
with your level-1 predictors ?
4. Random intercept with level 1 and level 2 variables• PURPOSE: how much (additional) variance can
be explained with your level-2 predictors ?
5
9
STEP 2 Null random intercept model
THEORETICAL: Yij= 00 + 0j + rij
In the example:00 = Overall intercept: mean income over all countries
0j = Non explained differences in income between the countries
rij = Non explained differences in income between the individuals
10
STEP 2 Null random intercept model
* 2. STEP 2: Null random intercept model;* **********************************************;
MIXED ink_centr/PRINT = SOLUTION TESTCOV/METHOD = REML/FIXED = INTERCEPT/RANDOM = INTERCEPT | SUBJECT(cntry2).
6
11
STEP 2 Null random intercept model
* 2. STEP 2: Null random intercept model;* **********************************************;
MIXED ink_centr/PRINT = SOLUTION TESTCOV/METHOD = REML/FIXED = INTERCEPT/RANDOM = INTERCEPT | SUBJECT(cntry2).
Put the Dependent variable after MIXED
12
STEP 2 Null random intercept model
* * 2. STEP 2: Null random intercept model;* **********************************************;
MIXED ink_centr/PRINT = SOLUTION TESTCOV/METHOD = REML/FIXED = INTERCEPT/RANDOM = INTERCEPT | SUBJECT(cntry2).
PRINTSOLUTION = Show the Fixed effects in the output
with the standard errorsTESTCOV = Show significance tests for the
variance components
METHOD The estimation methodUsed method here = Restricted Maximum Likelihood
7
13
STEP 2 Null random intercept model
* * 2. STEP 2: Null random intercept model;* **********************************************;
MIXED ink_centr/PRINT = SOLUTION TESTCOV/METHOD = REML/FIXED = INTERCEPT/RANDOM = INTERCEPT | SUBJECT(cntry2).
FIXEDAdd the fixed variables (independent variables) here.
RANDOMAdd the random parts here. HERE we add only the intercept (for now).
14
STEP 2 Null random intercept model
OUTPUT FOR: Yij= 00 + 0j + rij
Grouping variable
Number of parameters in the model
8
15
STEP 2 Null random intercept model
OUTPUT FOR: Yij= 00 + 0j + rij with
Estimated mean incomeover all countries is -0.097
200j :0,N ~ 2
ij 0,N ~ rr
Unexplained variance (differences) on Country level (level 2) equals 2.4855
Unexplained variance (differences) on individuea level equals 4.0058
16
STEP 2 Null random intercept model
• Summary table
0RI
Intercept -0.097596
σ²r 4.005854
σ²0 2.485587
σ²1 //
9
17
STEP 2 Null random intercept model
• No explanation of the variance in income• BUT a decomposition of the variance in income
in 2 parts
σ²0 = Inter-group variance: differences between countries
σ²r = Intra-group variance: differences between individuals
QUESTION: Are there enough differences between countries to justify a multi-level analysis ?
18
STEP 2 Null random intercept model
QUESTION: Are there enough differences between countries to justify a multi-level analysis ?
ANSWER: Intra-class correlation-coefficient (RHO)
1 = σ²0 / (σ²0 + σ²r)
10
19
STEP 2 Null random intercept model
Intra-class correlation-coefficient (RHO) (1 = σ²0 / (σ²0 + σ²r) )
1 = 2.4842 / (2.4842 + 4.0076) = 0.383
INTERPRETATION: 38.3 % of the total variance can be ascribed to level 2.
PUT DIFFERENTLY: For 38.3 %, the differences in income between countries in the ESS are due to differences between countries (richer and poorer countries). 61.7 % is due to differences in individuals within these countries (richer and poorer individuals within a country).
20
STEP 3 Random intercept (level 1)
THEORETICAL: Yij= 00 + 10 Xij + 0j + rij
In the example:00 = Overall intercept: mean income over all countries
0j = Non explained differences in income between the countries
rij = Non explained differences in income between the individuals
10 Xij = General slope of the independent variable X (gender or education)
QUESTION: How much of the original variance in 0j has been explained now ?
11
21
STEP 3 Random intercept (level 1)
BASIC IDEA:Add all your level-1 predictors
De variance components in the residuals should become smaller BECAUSE the level-1 predictors now take up a part of the explanation of the total variance.
QUESTION: How big is the explained part ?
22
STEP 3 Random intercept (level 1)
* 3. STEP 3: Random intercept model with level 1 predictors;* **********************************************************************;
MIXED ink_centr BY geslacht WITH opl_centr/PRINT = SOLUTION TESTCOV/METHOD = REML/FIXED = INTERCEPT geslacht opl_centr/RANDOM = INTERCEPT | SUBJECT(cntry2).
WITH: treat the independent as a continuous variableBY: treat the independent as a categorical variable
Add your Level-1 independent variables
12
23
STEP 3 Random intercept (level 1)
OUTPUT FOR: Yij= 00 + 10 Xij + 0j + rij
Error-variance on level 2
Overall intercept
Error-variance on level 1
200j :0,N ~ 2
ij 0,N ~ rr
Coefficients at level-1
24
STEP 3 Random intercept (level 1)
OUTPUT FOR: Yij= 00 + 10 Xij + 0j + rij 200j :0,N ~ 2
ij 0,N ~ rr
The effect of gender and education on income is significant.
Even after controlling for level-1 predictorsthere still are significant differences between individuals on level-1 and significant differences between the countries on level 2.
13
25
STEP 3 Random intercept (level 1)
Reduction of individual differences:
From 4.0058 to 3.4640
Reduction of differences between countries:
From 2.4855 to 2.1303
Is this enough ? We need an R²-measure
26
STEP 3 Random intercept (level 1)
Regression-like R²-measure on level 1:
rBasis - rModel / rBasis 4.0058 - 3.4640 / 4.0058
Regression-like R²-measure on level 2:
Basis - Model / Basis 2.4855 - 2.1303 / 2.4855
BUT: The estimation methods make this easy solution erroneous.
14
27
STEP 3 Random intercept (level 1)
No consensus yet about a good R²-measure.
Correction of Bosker&Snijders (1994) – level 1:
(rBasis + Basis) - (r Model + Model) / (rBasis + Basis)
Correction of Bosker&Snijders (1994) – level 2:
(rBasis + (Basis /n) )-(r Model + ( Model/n) ) / (rBasis + (Basis/n) )
Whereby n = mean cluster size at level 2
28
STEP 3 Random intercept (level 1)Correction of Bosker&Snijders (1994) – level 1:
(rBasis + Basis) - (r Model + Model) / (rBasis + Basis) = 0.14
Correction of Bosker&Snijders (1994) – level 2:
(rBasis + (Basis /n) )-(r Model + ( Model/n) ) / (rBasis + (Basis/n) ) = 0.14
15
29
STEP 4 Random intercept (level 1+2)
THEORETICAL : Yij= 00 + 10 Xij + 01 Zj + 0j + rij
In the example:00 = Overall intercept: mean income over all countries
0j = Non explained differences in income between the countries
rij = Non explained differences in income between the individuals
10 Xij = General slope of the independent variable X (gender or education)
01 Zj = Direct effect of country variables on income (BNP or religion)
QUESTION: How much variance in 0j from step 3 is now further explained ?
30
STEP 4 Random intercept (level 1+2)
BASIC IDEA:Look how much the level 2 predictors can
explain (as a group)
16
31
STEP 4 Random intercept (level 1+2)
* 4. STEP 4: Random intercept model with level 1 and level 2 predictors;* *********************************************************;
MIXED ink_centr BY geslacht WITH opl_centr bnp_centr bnppps_centr romkath_centr relig_centr /PRINT = SOLUTION TESTCOV/METHOD = REML/FIXED = INTERCEPT geslacht opl_centr bnp_centr bnppps_centr romkath_centr relig_centr /RANDOM = INTERCEPT | SUBJECT(cntry2).
32
STEP 4 Random intercept (level 1+2)
OUTPUT FOR: Yij= 00 + 10 Xij + 01 Zj + 0j + rij 200j :0,N ~ 2
ij 0,N ~ rr
Error-variance on level 2
Overall intercept
Error-variance on level 1
Coefficients on level-1
Coefficients on level-2
17
33
STEP 4 Random intercept (level 1+2)
OUTPUT FOR: Yij= 00 + 10 Xij + 01 Zj + 0j + rij 200j :0,N ~ 2
ij 0,N ~ rr
BNPPPS and % Roman Catholics are significant, BNP and Religiosity not.
Even after controlling for level-2 predictorsthere still are significant differences between individuals on level-1 and significant differences between the countries on level 2.
34
STEP 4 Random intercept (level 1+2)
OUTPUT FOR: Yij= 00 + 10 Xij + 01 Zj + 0j + rij 200j :0,N ~ 2
ij 0,N ~ rr
Problem with BNP ?
18
35
STEP 4 Random intercept (level 1+2)
Reduction of individual differences:
3 vs 2 From 3.464 to 3.4763 vs 1 From 4.006 to 3.476
Reduction of country differences:
3 vs 2 From 2.130 to 0.5523 vs 1 From 2.486 to 0.552
36
STEP 4 Random intercept (level 1+2)
Correction of Bosker&Snijders (1994) – level 1:
Pseudo – R² = 0.38
Correction of Bosker&Snijders (1994) – level 2:
Pseudo – R² = 0.78
19
37
STEP 4 Random intercept (level 1+2)
Preliminary conclusions:
1. The error-variance on level 2 is only significant to the 0.006 level.
2. Two level-2 variables do not seem to work in this model: BNP and Religiosity. We remove them from the model when they are less important from a theoretical perspective.
3. Our model strongly explains differences on level 2. But also on the individual level the explanatory power of the independent variables is satisfactory.
PHASE 3: testing random slopes5. Testing random slopes level 16. Testing significant random slopes Level 17. Testing random slopes with level-2 predictors8. Reducing the parameters in var-covar
20
39
PHASE 35. Testing random slopes level 1
• PURPOSE: looking which level-1 predictor has significant slopes (without the level-2 predictors).
6. Testing significant random slopes Level 1• PURPOSE: joining all significant random slopes from
step 5 in one model
7. Testing random slopes with level-2 predictors• PURPOSE: adding the level 2 predictors back to
model, including the random slopes from step 6
8. Reducing parameters in var-covar• PURPOSE: simplifying the model (removing non-
significant co-variances) in order to ease the estimation of the model.
40
STEP 5 Testing random slopes level 1
THEORETICAL : Yij= 00 + 10 Xij + 1j Xij + 0j + rij
In the example:00 = Overall intercept: mean income over all countries
0j = Non explained differences in income between the countries
rij = Non explained differences in income between the individuals
10 Xij = General slope of the independent variable X (gender or education)
1j Xij = Differences in slopes (effect) of variable X (gender or education) between the countries
REMARK: No longer country differences in this model !REMARK : Model is estimated for each independent variable on level 1 separatelyQUESTION: Are there significant differences in the effect of X between the countries?
21
41
STEP 5 Testing random slopes level 1
BASIC IDEA:Until now we did not allow for differences in
slopes of the independent variables at level 1.
FOR EACH INDEPENDENT, we will test if there are significant differences in slopes at level-2.
42
STEP 5 Testing random slopes level 1
* STEP 5: Testing the significance of the random slopes step-by-step.***********************************************************************************;
MIXED ink_centr BY geslacht WITH opl_centr/PRINT = SOLUTION TESTCOV/METHOD = REML/FIXED = INTERCEPT geslacht opl_centr/RANDOM = INTERCEPT opl_centr | SUBJECT(cntry2) COVTYPE(UN).
COVTYPE should be “Unstructured”.
We make eduction RANDOM here.
22
43
STEP 5 Testing random slopes level 1
OUTPUT FOR: Yij= 00 + 10 Xij + 1j Xij + 0j + rij
Error-variance on level 2
Coefficients on level-1
Error-variance on level 1
Overall intercept
2ij 0,N ~ rr
21
210
20
1j
0j:0,N ~
44
STEP 5 Testing random slopes level 1
Structure of the variance-covariance matrix
Theoretical:
In SPSS:
21
210
20
1j
0j :0,N ~
2,21,2
1,1
UNUN
UN
01991.003258.0
1123.2
23
45
STEP 5 Testing random slopes level 1
Structure of the variance-covariance matrix
In SPSS:
2,21,2
1,1
UNUN
UN
01991.003258.0
1123.2
The random slope is significant on the 0.01 level.MEANING: there is a significant different effect of education between the countries
46
STEP 5 Testing random slopes level 1
* STEP 5: Testing the significance of the random slopes step-by-step.**********************************************************************************.
MIXED ink_centr BY geslacht WITH opl_centr/PRINT = SOLUTION TESTCOV/METHOD = REML/FIXED = INTERCEPT geslacht opl_centr/RANDOM = INTERCEPT geslacht | SUBJECT(cntry2) COVTYPE(UN).
Gender is made RANDOM here.
24
47
STEP 5 Testing random slopes level 1
OUTPUT FOR: Yij= 00 + 10 Xij + 1j Xij + 0j + rij
Error-variance on level 2
Coefficients on level-1
Error-variance on level 1
Overall intercept
2ij 0,N ~ rr
21
210
20
1j
0j:0,N ~
48
STEP 5 Testing random slopes level 1
Structure of the variance-covariance matrix
Theoretical:
In SPSS:
21
210
20
1j
0j :0,N ~
2,21,2
1,1
UNUN
UN
6207.04728.0
144.1
25
49
STEP 5 Testing random slopes level 1
Evaluation of the random slope of gender
In SPSS:
2,21,2
1,1
UNUN
UN
The random slope of the dichotomous variable gender can not be tested. If you include “gender” as a continuous variable in the model, this random slope turns out to be significant.
0097.004622.0
1725.2
50
STEP 5 Testing random slopes level 1
Conclusion:
- Education and gender have significant slopes between the countries.
- Gender is only estimable when you include the variable as a continuous measure.
BUT: the effect of gender is unclear.
In this example I will keep gender in the model with a random slope (only for educational purposes).
26
51
STEP 6 Testing random slopes level 1+2
THEORETICAL : Yij= 00 + 10 Xij + 1j Xij + 0j + rij
00 = Overall intercept: mean income over all countries
0j = Non explained differences in income between the countries
rij = Non explained differences in income between the individuals
10 Xij = General slope of the independent variable X (gender or education)
1j Xij = Differences in slopes (effect) of variable X (gender or education) between the countries
Theoretically this is the same model as in step 5 but now we include ALL significant random slopes in the model.
52
STEP 6 Testing random slopes level 1+2
* STEP 6: Including all level-1 variables with significant random slopes.* ****************************************************************************************.
MIXED ink_centr BY geslacht WITH opl_centr/PRINT = SOLUTION TESTCOV/METHOD = REML/FIXED = INTERCEPT geslacht opl_centr/RANDOM = INTERCEPT opl_centr geslacht | SUBJECT(cntry2) COVTYPE(UN).
27
53
STEP 6 Testing random slopes level 1+2
OUTPUT FOR: Yij= 00 + 10 Xij + 1j Xij + 0j + rij
Error-variance on level 2
Error-variance on level 1
2ij 0,N ~ rr
21
210
20
1j
0j:0,N ~
Equal digits are error-variances:
UN(1,1) Intercept
UN(2,2) Error-variance of education
UN (3,3) Error-variance of gender
54
STEP 6 Testing random slopes level 1+2
OUTPUT FOR: Yij= 00 + 10 Xij + 1j Xij + 0j + rij 2ij 0,N ~ rr
21
210
20
1j
0j:0,N ~
Equal digits are error-variances:
UN(1,1) Intercept
UN(2,2) Error-variance of education Significant on the 0.01 level
UN (3,3) Error-variance of gender NOT significant: SO: Gender NOT random in the model!
28
55
STEP 7 Testing random slopes with level-2 predictors
THEORETICAL : Yij= 00 + 10 Xij + 01 Zj + 1j Xij + 0j + rij
00 = Overall intercept: mean income over all countries
0j = Non explained differences in income between the countries
rij = Non explained differences in income between the individuals
10 Xij = General slope of the independent variable X (gender or education)
1j Xij = Differences in slopes (effect) of variable X (gender or education) between the countries
01 Zj = Direct effect of country variables on income (BNP or religion)
56
STEP 7 Testing random slopes with level-2 predictors
BASIC IDEA:We now check which influence the level-2
predictors have on income in a model with random slopes (= step 6).
29
57
STEP 7 Testing random slopes with level-2 predictors
* STEP 7: Including all level-2 variables in the model of step 6*. ******************************************************************************.
MIXED ink_centr BY geslacht WITH opl_centr bnppps_centr romkath_centr/PRINT = SOLUTION TESTCOV/METHOD = REML/FIXED = INTERCEPT geslacht opl_centr bnppps_centr romkath_centr /RANDOM = INTERCEPT opl_centr | SUBJECT(cntry2) COVTYPE(UN).
58
STEP 7 Testing random slopes with level-2 predictors
OUTPUT FOR: Yij= 00 + 10 Xij + 01 Zj + 1j Xij + 0j + rij
Error-variance on level 2
Coefficients on level-1
Error-variance on level 1
Overall intercept
2ij 0,N ~ rr
21
210
20
1j
0j:0,N ~
Coefficients on level-2
30
59
STEP 7 Testing random slopes with level-2 predictors
OUTPUT FOR: Yij= 00 + 10 Xij + 01 Zj + 1j Xij + 0j + rij 2ij 0,N ~ rr
21
210
20
1j
0j:0,N ~
Significant differences remain on level 2 for the effect of education
Both the predictors on level 1 as on level 2 are significant.
Significant differences remain between the intercepts of the countries.
60
STEP 8 Simplifying the model
THEORETICAL : Yij= 00 + 10 Xij + 01 Zj + 1j Xij + 0j + rij
00 = Overall intercept: mean income over all countries
0j = Non explained differences in income between the countries
rij = Non explained differences in income between the individuals
10 Xij = General slope of the independent variable X (gender or education)
1j Xij = Differences in slopes (effect) of variable X (gender or education) between the countries
01 Zj = Direct effect of country variables on income (BNP or religion)
We now look at the variance-covariance matrix
31
61
STEP 8 Simplifying the model
Differences in intercept and slope between units of level 2.
Yij= 00 + 10 Xij + 1j Xij + 0j + rij
These are the conditions:
rij ~ N(0, σ²r)
This is the covariance between the random intercepts and the random slopes for variabele Xij.
21
210
20
1j
0j :0,N ~
If this covariance is significant (= different from 0), then it means that the changes in intercepts is systematicallycorrelated to the changes in slopes.
62
STEP 8 Simplifying the model
Model intercept Slopes Covariance
FR many many Negative
FR many many Positive
FR many manyNot
significant (0)
32
63
STEP 8 Simplifying the model
With 1 independent X-variable on level 1
With 2 independent X-variables on level 1
21
210
20
1j
0j :0,N ~
Covariance between the random intercepts and the random slopes for variabele Xij.
22
221
220
21
210
20
2j
1j
0j
:0,N ~
2 independents = 3 covariances
64
STEP 8 Simplifying the model
With 3 independent X-variable on level 1
With random slopes models, the number of parameters to be estimated increases very rapidly.
23
232
231
230
22
221
220
21
210
20
3j
2j
1j
0j
:0,N ~
3 independents = 6 covariances
33
65
STEP 8 Simplifying the model
With 3 independent X-variables on level 1
23
232
231
230
22
221
220
21
210
20
3j
2j
1j
0j
:0,N ~
Non-significant covariances = The covariances is not significantly different from 0.
SO: We might as well fix these to 0.
CONSEQUENCE: Each fixed covariance is one parameter less to be estimated.
66
STEP 8 Simplifying the model
With 3 independent X-variables on level 1
23
232
231
230
22
221
220
21
210
20
3j
2j
1j
0j
:0,N ~
SPSS does not allow to fix each separate covariance to 0.
BUT: there is an option to fix all co-variances to 0 in one time.
34
67
STEP 8 Simplifying the model
With 3 independent X-variables on level 1
23
232
231
230
22
221
220
21
210
20
3j
2j
1j
0j
:0,N ~
COVTYPE(UN)
23
22
21
20
3j
2j
1j
0j
000
00
0:0,N ~
Without COVTYPE
68
STEP 8 Simplifying the model
Look at the (significance of) the co-variances in the output of step 7
Covariance is NOT significant
Consequence: Estimate the model in STEP 7 again without the COVTYPE-option.
35
69
STEP 8 Simplifying the model
•STEP 8: Reduction of parameters in var-covar matrix•*********************************************************.
MIXED ink_centr BY geslacht WITH opl_centr bnppps_centr romkath_centr/PRINT = SOLUTION TESTCOV/METHOD = REML/FIXED = INTERCEPT geslacht opl_centr bnppps_centr romkath_centr /RANDOM = INTERCEPT opl_centr | SUBJECT(cntry2).
70
STEP 8 Simplifying the model
The co-variances are fixed at 0 : there is no estimation of co-variances any more. Only the variances are given in the output.
36
PHASE 4: Refining and finishing9. Testing cross-level interactions10.Change the estimation method and compare your
models11.Standardizing
72
PHASE 4
9. Testing cross-level interactions• PURPOSE: checking if level-2 predictors correlate
with level-1 predictors
10.Changing the estimation method and comparing models• PURPOSE: Compare models with Full Maximum
Likelihood
11.Standardizing• PURPOSE: Estimating the end model again with
standardized variables to check the relative influence
37
73
STEP 9 Testing cross-level interactions
THEORETICAL : Yij= 00 + 10 Xij + 01 Zj + 01 Zj Xij + 1j Xij + 0j + rij
00 = Overall intercept: mean income over all countries
0j = Non explained differences in income between the countries
rij = Non explained differences in income between the individuals
10 Xij = General slope of the independent variable X (gender or education)
1j Xij = Differences in slopes (effect) of variable X (gender or education) between the countries
01 Zj = Direct effect of country variables on income (BNP or religion)
01 Zj Xij = Interaction of country variables and individual variables on income
74
STEP 9 Testing cross-level interactions
What is an interaction effect ?
A (significant) interaction effect shows how the effect of one independent variable changes within the levels of another independent variable.
38
75
STEP 9 Testing cross-level interactions
• In regular OLS regression
EG. Interaction between gender and education:
How does the effect of education on income change between boys and girls?
• In multilevel regression
EG. Interaction between education and GDP:
How does the effect of education on income change for countries with a different GDP ?
76
STEP 9 Testing cross-level interactions
TWO BASIC PRINCIPLES
1.When an interaction effect is significant, the main effects NEED TO BE included in the model.
EG: When the interaction effect between education and GDP_PPS is included THEN you also need the main effect of education and the main effect of GDP_PPS in the model.
39
77
STEP 9 Testing cross-level interactions
TWO BASIC PRINCIPLES
2. When an interaction effect is present in a model, the main effects have a different meaning.
Meaning without interaction-effect- Education: the change in income when the educational
level rises with one unit
- BNP_PPS: the change in income when BNP_PPS rises with one unit
78
STEP 9 Testing cross-level interactions
TWO BASIC PRINCIPLES
2. When an interaction effect is present in a model, the main effects have a different meaning.
Meaning without interaction-effect- Education: the change in income when the educational
level rises with one unit at a level of 0 from BNP_PPS
- BNP_PPS: the change in income when BNP_PPS rises with one unit at a level of 0 from education
40
79
STEP 9 Testing cross-level interactions
* * STEP 9: Testing cross-level interactions* **********************************************************************.
MIXED ink_centr BY geslacht WITH opl_centr bnppps_centr romkath_centr/PRINT = SOLUTION TESTCOV/METHOD = REML/FIXED = INTERCEPT geslacht opl_centr bnppps_centr romkath_centr
opl_centr*bnppps_centr/RANDOM = INTERCEPT opl_centr | SUBJECT(cntry2).
Specifying an interaction effect = repeating both variables with * between both.
80
STEP 9 Testing cross-level interactions
Main effects (both are significant)
Interaction effect (not significant)
41
81
STEP 9 Testing cross-level interactions
How to interpret the interaction effect ?(if it had been significant)
1. Interpret the main effects
2. Choose an interpretation perspective
3. Write the regression equation for one of the independent variables
82
STEP 9 Testing cross-level interactions
Interpretation 1. Interpret the main effects
Main effect education: When BNPPPS_centr is equal to 0, someone's income will increase with 0.5192 scale points when his educational level increases with one unit.
Main effect GDP: When education is equal to 0, someone's income will increase with 0.0265 scale points when his countries BNP_PPS increases with one unit.
42
83
STEP 9 Testing cross-level interactions
Interpretation 1. Interpret the main effects
Because of the centering (mean = 0) we can also say:
Main effect education: At a mean BNPPPS_centr level, someone's income will increase with 0.5192 scale points when his educational level increases with one unit.
Main effect GDP: At a mean educational level, someone's income will increase with 0.0265 scale points when his countries BNP_PPS increases with one unit.
84
STEP 9 Testing cross-level interactions
Interpretation 2. Choose an interpretation perspective
2 perspectives:
1. How does the effect of education on income changes for different levels of BNP_PPS ?
2. How does the effect of BNP_PPS on income changes for different levels of education ?
43
85
STEP 9 Testing cross-level interactions
Interpretation 2. Choose an interpretation perspective
“How does the effect of education on income changes for different levels of BNP_PPS ?”
is theoretically the only possibility here.
BECAUSE: BNP_PPS will not change if people have a higher degree. But the educational effects can indeed change for countries with different BNP_PPS.
BUT REMEMBER: Statistically, both perspectives are always possible.
86
STEP 9 Testing cross-level interactions
Interpretation 3. Work out the regression equation
PERSPECTIVE: How does the effect of education on income changes for different levels of BNP_PPS ?
- Write out the regression equation for education, when BNP_PPS takes on different levels.
- WHICH levels of BNP_PPS ? Usually: minimum, 25th percentile, median, mean, 75th
percentile and maximum.
44
87
STEP 9 Testing cross-level interactions
Interpretation 3. Work out the regression equation
STEP 1: Search in the EXAMINE output for the values of the minimum, 25th percentile, median, mean, 75th percentile and the maximum of BNPPPS_centr.
EXAMINE VARIABLES=bnppps_centr/PERCENTILES(25,75) /STATISTICS DESCRIPTIVES.
88
STEP 9 Testing cross-level interactions
Interpretation 3. Work out the regression equation
STEP 1: Search in the EXAMINE output for the values of the minimum, 25th percentile, median, mean, 75th percentile and the maximum of BNPPPS_centr.
45
89
STEP 9 Testing cross-level interactions
Interpretation 3. Work out the regression equation
STEP 1
Minimum -65.8815Q1 -23.2815Mean 0Median 2.1185Q3 14.6185maximum 126.9125
Remark: We assume that the other independent variable also take on the value of 0 (and disappear from the equation).
90
STEP 9 Testing cross-level interactions
Interpretation 3. Work out the regression equation
Minimum Income = -0.2532 + 0.5192 opleiding + 0.0265 (-65.88) + 0.0008 OPL*(-65.88)Q1 Income = -0.2532 + 0.5192 opleiding + 0.0265 (-23.2815) + 0.0008 OPL*(-23.2815)Median Income = -0.2532 + 0.5192 opleiding + 0.0265 (0) + 0.0008 OPL*(0)Mean Income = -0.2532 + 0.5192 opleiding + 0.0265 (2.1185) + 0.0008 OPL*(2.1185)Q3 Income = -0.2532 + 0.5192 opleiding + 0.0265 (14.6185) + 0.0008 OPL*(14.6185)maximum Income = -0.2532 + 0.5192 opleiding + 0.0265 (126.92) + 0.0008 OPL*(126.92)
Income = -0.2532 + 0.5192 education + 0.0265 BNPPPS + 0.0008 EDUC*BNPPPS
STEP 2: Vul in
46
91
STEP 9 Testing cross-level interactions
Interpretation 3. Work out the regression equation
Minimum Income = -1,999 + 0,4665 OPLQ1 Income = -0,8702 + 0,5006 OPLMedian Income = -0,1971 + 0,5209 OPLMean Income = -0,2532 + 0,5192 OPLQ3 Income = 0,1342 + 0,5309 OPLmaximum Income = 3,11 + 0,6207 OPL
Income = -0.2532 + 0.5192 education+ 0.0265 BNPPPS + 0.0008 EDUC*BNPPPS
STEP 2: Vul in
Cfr main effect of education when BNP_PPS is equal to 0 (see the main effect in the output).
Interpretation interaction effectThe effect of education rises (very slightly) as BNP_PPS increases.
92
STEP 10 Comparing models with Full Maximum Likelihood
• Estimation in multilevel analysis
- Maximum Likelihood
- Generalized Least Squares (not in SPSS)- Generalized Estimation Equations (not in SPSS)- Markov Chain Monte Carlo (not in SPSS)- Bootstrapping (not in SPSS)
47
93
STEP 10 Comparing models with Full Maximum Likelihood
• Maximum Likelihood
- Maximizing the likelihood function- Population parameters are estimated in such a way
that the probability of finding that particular model is maximized with the data at hand.
- TWO versions: Full Maximum Likelihood (ML)Restricted Maximum Likelihood (REML)
94
STEP 10 Comparing models with Full Maximum Likelihood
• Restricted Maximum Likelihood
- Only the variance components are used in the estimation.
- Standard in SAS (but not in SPSS) and in almost all multilevel programs
- Best estimation for the variance components
- CONSEQUENCE: Gives you the best estimation for the parameters and you should use this estimation method to obtain your parameters
48
95
STEP 10 Comparing models with Full Maximum Likelihood
• Full Maximum Likelihood
- Variance components and regression coefficients are involved in the estimation.
- Disadvantage: Variance components are underestimated.
USEFULL ?- Easier estimation method- The difference between two models can be used in a
chi²-difference test (is not possible with REML !).
96
STEP 10 Comparing models with Full Maximum Likelihood
• CONSEQUENCE
- Use REML for all models
- Use ML at the end to compare estimated models with each other.
49
97
STEP 10 Comparing models with Full Maximum Likelihood
* STEP 10: Using Full Maximum Likelihood to compare models.* ************************************************************************************.
*Step2.
MIXED ink_centr/PRINT = SOLUTION TESTCOV/METHOD = ML/FIXED = INTERCEPT/RANDOM = INTERCEPT | SUBJECT(cntry2).
Use Method=ML (or leave the METHOD line out).
98
STEP 10 Comparing models with Full Maximum Likelihood
Use the difference in DEVIANCE (-2 LOG LIKELIHOOD) and the difference in degrees of freedom in a Chi²-difference test
50
99
STEP 10 Comparing models with Full Maximum Likelihood
• Calculating the degrees of freedom of your model
Calculating the degrees of freedom by hand:
Intercept 1
Error variance level-1 1
Fixed slopes on level-1 p
Error-variances for slopes on level-1 p
Co-variances of slopes and intercept p
Co-variances between the slopes p(p-1)/2
Fixed slopes on level-2 q
Slopes for cross-level interactions p*q
Whereby:
p = number of level-1 predictorsq = number of level-2 predictors
100
STEP 10 Comparing models with Full Maximum Likelihood
Make an overview table with tests
DEVIANCE D(f)Prob. Chi² vs null
modelProb. Chi² vsprevious model
STEP 2 Null random intercept model 141642,268 2
STEP 3 Random intercept with level-1 136763,937 9 0,000
STEP 4 Random intercept w level 1 and 2 122423,421 13 0,000 0,000
STEP 6 Random slopes model 136407,950 9 0,000
STEP 7 Random slopes with level-2 122075,321 11 0,000 0,000
STEP 8Random slopes with level-2
(reduction)122077,357 7 0,000 0,729
STEP 9 Cross level interactions 122077,357 12 0,000 1,000
51
101
STEP 10 Comparing models with Full Maximum Likelihood
Make an overview table with tests
DEVIANCE D(f)Prob. Chi² vs null
modelProb. Chi² vsprevious model
STEP 2 Null random intercept model 141642,268 2
STEP 3 Random intercept with level-1 136763,937 9 0,000
STEP 4 Random intercept w level 1 and 2 122423,421 13 0,000 0,000
STEP 6 Random slopes model 136407,950 9 0,000
STEP 7 Random slopes with level-2 122075,321 11 0,000 0,000
STEP 8Random slopes with level-2
(reduction)122077,357 7 0,000 0,729
STEP 9 Cross level interactions 122077,357 12 0,000 1,000
This is the basic model.
In the fourth column, you compare with this model
Chi²-difference test
- Test value = 141642.268 – 136763.937 = 4878- Degrees of freedom = 9 – 2 = 7
- Result: Probability = 0.000
- Interpretation: The model in STEP3 is a significant improvement of the Null random intercept model (STEP 2)
102
STEP 10 Comparing models with Full Maximum Likelihood
Make an overview table with tests
DEVIANCE D(f)Prob. Chi² vs null
modelProb. Chi² vsprevious model
STEP 2 Null random intercept model 141642,268 2
STEP 3 Random intercept with level-1 136763,937 9 0,000
STEP 4 Random intercept w level 1 and 2 122423,421 13 0,000 0,000
STEP 6 Random slopes model 136407,950 9 0,000
STEP 7 Random slopes met level-2 122075,321 11 0,000 0,000
STEP 8Random slopes met level-2
(reductie)122077,357 7 0,000 0,729
STEP 9 Cross level interacties 122077,357 12 0,000 1,000
Comparison with the Null random intercept model (STEP 2)
Interpretation: Is this model a significant improvement vs STEP 2 ?
Comparison with the previous model (= STEP 3)
Interpretation: Is this model a significant improvement vs STEP 3?
52
103
STEP 10 Comparing models with Full Maximum Likelihood
Make an overview table with tests
DEVIANCE D(f)Prob. Chi² vs null
modelProb. Chi² vsprevious model
STEP 2 Null random intercept model 141642,268 2
STEP 3 Random intercept met level-1 136763,937 9 0,000
STEP 4 Random intercept met level 1 en 2 122423,421 13 0,000 0,000
STEP 6 Random slopes model 136407,950 9 0,000
STEP 7 Random slopes met level-2 122075,321 11 0,000 0,000
STEP 8Random slopes met level-2
(reductie)122077,357 7 0,000 0,729
STEP 9 Cross level interacties 122077,357 12 0,000 1,000
BE CAREFULL here !!
Previous model is the model with co-variances. SO is the model without co-variances a significant worsening vs. the previous model ?
CONSEQUENCE: A probability higher than 0.05 points to a good model.
104
STEP 11 The final model with standardized variables
• Standard output of multilevel models = not standardized.
• Interpretation in original (scale) units
• DISADVANTAGE: no comparison of the actual size of the parameters.
53
105
STEP 11 The final model with standardized variables
• 2 methods
1. Estimate the last model again with standardized variables
(disadvantage = variance components also change)
2. Standardize the parameter by hand
106
STEP 11 The final model with standardized variables
METHOD 1
* STEP 11: Standardising in order to compare parameters. * ********************************************************************************************;
MIXED ink_std BY gesl_std WITH opl_std bnppps_std romkath_std/PRINT = SOLUTION TESTCOV/METHOD = ML/FIXED = INTERCEPT geslacht opl_std bnppps_std romkath_std/RANDOM = INTERCEPT opl_std | SUBJECT(cntry2).
54
107
STEP 11 The final model with standardized variables
METHOD 2
Stand. Coeff. = (non stand. Coeff) x (stan dev INdep.)(stan dev DEpendent)
108
STEP 11 The final model with standardized variables
METHOD 2
DESCRIPTIVES VARIABLES=ink_centr geslacht opl_centr bnppps_centr romkath_centr /STATISTICS=STDDEV.
55
109
STEP 11 The final model with standardized variables
RESULT:
Nonstandardized Standardized Non
standardized Standardized
Gender 0,281 0,06 BNP PPS 0,027 0,40
Education 0,519 0,31 % Rom Cath -0,095 -0,15
MULTILEVEL MODELS
Multilevel-analysis in SPSS - step by step
Dimitri MortelmansCentre for Longitudinal and Life Course Studies (CLLS)University of Antwerp