a practical guide to multilevel modeling

A Practical Guide to Multilevel Modeling

Amie M. GordonUCSF

SPSP – Online Learning Webinar

Usual caveat – this is what I know to be true as of Fall 2018, this is an area of statistics that is frequently updating

Table of Contents

• What it is, when you need it (and when you don’t)• Identifying the structure of your data• Fixed and random effects• Centering• Covariance matrices• Setting Up Data & Sample Syntax• Interpreting Basic Output• Power and Sample Size Calculations• Resources

Note: Click on a topic to go directly to that content

MLM: What it is

• An approach to dealing with non-independence between data points (i.e., clustered data)

• Multilevel modeling (MLM) is also known as…• Hierarchical Linear Modeling (HLM)• Linear Mixed Modeling• Random Coefficient Modeling• Variance Component Modeling

When do you need MLM?• If your data violates the “Assumption of Independence” required for

basic statistical approaches

Repeated measures nested within individuals• Event-contingent sampling, daily diary, longitudinal, multiple videos watched

within a lab session, rating multiple targets on traitsIndividuals nested within groups- Students in a school, employees on teams, romantic partners, families

• The basic problem: observations within a group are likely to be correlated, which leads us to underestimate the SEs for our effects, leading to Type I error.

Do you need to use MLM?One reason you wouldn’t use MLM:Although your data may be clustered, it’s possible your data points are not actually interdependent

• Strangers brought into the lab to interact likely to have similar hormonal profile? Students in classrooms (all from same grade) likely to have similar height?

• You can test this with the Intraclass Correlation (ICC)• Amount of variance due to clustering• How correlated two random data points within a cluster are expected to be, are

they more highly correlated than two data points from two different clusters?

Testing for Interdependence• To test the strength of the ICC:

• One-way ANOVA with clusters as grouping variable • possible to calculate ICC using MSbetween and Mserror

• Intercept-only MLM • ρ (rho) = relative proportion of cluster variance to total variance

A few caveats:• You must test each variable separately• Even if the ICC is low, it may make sense conceptually to treat the data as

hierarchical • No hard and fast rules, but have seen ICC = .10 to be considered high enough

that MLM should be used.Typically factors you think will be clustered are clustered. It’s just that occasionally you collect clustered data, but the variables you are interested in aren’t at all related to the clustering (e.g., brought randomly-paired strangers into the lab to interact but you are looking at their personality and not interested in interaction-related variables)

Do you need to use MLM?

• Another reason you wouldn’t use MLM: Although your data are clustered, you have a very small number of

clusters and/or are interested in directly comparing the clusters to each other

– Gender– Culture, Ethnicity– 4 classes in a school– 10 teams in a company

• This is called the “Fixed Effects” approach

Why not always use the fixed effects approach?• When you have a large number of clusters and you are not trying to

meaningfully compare one cluster to another• If you have 100 participants – are you going to enter in 99 dummy codes to

control for their differences?

• MLM captures all of the variability of 99 dummy codes in one parameter: amount of variance due to cluster differences

• Allows you to look at cluster-level variables (for individuals: gender, attachment, SES) without having to interact it with 99 dummy codes

Other Ways to Deal with Non-Independence• Other ways you can deal with clustering without using MLM

• Aggregation: take all data points at lowest level and aggregate them into a single data point

• Average depressive mood across a year, proportion of time spent hanging out with friends during ESM study, number of conflicts with partner reported during two week diary

• Disaggregation: take data point from high level and assign it to each data point at the lower level

• Every time-point gets same score for P’s Gender; Depression Diagnosis

• Statistical Problems with these approaches:• Over or underestimate sample size, SEs, may misrepresent population

How to Deal with Non-Independence• It’s not just about these statistical limitations, there are also conceptual

limitations• What is happening at the within-group level may not reflect what is happening

between groups clustercluster

The Difference between MLM and Aggregation

clustercluster

Aggregation Problems: An ExampleLinear Regression

Multilevel Modeling

Male = 1, Female = 0

Do you need to use MLM?

1. A principal rates the performance of his 50 teachers for competence in the classroom.

2. Data about recovery from 300 people in therapy. Data is drawn from the clientele of 50 different therapists.

3. 100 students report on their feelings of belonging several times a week for one month.

4. 1000 people from 5 different cultures report on their emotional experiences one time.

5. 300 Students complete questionnaires in random groups of 3-5. Students complete questions about their current GPA, class schedule, where they grew up, and other basic demographics.

Yes

Yes

No – fixed effects approach

No

No

Creating a Multilevel Model

1. What is the structure of my nested data?2. Are my effects fixed or random?3. What type of centering should I use?4. Which covariance matrices should I use?

The Structure of Nested Data

Level 1 (lowest level)

Level 2


Diary Days(level 1)

Individuals(level 2)


Longitudinal Waves

(level 1)

Individuals(level 2)


Employees(level 1)

Teams(level 2)


Individual(level 3)

Daily diary(level 2)

Momentary Assessment

(level 1)

Number of levels is determined by the nature of the data


Companies(level 3)

Teams(level 2)

Employees(level 1)

Number of levels is determined by the nature of the data


Individual

Day

Interdependence Within Individuals

The Cross-Classified Model

1 2 3 1 2 3 1 2 3 1 2 3


Individual

Day

Interdependence Within Days (can be, but not always the case with diary data)

1 2 3 1 2 3 1 2 3 1 2 3



Individual

Diary 1 2 3 1 2 3 1 2 3 1 2 3

Day

Not sure if your data has interdependence within days? Test ICC using Day as cluster variable



Participant

Ratings 1 2 3 1 2 3 1 2 3 1 2 3

Target


Judd, Westfall, & Kenny, 2012 (JPSP)


Participant

Ratings 1 3 2 3 1 2 2 3

Target



Dyad

Person

Dyadic Data

A BA BA BA B


Dyad

Person

Longitudinal Dyadic Data

Day 1 2 3 1 2 3 1 2 3 1 2 3

A B A B

Note: THIS IS INCORRECT!


Dyad

Longitudinal Dyadic Data

Score A1 B1 A2 B2 A3 B3

Dyad

A1 B1 A2 B2 A3 B3

What is the structure of this data?

1. Negotiation outcome score for stranger dyads who interact in the lab

Dyad-level (one outcome per dyad)

2. Employee productivity from 2000 employees in 100 companies. Employees came from 5 different departments in each company

Employee Productivity within Departments within CompaniesEmployees within Companies with Departments dummy-coded

3. Ratings of intelligence for 50 faces by 500 participants who randomly saw 25 of the 50 faces

Intelligence within Faces and within Participants (cross-classified)

Fixed and Random Effects

Random Factor versus Random Effect

• The random factor is your clustering variable• Days within people

• Random factor = People• Classes within school

• Random factor = School• Participants rating 50 target faces

• 2 Random factors: Participants & Faces• ESM report 3 times per day

• Random factor(s):• Teams within companies

• Random factor(s):

Days & People (nested)

Company

Fixed and Random Effects

Unique about MLM is the ability to look not just at average associations between predictors and

outcomes, but how people vary around the average.

The average associations are the fixed effects,the variations are the random effects

(for the different random factors)

Sleep Quality

Gra

titud

e

Sleep Quality

Gra

titud

e

Linear Regression Equation

Y = a + βX1 + r

Sleep Quality

Gra

titud

e

Random Effects

• Because random effects capture cluster variability, you can only have a random effect for a lower level variable

• Days within people: Can have random effects for day-level variables• ESM within days within people: Can have random effects for variables from

ESM and days

What about:• Departments within companies:• Teams within departments within companies:• Participants rating 50 target faces:

DepartmentsTeams & Departments

Participants’ rating of faces (kind, trusting) note: can be random by participant and/or by face

Random Effects• You have a choice about whether or not to allow your Level 1 intercept and

slopes to have variability (i.e., have random effects).• MUST HAVE AT LEAST ONE RANDOM EFFECT FOR IT TO BE A MULTILEVEL MODEL, BUT

NOT ALL RANDOM EFFECTS HAVE TO BE INCLUDED• If you allow random effects, you are allowing variability among individuals, this is a strength of

MLM (and why you might choose MLM over another model!)• Intercept: typically random (weird to force everyone to have same mean, but may be

occasions where this makes sense)• Slope: most conservative is to allow it to be random BUT

• DFs go down (calculated differently depending on software program)• Makes more sophisticated analyses more complicated (e.g., Level 1 mediation, moderation)

• How do you decide?• Compare models with and without random effects• Theoretical reason why individuals/groups would differ?• Test how fixed effects change with and without random component

Random EffectsSpaghetti Plots

0

2

4

6

8

10Fixed Intercept & Slope

0

2

4

6

8

10Fixed Intercept & Random

Slope

0

2

4

6

8

10Random Intercept & Fixed

Slope

0

2

4

6

8

10Random Intercept & Slope

clustercluster

Random Effects Make it MLM

Slope without random coefficients

Slope withrandom intercept

Random EffectsALL FIXED EFFECTS

RANDOM INTERCEPT

Random EffectsYou might now feel like you have to include random slopes for all of your predictors, and this is a good default/starting point, but there may be times when it makes sense not to include at least some random effects:

Why not always include random effects?-Models with overly complicated random effects may not converge-Model will have trouble converging or give an error message “final hessian matrix is not positive definite” when random effects are essentially 0-Model has more power (dfs) and is simpler without random effects, so it makes sense not to include them if there is no variability. -Random effects complicate mediation and moderation analyses

So what do you do?• How do you decide?

• Compare models with and without random effects to see if model fit changes (Can test for significance of random effect using Wald test in some programs but some advice against this because 0 is near edge of distribution so SE may be biased).

• Theoretical reason why individuals/groups would differ?• Often the associations I wouldn’t expect to differ don’t

• Test how fixed effects change with and without random slopes (are the regression estimates similar either way?)

Fixed Versus Random Effects

• What is the difference between a fixed effect and a random effect?

1. A fixed effect does not vary by cluster. Instead these represent average values across clusters, such as the grand mean of the sample.

2. A random effect does vary by cluster. These estimates capture the variability between clusters.

Centering the DataThree typical types of centering: • Uncentered• Grand-mean centered• Cluster-mean centered

Uncentered• Uncentered:

Data is in raw units• Intercept = expected value of outcome when predictor is 0

• Example 1: My level of gratitude when I score 0 on sleep• Example 2: Level of gratitude for people whose income is 0

• Because models with random intercepts (most models) estimate between-cluster variability of the intercept (do people differ in their level of gratitude when they score 0 on sleep?), it is important to make 0 a meaningful score.

Grand-Mean Centered• Grand-mean centered:

0 of variable now represents the grand mean of the entire sample

• Just like centering in linear regression, this will change the value of the intercept but NOT the value of the slope

• Intercept = expected value of outcome for grand mean of predictor• Example 1: My level of gratitude when I experience the average amount of sleep for the

entire sample• Example 2: Gratitude for person who has average income

Cluster-mean Centered• Cluster-mean centered (or group-mean): 0 of variable now represents the within-cluster mean for each cluster (cannot be done for highest level) • This will change the value of the intercept and the meaning of the

predictor estimates– Level 1 Intercept: expected value of outcome for individual average

• Eg. My level of gratitude when I experience my own average level of sleep– Level 1 Slope: expected change in outcome corresponding to a 1 unit change

in a predictor relative to the cluster average• Example 1: My level of gratitude when I experience more or less sleep than I typically do

across the two weeks.• Example 2: Can’t do for income (only 1 income score per person/cluster)

Which Centering To Choose• When to uncenter or grand-mean centered:

• You just want to know if the effect exists, you don’t care whether it is due to within or between-cluster variability.

• Disagreement about whether this should be your default (some say yes, others say person/group-centered should be default)

Example: I want to know whether an additional hour of sleep promotes gratitude

• When to cluster-mean center• You are interested in relative differences within clusters and want to get

rid of between-cluster varianceExample: I want to know if people are less grateful when they sleep more than they usually do, regardless of whether they tend to sleep a lot of a little

Centering your Data• Some people argue data should always be cluster-mean centered

because if you don’t, you are confounding within- and between-cluster variation

An effect for sleep and gratitude could be due to:• Between-Person Effects: Certain participants are good sleepers every day,

and also more grateful relative to other people in the sample.• Within-Person Effects: Regardless of how well participants typically sleep,

on days when they sleep better than they usually do, they’re more grateful -An uncentered or grand-mean centered effect could be due to one of these effects or to both of them.

-At times, these effects can go in different directions:Holding back your opinions to prevent relationship conflict more than you usually do one day might not affect your relationship quality that day, but at the aggregate level, being someone who is always holding back their opinions might negatively affect relationship quality

Centering in MLM

Cf. Enders & Tofighi, 2007

Grand-Mean Centering (Confounding)Raw Data (Confounding)

Centering in MLM

Cf. Enders & Tofighi, 2007

Between Person Effect Within Person Effect

Centering your DataHow to unconfound within-cluster and between-cluster effects in a Level 1 predictor:1. Cluster-mean center the predictor at Level 1

1. Subtract cluster mean from raw scores in each cluster

2. Create aggregate variable of the Level 1 predictor and enter that as a predictor as well

• Cluster-Centered variable: within-cluster• Aggregate variable: between-cluster

• This will separate the within- and between-cluster effects and based on significance tests, you will know the locus of the effect (within-cluster, between-cluster, or both)

Centering Choices1. I’ve centered my data so that the intercept is equal to the value of my outcome variable when

my predictor is zero in raw units (scale: 0 to 9). Which type of centering have I used?UNCENTERED

1. I’ve centered my data so that the intercept is equal to the average of the corresponding cluster (e.g., my gratitude after nights when I sleep my average amount). Which type of centering have I used?

CLUSTER-MEAN CENTERED2. I’ve centered my data so that the slope of my level 1 predictor reflects the amount the

outcome changes for each unit increase in the predictor scale (e.g., the increase in gratitude participants experience when sleeping for 6 hours instead of 5 hours). Which type of centering have I used?

A OR B3. I’ve centered my data so that the slope of my level 1 predictor reflects the amount the

outcome changes for each unit increase in the predictor relative to the cluster mean (e.g., the increase in gratitude participants experience when sleep one hour more than their average). Which type of centering have I used?

CLUSTER-MEAN CENTERED

Centering your Data

• Resources:• http://web.pdx.edu/~newsomj/mlrclass/ho_centering.pdfAlgina, J., & Swaminathan, H. (2011). Centering in two-level nested designs. In.

J. Hox, & K. Roberts (Eds.), The Handbook of Advanced Multilevel Data Analysis (pp 285-312).New York: Routledge.

Enders, C.K., & Tofighi, D. (2007). Centering predictor variables in cross-sectional multilevel models: A new look at an old issue. Psychological Methods, 12, 121-138.

Wang, L. P., & Maxwell, S. E. (2015). On disaggregating between-person and within-person effects with longitudinal data using multilevel models. Psychological methods, 20(1), 63.

http://web.pdx.edu/%7Enewsomj/mlrclass/ho_centering.pdf

Mediation and Moderation

• Random effects and centering are both important issues to consider when doing mediation or moderation in MLM

• Random effects – make interpreting interactions and computing indirect effects more difficult

• Centering – critical in mediation to unconfound within and between-cluster effects

• If people are less grateful after sleeping poorly only at the aggregate (people who are generally poor sleepers tend to be less grateful, but one night of poor sleep isn’t enough to do anything), then cannot be mediated by daily fluctuations in negative affect

Random Effects and Centering Matter a lot in Moderation & Mediation – See Resources Below

• General resources (both of these sites are great sources of info!):• http://davidakenny.net/cm/mediate.htm• http://www.quantpsy.org/medn.htm

• Moderation in MLM• Bauer & Curran article on probing multilevel moderations:

http://www.unc.edu/~curran/pdfs/Bauer%26Curran(2005).pdf• Barr article on random effects: click here• Calculators: http://www.quantpsy.org/interact/hlm2.htm

• Mediation in MLM• Readable slides from Chris Preacher on multilevel mediation:

http://afhayes.com/public/aps2013.pdf (this has many additional references at the end)• Zhang, Z., Zypher, M. J., & Preacher, K. J. (2009). Testing multilevel mediation using hierarchical

linear models: Problems and solutions. Organizational Research Methods, 12, 695-719.From prior slides:• http://quantpsy.org/pubs/bauer_preacher_gil_2006.pdf (1 model approach to MLM mediation)

• https://njrockwood.com/mlmed/ (multilevel mediation macro for SPSS)• http://www.quantpsy.org/pubs/preacher_zhang_zyphur_2011.pdf (multilevel mediation in

SEM)Calculators from prior slides:

Calculator if 1 or 0 random effects: http://www.quantpsy.org/medmc/medmc.htmCalculator for 1-1-1 with random effects: http://www.quantpsy.org/medmc/medmc111.htm

http://davidakenny.net/cm/mediate.htm

http://www.quantpsy.org/medn.htm

http://www.unc.edu/%7Ecurran/pdfs/Bauer&Curran(2005).pdf

https://www.frontiersin.org/articles/10.3389/fpsyg.2013.00328/full

http://www.quantpsy.org/interact/hlm2.htm

http://afhayes.com/public/aps2013.pdf

http://quantpsy.org/pubs/bauer_preacher_gil_2006.pdf%20(1

https://njrockwood.com/mlmed/

http://www.quantpsy.org/pubs/preacher_zhang_zyphur_2011.pdf

http://www.quantpsy.org/medmc/medmc.htm

http://www.quantpsy.org/medmc/medmc111.htm

Covariance Matrices

Two primary covariance matrices you have to deal with in MLM:• G –covariance matrix for random effects (What are the variances of

the random effects? Are random effects allowed to covary?)• R – underlying error structure (residuals)• Note, these matrices are referred to by different letters in different

software programs

The Random Variance-Covariance Matrix

Models with random slopes can estimate covariance between random effects as well

as random effect variance.

Sleep Quality

Gra

titud

e

Random Effect Covariance Matrix

The Random Variance-Covariance Matrix

Models with random slopes can estimate covariance between random effects as well

as random effect variance.

Intercept Slope1 Slope2

Intercept Var1,1 Cov1,2 Cov1,3

Slope1 Cov1,2 Var2,2 Cov2,3

Slope2 Cov1,3 Cov2,3 Var3,3

How to Structure the Random Covariance Matrix

• Can specify different matrices, here are two common ones: • Unstructured: estimates all variances and covariances

• Sometimes has trouble converging if there are lots of random effects or if some variances/covariances are essentially 0

• Can start here and trim effects that are clearly non-significant• Variance Components: estimates variances but assumes covariances are 0

• Default for SPSS & SAS

• Possible to have multiple RANDOM lines to specify covariancesbetween particular random effects, e.g.,:

• RANDOM X1 X2 | UN• RANDOM X3 X4 | UNNote: can only have a variable appear on ONE random line

The RESIDUAL Variance-Covariance Matrix

Don’t have to specify this matrix when doing MLM, and sometimes doing so will cause error because it will duplicate what is being done with random factors, but occasionally there is additional structure to the error that can be accounted for with this matrix (e.g. repeated measures data).

-Some programs, such as R’s “Lme4” package won’t let you specify this matrix. SPSS and SAS will, R’s nlme package lets you specify a few specific matrices.

Autoregressive Covariance MatrixThis is the type of residual error matrix used with repeated measures. It specifies that time points closer to each other have stronger correlations than time points farther apart. This helps reduce the error or noise since it provides a structure to explain some of the error. Typically the expectation is that the time points are equally spaced (1-2 is the same distance as 3-4).

Day 1 Day 2 Day 3 Day 4

Day 1

Day 2

Day 3

Day 4

The RESIDUAL Variance-Covariance MatrixWith clustered data you have the option of using JUST the residual error matrix to adjust for non-independence of errors.

This approach is called the marginal model, population-averaged model, or generalized estimating equation (GEE) model• These are models that use ONLY the R matrix (residual covariances) to

account for clustering• Adjusts for covariances in residuals without directly estimating variability

between clusters (i.e., no random effects)• Can be useful for repeated measures ANOVAs with missing data or time-

varying covariates

Covariances Matrices: Poll Questions• All variances and covariances are freely estimated

• Unstructured• Variance Components/Diagonal• Autoregressive • Other

• Variances are freely estimated, covariances are constrained to 0• Unstructured• Variance Components/Diagonal• Autoregressive • Other

• Variances are assumed to be equal, covariances have weakening correlations for further apart data points• Unstructured• Variance Components/Diagonal• Autoregressive• Other

• All variances and covariances are constrained to 0• Unstructured• Variance Components/Diagonal• Autoregressive • Other – IF THIS IS A RANDOM MATRIX, THEN IT IS SAYING THERE ARE NO RANDOM EFFECTS! WOULDN’T DO THIS.

Setting Up Data & Sample Syntax

Data Preparation

• Variables from all levels in 1 file• Level 2, 3, etc will just have the same

score repeated for each cluster

• Know levels of variables (is it level 1, level 2?)

• Centering • (for some programs such as SPSS, must

be done ahead of times, for others (e.g., R) can define it as part of model syntax

ID Day Happy Grateful AvgSleep1 0 3 4 6.171 1 2 3 6.171 2 5 6 6.171 3 3 6 6.171 4 4 4 6.172 0 1 4 5.332 1 4 5 5.332 2 3 4 5.332 3 2 3 5.332 4 3 4 5.333 0 2 3 0.93 1 3 2 0.93 2 3 3 0.93 3 3 2 0.93 4 2 3 0.9

Data Preparation

• Cross-Classified DataID Target Liking Morality Neurot Targ_Attract

1 0 7 3 5.2 31 2 8 4 5.2 41 4 7 5 5.2 51 6 9 3 5.2 21 8 8 3 5.2 52 1 6 5 3 62 3 7 6 3 32 5 9 6 3 12 7 6 6 3 22 9 5 5 3 53 0 4 4 3.5 33 1 7 6 3.5 63 2 6 5 3.5 43 3 8 4 3.5 33 4 6 6 3.5 5

Some Common MLM Programs

• SPSS: Mixed Models (linear and generalized)• SAS: Proc Mixed• R: LME4 NLME (may be other packages)

• https://stats.stackexchange.com/questions/5344/how-to-choose-nlme-or-lme4-r-library-for-mixed-effects-models

• http://glmm.wikidot.com/pkg-comparison

• HLM (free student version available online)• Mplus (can handle MLM SEM)

https://stats.stackexchange.com/questions/5344/how-to-choose-nlme-or-lme4-r-library-for-mixed-effects-models

http://glmm.wikidot.com/pkg-comparison

Sample Syntax: SPSS

2-LevelMIXED DV WITH Predictor 1/CRITERIA=CIN(95) MXITER(100) MXSTEP(10) SCORING(1) SINGULAR(0.000000000001)

HCONVERGE(0, ABSOLUTE) LCONVERGE(0, ABSOLUTE) PCONVERGE(0.000001, ABSOLUTE)

/FIXED=Predictor1 | SSTYPE(3)

/METHOD=REML/PRINT=SOLUTION TESTCOV

/RANDOM=INTERCEPT Predictor1 | SUBJECT(ID) COVTYPE(UN).

all unstructured random variance-covariance matrix

Sample Syntax: SPSS

3-LevelMIXED DV WITH Predictor 1/CRITERIA=CIN(95) MXITER(100) MXSTEP(10) SCORING(1) SINGULAR(0.000000000001)



/METHOD=REML/PRINT=SOLUTION TESTCOV/RANDOM=INTERCEPT Predictor1 | SUBJECT(ID) COVTYPE(UN)

/RANDOM=INTERCEPT Predictor 1 | SUBJECT(ID*DAY) COVTYPE(UN).


Sample Syntax: SPSS

CROSS-CLASSIFIEDMIXED DV WITH Predictor 1/CRITERIA=CIN(95) MXITER(100) MXSTEP(10) SCORING(1) SINGULAR(0.000000000001)



/METHOD=REML/PRINT=SOLUTION TESTCOV/RANDOM=INTERCEPT Predictor1 | SUBJECT(ID) COVTYPE(UN)

/RANDOM=INTERCEPT Predictor 1 | SUBJECT(TARGET) COVTYPE(UN).


Sample Syntax: SAS

2-LevelPROC MIXED data=data COVTEST;

CLASS ID;MODEL DV = Predictor1/CL S DDFM=satterth;RANDOM INTERCEPT Predictor1 / SUB=ID TYPE=UN;

RUN;


Sample Syntax: SAS

3-LevelPROC MIXED data=data COVTEST;

CLASS DAY ID;MODEL DV = Predictor1/CL S DDFM=satterth;RANDOM INTERCEPT Predictor1 / SUB=DAY(ID) TYPE=UN;

RANDOM INTERCEPT Predictor1 / SUB=ID TYPE=UN;RUN;


Sample Syntax: SAS

CROSS-CLASSIFIEDPROC MIXED data=data COVTEST;

CLASS ID TARGET;MODEL DV = Predictor1/CL S DDFM=satterth;RANDOM INTERCEPT Predictor1 / SUB=ID TYPE=UN;

RANDOM INTERCEPT Predictor1 / SUB=TARGET TYPE=UN;RUN;


Sample Syntax: R

2-LEVEL• Lmermodel <-lmer(DV ~ Predictor1 + (1 + Predictor1|ID), data=data, na.action = "na.exclude") summary (model)

• Nlmemodel <- lme(DV ~ Predictor1, random = ~Predictor1|ID, na.action = "na.exclude", data=data)summary(model)

Sample Syntax: R

3-LEVEL• Lmermodel <-lmer(DV ~ Predictor1 + (1 + Predictor1|ID:DAY) + (1 + Predictor1|ID), data=data, na.action = "na.exclude") summary (model)

• Nlmemodel <- lme(DV ~ Predictor1, random = list(ID = ~Predictor1, DAY = ~Predictor1), na.action = "na.exclude", data=data)summary(model)

Sample Syntax: R

CROSS-CLASSIFIED• Lmermodel <-lmer(DV ~ Predictor1 + (1 + Predictor1|ID) + (1 + Predictor1|Target),data=data, na.action = "na.exclude") summary (model)

• NlmeCrossed models are slow

Notes on Syntax

• These are very basic syntax with 1 predictor there are LOTS of other choices you can make with these syntax and much more complicated models you can specify (mediation, moderation, growth-curve modeling, etc)

• Models assume random effects for every random factor BUT:• In 3 level models: could choose to only allow the predictor to have random effect at

one level (usually a lower level)• See whether effect of stress on well-being (measured several times a day) varies from day to

day (random effect for day), but not person to person (no random effect for ID)• In cross-classified models: could choose to only allow predictor to have random

effect for one random factor• See whether association between participant’s liking of target and rating of target morality

differs from participant to participant (random effect for ID), but not target to target (no random effect for TARGET)

• WHY DO THIS? If you are not interested in the effect of target and find that allowing for the random effect of liking for target (relationship between liking and morality is different for different targets) doesn’t increase model fit

Interpreting Basic Output

Interpreting Basic Output – Intercept OnlyFIXED EFFECTS

RANDOM EFFECTSResidual error

Random intercept (variability in appreciation by person)

ICC (also known as rho/ρ) = between variance + within (error) variance / total varianceTOTAL: 1.15 + 1.18 = 2.33ICC: 1.18/2.33 = .51

DFs are calculated using the Satterthwaiteapproximation which yields degrees of freedom that are somewhere between the number of repeated measures and the number of individuals depending on ICC (lower ICC = higher DFs)

Interpreting Basic Output -grand-mean centered level 1 predictor

FIXED EFFECTS

RANDOM EFFECTS

Random variances and covariances(Unstructured Matrix)

UN 1,1, is intercept variance, UN 2,2 is slope variance for sleep_GranCent, UN 2,1, is their covariance

Interpreting Basic Output -cluster-mean centered level 1 predictor & grand-mean centered aggregate at level 2

FIXED EFFECTS

RANDOM EFFECTS

Random variances and covariances(Unstructured Matrix)

UN 1,1, is intercept variance, UN 2,2 is slope variance for sleep_withinperson, UN 2,1, is their covarianceNO RANDOM EFFECT FOR Sleep_Between Person BECAUSE IT IS A LEVEL 2 VARIABLE (ONE SCORE PER PERSON/CLUSTER)

Power & Sample Size Calculations in MLM

It can be complicated. There is more out there than what I am presenting. But this should help you be aware of the issues you’ll face.

Calculating Power/Sample Size in MLM• Have to consider power/sample size for each level of data

• How many participants/groups to collect?• How many days/members of group to collect? (i.e., cluster size)

• Gain more power by increasing upper level units• 100 participants with 5 days of data = more power• 50 participants with 10 days of data

• ICC matters: if ICC is very low, then increasing the cluster size is useful, if very high then additional days/members of group etc. are not useful

Calculating Power/Sample Size in MLM• One approach is using the DESIGN EFFECT:

neffective = n/[1+(ncluster - 1)*ρ]• neffective = effective sample size• n= total sample size• ncluster = number of days/people in group etc. (e.g.,7 for a week-long diary)• ρ = ICC for that variableThis tells you for a given sample size, what your effective sample size is. Ex: If ρ = 1 (no new info gained from additional level 1 units), then sample size is number of level 2 units. Ex: If ρ = 0 (no actual nesting/non-independence between level 1 units), then sample size is number of level 1 units. In actuality, will likely end up somewhere in between

• To figure out needed sample size, use ICC from pilot testing/prior research to determine total sample size needed given effective sample size

n= neffective(1+(ncluster - 1)*ρ]

Calculating Power/Sample Size in MLM

• PINT: calculator for power in two level models• https://www.stats.ox.ac.uk/~snijders/multilevel.htm

Written by Tom Snijders, Roel Bosker, and Henk Guldemond

https://www.stats.ox.ac.uk/%7Esnijders/multilevel.htm

A Few Online Resources for Multilevel Modeling• UCLA Resources intro to MLM:

• https://stats.idre.ucla.edu/other/mult-pkg/introduction-to-linear-mixed-models/

• The Analysis Factor• https://www.theanalysisfactor.com/resources/by-topic/mixed-multilevel-models/

• Dave Kenny’s Website (great for dyadic data):• http://davidakenny.net/

• Conducting MLM analyses in R:• http://cran.r-project.org/doc/contrib/Bliese_Multilevel.pdf• https://www.jaredknowles.com/journal/2013/11/25/getting-started-with-mixed-effect-models-in-r

• Guide to Linear Mixed Models in SPSS:• http://www.spss.ch/upload/1126184451_Linear%20Mixed%20Effects%20Modeling%20in%20SPSS.pdf

• A recent Psych Methods article with practical advice for novices:• http://psycnet.apa.org/doiLanding?doi=10.1037%2Fmet0000159

• Recommended Books: • Multilevel Analysis: Techniques and Applications by Joop Hox• Intensive Longitudinal Methods (Bolger & Laurenceau, 2013)

https://stats.idre.ucla.edu/other/mult-pkg/introduction-to-linear-mixed-models/

https://www.theanalysisfactor.com/resources/by-topic/mixed-multilevel-models/

http://davidakenny.net/

http://cran.r-project.org/doc/contrib/Bliese_Multilevel.pdf

https://www.jaredknowles.com/journal/2013/11/25/getting-started-with-mixed-effect-models-in-r

http://www.spss.ch/upload/1126184451_Linear%20Mixed%20Effects%20Modeling%20in%20SPSS.pdf

http://psycnet.apa.org/doiLanding?doi=10.1037/met0000159

a practical guide to multilevel modeling

Documents