issues with mixed models. model doesn’t converge… or
TRANSCRIPT
Issues withMixed Models
Model doesn’t converge…
OR
Convergence
Likelihood Landscape
Likelihood Landscape
Maximum Likelihood Estimation
Mea
n
Maximum Likelihood Estimation
Mea
n
Maximum Likelihood Estimation
Mea
n
Maximum Likelihood Estimation
Mea
n
Maximum Likelihood Estimation
Likelihood = the probability of seeing the data we actually collected given a particular model
Maximum Likelihood Estimates = those values that make the observed data most likely to have happened
Sources of Convergence Problems
• You estimate more parameters than data (or, in general, too many parameters
• Severe collinearity (e.g., two predictors are exactly correlated)
• Missing cells in your design
• Predictors of vastly different metrics
Failure to converge
GENDERATTITUDE male
femalepolite 16
0informal 1632
… and then trying to test the ATTITUDE*GENDER interaction
How can this happen?
“Death by Design”
(coined byRoger Mundry)
designanalysis
Solutions to Convergence Problems
• Drop a random slope(not preferred, should be reported)
• Drop subjects/items for which there is not enough data (not preferred, should be reported)
• Rescale variables so that they lie range between 0 and 1; or make them on similar metrics overall
• Center continuous predictors
• Nonlinear transformations of skewed predictors
Solutions to Convergence Problems
• Change order of variable names in model formula
• Have a balanced and complete design
p-values
The p-value conundrum
What are the degrees of freedom?
How to get p-values out ofmixed models is not entirely straightforward…
DouglasBates
“There are a number of ways to compute p-values from LMEMs, none of which is
uncontroversially the best.”
Barr et al. (2013)
Ways to get p-values
• t-test/F-test with normal approximation
• Likelihood Ratio Test
• Boostrapping
• Permutation
• Markov Chain Monte Carlo (MCMC)
Getting p-vals with normal approximation
xmdlcoefs=data.frame(summary(xmdl)@coefs)coefs$p = 2*(1-pnorm(abs(coefs$t.value)))coefs
Function for getting p-vals with normal approximation
create.sig.table = function(x){
coefs=data.frame(summary(x)@coefs)coefs$p = 2*(1-pnorm(abs(coefs$t.value)))coefs$sig = character(nrow(coefs))coefs[which(coefs$p < 0.05),]$sig = "*"coefs[which(coefs$p < 0.01),]$sig = "**"coefs[which(coefs$p < 0.001),]$sig = "***"return(coefs)
}
Likelihood Ratio Test
First model needs to be nested in second
Likelihood Ratio
The likelihood ratio expresses how many times more likely the data are under one model than the other
Mea
n
Mea
n
Likelihood Ratio Test
Likelihood Ratio Test
Important whendoing likelihood ratio tests
lmer(…,REML=FALSE)
http://anythingbutrbitrary.blogspot.com/2012/06/random-regression-coefficients-using.html
Final issue:Random slopes
DANGEROUS!!!
Random intercept onlymodels are known to bevery anti-conservativein many circumstances
(cf. Barr et al., 2013,Schielzeth & Forstmeier, 2008)
Schielzeth & Forstmeier (2008)
Random intercept only
Type I error simulation
10 subjects
10 data points each
5 of those in condition A,5 in B
LRT intercept ML 0.052LRT slope ML 0.035LRT intercept REML 0.052LRT slope REML 0.035
z-test intercept ML 0.053z-test slope ML 0.039z-test intercept REML 0.054z-test slope REML 0.042
Add to this explicit subject slopesfor A/B
10 subjects
10 data points each
5 of those in condition A,5 in B
LRT intercept ML 0.24LRT slope ML 0.15LRT intercept REML 0.24LRT slope REML 0.069
z-test intercept ML 0.24z-test slope ML 0.079z-test intercept REML 0.25z-test slope REML 0.091
Add to this explicit subject slopesfor A/B
10 subjects
10 data points each
5 of those in condition A,5 in B
LRT intercept ML 0.24LRT slope ML 0.15LRT intercept REML 0.24LRT slope REML 0.069
z-test intercept ML 0.24z-test slope ML 0.079z-test intercept REML 0.25z-test slope REML 0.091
Add to this explicit subject slopesfor A/B + take item slopes
10 subjects
10 data points each
5 of those in condition A,5 in B
LRT intercept ML 0.18LRT slope ML 0.085LRT intercept REML 0.18LRT slope REML 0.052
z-test intercept ML 0.21z-test slope ML 0.064z-test intercept REML 0.23z-test slope REML 0.08
LRT intercept ML 0.18LRT slope ML 0.085LRT intercept REML 0.18LRT slope REML 0.052
z-test intercept ML 0.21z-test slope ML 0.064z-test intercept REML 0.23z-test slope REML 0.08
Add to this explicit subject slopesfor A/B + take item slopes
10 subjects
10 data points each
5 of those in condition A,5 in B
“Keep it maximal”
“Keep it maximal”
random effectsjustified by the design
vs.
random effectsjustified by the data
Barr et al. (2013)
“Keep it maximal”
“for whatever fixed effects are of critical interest, the
corresponding random effects should be in that analysis”
Barr et al. (2013)
That’s it(for now)