a gentle introduction to linear mixed modeling and proc mixed richard charnigo

A Gentle Introduction to Linear Mixed Modeling and PROC MIXED

Richard CharnigoAssociate Professor of Statistics and BiostatisticsDirector of Statistics and Psychometrics Core, [email protected]

Objectives

First hour:

1. Be able to formulate linear mixed models forlongitudinal data involving a categorical and a continuous covariate.

2. Understand how linear mixed modeling goes beyond linear regression and repeated measures ANOVA.

Second hour:

3. Be able to use PROC MIXED to fit a linear mixed model for longitudinal data involving acategorical and a continuous covariate.

Motivating example

The Excel file at {www.richardcharnigo.net/mixed}contains a simulated data set:

Two hundred college freshmen (“ID”) who drink alcohol are asked to estimate the number of drinks consumed during the preceding year. From this number we obtain an estimate of the average weekly number of drinks (“Drink”).

The students are also assessed on negative urgency; the results are expressed as Z scores (“NegUrg”).

One and two years later (“Time”), the students supply updated estimates of their drinking.

Motivating example

Two obvious “research questions” are:

i. Is there an association between negative urgency and drinking at baseline ?

ii. Does drinking tend to change over time and, if so, is the change predicted by negative urgency at baseline ?

Of course, we can envisage more complicated and realistic scenarios ( e.g., with additional personality variables and/or interventions ), but this simple scenario will help us get a hold of linear mixed modeling and PROC MIXED.

Exploratory data analysis

Before pursuing linear mixed (or other statistical) modeling, we are well-advised to engage in exploratory data analysis.

This can alert us to any gross mistakes in the data set, heretofore undetected, which may compromise our work.

This can also suggest a structure for the linear mixed model and help us to anticipate what the results should be.

Exploratory data analysisA

ver

age

# o

f D

rin

ks

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22

Negative Urgency

-3 -2 -1 0 1 2 3

Drinking vs Negative Urgency (freshman year)


ver

age

# o

f D

rin

ks

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22

Negative Urgency

-3 -2 -1 0 1 2 3

Drinking vs Negative Urgency (sophomore year)


ver

age

# o

f D

rin

ks

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22

Negative Urgency

-3 -2 -1 0 1 2 3

Drinking vs Negative Urgency (junior year)


ver

age

# o

f D

rin

ks

0

1

2

3

4

5

6

7

8

Year

Freshman Sophomore Junior

Means and standard errors of drinking by negative urgency and year

Negative urgency strata: Low Average High

Exploratory data analysis

The scatterplots suggest the following:

• There are some outlying values, and drinking is not normally distributed, but there are not any values that are obviously fabricated or miskeyed.

• There appears to be a positive association between drinking and negative urgency at baseline, which strengthens over time as those higher in negative urgency seem to be drinking more in later years.

The latter impression is also conveyed by the plot of means and standard errors.

A first linear mixed model

We will log-transform drinking before fitting any linear mixed models, since linear mixed modeling assumes approximate normality of the outcome variable at fixed values of the predictor variables.

Hereafter let Yjk denote subject j’s log-transformed drinking score at time k. Consider these three equations:

Yjk = a0 + a1 k + error, if subject j is lowYjk = b0 + b1 k + error, if subject j is averageYjk = c0 + c1 k + error, if subject j is high

on negative urgency.


Three comments are in order:

First, we are in essence regressing (log-transformed) drinking on time but allowing each subject to have one of three intercepts and one of three slopes, according to his/her negative urgency.

Second, our research questions amount to asking whether a0 , b0 , c0 differ from each other, whether a1 , b1 , c1 differ from zero, andwhether a1 , b1 , c1 differ from each other.


Third, the linear mixed model defined by the three equations can be expressed as a linear regression model. Let X1 and X2 respectively be dummy variables for low and high negative urgency.

Then we may write

Yjk = b0 + (a0 – b0) X1j + (c0 – b0) X2j +( b1 + (a1 – b1) X1j + (c1 – b1) X2j ) k + error.


Now let us examine the results from fitting the linear mixed model using PROC MIXED.

We see that PROC MIXED used all available observations ( 540 ), including observations from subjects who dropped out early ( 60 ). Along with accommodating a continuous covariate ( time ), this is why linear mixed modeling goes beyond a standard repeated measures ANOVA.

Number of ObservationsNumber of Observations Read 540

Number of Observations Used 540

Number of Observations Not Used 0


The variance of the error term is estimated to be 0.41. The estimates of the intercepts a0 , b0 , c0 are 0.87, 1.05, and 1.16. The estimates of the slopes a1 , b1 , c1 are -0.13, 0.08, and 0.27.

Covariance Parameter Estimates

Cov Parm EstimateStandard

ErrorZ

Value Pr > ZResidual 0.4126 0.02525 16.34 <.0001

Solution for Fixed Effects

Effect negurgstratum EstimateStandard

Error DF t Value Pr > |t|negurgstratum 0 0.8726 0.08812 534 9.90 <.0001negurgstratum 1 1.0536 0.05869 534 17.95 <.0001negurgstratum 2 1.1605 0.08143 534 14.25 <.0001Time*negurgstratum 0 -0.1309 0.07166 534 -1.83 0.0683Time*negurgstratum 1 0.08094 0.04769 534 1.70 0.0902Time*negurgstratum 2 0.2688 0.06580 534 4.08 <.0001


We can also use PROC MIXED to estimate any linear combinations of a0 , b0 , c0 , a1 , b1 , c 1. For example, below are estimates of

c0 – a0 ( high vs. low negative urgency freshmen ),( c0 + c1 ) – ( a0 + a1 )( high vs. low negative urgency sophomores ), and( c0 + 2c1 ) – ( a0 + 2a1 )( high vs. low negative urgency juniors ).

Estimates

Label EstimateStandard

Error DF t Value Pr > |t|High vs low freshman 0.2879 0.1200 534 2.40 0.0168

High vs low sophomore 0.6876 0.07930 534 8.67 <.0001

High vs low junior 1.0873 0.1308 534 8.31 <.0001

A second linear mixed model

As noted earlier, our first linear mixed model can be expressed as a linear regression model. How, then, does linear mixed modeling go beyond linear regression ?

The answer is that we may also allow each subject to have his/her own personal intercept and slope, not merely choose from among three intercepts and three slopes. The personal intercept and slope may be related to negative urgency and to unmeasured or unmeasurable characteristics.


More specifically, we may propose the following:

Yjk = b0 + (a0 – b0) X1j + (c0 – b0) X2j + P1j

( b1 + (a1 – b1) X1j + (c1 – b1) X2j + P2j ) k + error.

Above, P1j and P2j are unobserved zero-mean variables that adjust the intercept and slope for subject j. Thus, the interpretations of a0 , b0 , c0 , a1 , b1 , c1 are subtly altered. They are now the average intercepts and slopes for subjects who are low, average, and high on negative urgency.

Even so, our research questions are still addressed by estimating a0 , b0 , c0 , a1 , b1 , c1.


While we can “predict” P1j and P2j from the data, in practice this is rarely done. However, their variances and covariance are routinely estimated.


Cov Parm Subject EstimateStandard

ErrorZ

Value Pr ZUN(1,1) ID 0.02333 0.03466 0.67 0.2505UN(2,1) ID 0.007716 0.02430 0.32 0.7509UN(2,2) ID 0.07522 0.02771 2.71 0.0033Residual 0.2621 0.02868 9.14 <.0001


Effect negurgstratum EstimateStandard

Error DF t Value Pr > |t|negurgstratum 0 0.8689 0.07399 160 11.74 <.0001negurgstratum 1 1.0573 0.04924 160 21.47 <.0001negurgstratum 2 1.1633 0.06829 160 17.04 <.0001Time*negurgstratum 0 -0.1245 0.07279 160 -1.71 0.0892Time*negurgstratum 1 0.06846 0.04862 160 1.41 0.1610Time*negurgstratum 2 0.2580 0.06705 160 3.85 0.0002


Which model is better: the first or second ?

Conceptually, the second model is appealing because P1j and P2j induce correlations among the repeated observations on subject j. Thus, we avoid the unrealistic assumption, present in linear regression, that observations are independent.

Empirically, we may examine a model selection criterion such as the BIC; a smaller value is better. Here are results for the first and second models.

Fit Statistics-2 Res Log Likelihood 1072.2

AIC (smaller is better) 1074.2

AICC (smaller is better) 1074.2

BIC (smaller is better) 1078.5





A third linear mixed model

So far we have treated negative urgency as categorical, but this is not necessary and perhaps not optimal. Let us now consider the following:

Yjk = ( d0 + e0 Nj + P1j ) + ( d1 + e1 Nj + P2j ) k + error.

Above, Nj denotes the continuous negative urgency variable, while P1j and P2j are, as before, adjustments to the intercept and slope.


Since negative urgency was expressed as a Z score, d0 and d1 are the average intercept and slope among those average on negative urgency.

Likewise, d0 + e0 and d1 + e1 are the average intercept and slope among those one standard deviation above average on negative urgency.

And, d0 – e0 and d1 – e1 are the average intercept and slope among those one standard deviation below average on negative urgency.


We estimate the variances and covariance of P1j and P2j as well as estimating d0 , e0 , d1 , e1 .


Cov Parm Subject EstimateStandard

ErrorZ

Value Pr ZUN(1,1) ID 0.01952 0.03453 0.57 0.2859

UN(2,1) ID 0.005827 0.02425 0.24 0.8101

UN(2,2) ID 0.07199 0.02743 2.62 0.0043

Residual 0.2633 0.02885 9.13 <.0001


Effect EstimateStandard

Error DF t Value Pr > |t|Intercept 1.0414 0.03495 198 29.79 <.0001

NegUrg 0.1152 0.03613 160 3.19 0.0017

Time 0.07230 0.03438 178 2.10 0.0369

NegUrg*Time 0.1446 0.03533 160 4.09 <.0001


In addition, we may estimate linear combinations ofd0 , e0 , d1 , e1 . For example, 2e0 compares freshmen one standard deviation above to freshmen one standard deviation below, 2e0 + 2e1 compares sophomores, and 2e0 + 4e1 compares juniors. Moreover, the BIC prefers this model over either of the first two.

Estimates

Label EstimateStandard

Error DF t Value Pr > |t|High vs low freshman 0.2304 0.07225 160 3.19 0.0017

High vs low sophomore 0.5196 0.06747 160 7.70 <.0001

High vs low junior 0.8087 0.1178 160 6.87 <.0001





What’s next ?

After a short break, we will launch SAS and examine the PROC MIXED implementations of the three linear mixed models as well as the exploratory data analyses that preceded them.

In addition to replicating the results shown in this presentation, we will discuss some potentially useful modifications of and additions to the SAS code ( e.g., how to estimate other linear combinations of coefficients in PROC MIXED).

a gentle introduction to linear mixed modeling and proc mixed richard charnigo

Documents

linear mixed modelwe

hold of linear mixed

linear regression

drinking score

simulated data set

highon negative urgency

statistical modeling

subject js log