1. A Gentle Introduction to Linear Mixed Modeling and PROC MIXED
Richard Charnigo
Associate Professor of Statistics and Biostatistics
Director of Statistics and Psychometrics Core, CDART

2. Objectives First hour: 1. Be able to formulate linear mixed models for
longitudinal data involving a categorical and a
continuous covariate.
2. Understand how linear mixed modeling goes beyond linear regression and repeated measures ANOVA.
Second hour:
3. Be able to use PROC MIXED to fit a linear
mixed model for longitudinal data involving a
categorical and a continuous covariate.

3. Motivating example The Excel file at {www.richardcharnigo.net/mixed}
contains a simulated data set:
Two hundred college freshmen (“ID”) who drink alcohol are asked to estimate the number of drinks consumed during the preceding year. From this number we obtain an estimate of the average weekly number of drinks (“Drink”).
The students are also assessed on negative urgency; the results are expressed as Z scores (“NegUrg”).
One and two years later (“Time”), the students supply updated estimates of their drinking.

4. Motivating example Two obvious “research questions” are:
Is there an association between negative urgency and drinking at baseline ?
Does drinking tend to change over time and, if so, is the change predicted by negative urgency at baseline ?
Of course, we can envisage more complicated and realistic scenarios ( e.g., with additional personality variables and/or interventions ), but this simple scenario will help us get a hold of linear mixed modeling and PROC MIXED.

5. Exploratory data analysis
Before pursuing linear mixed (or other statistical) modeling, we are well-advised to engage in exploratory data analysis.
This can alert us to any gross mistakes in the data set, heretofore undetected, which may compromise our work.
This can also suggest a structure for the linear mixed model and help us to anticipate what the results should be.

6. Exploratory data analysis

7. Exploratory data analysis

8. Exploratory data analysis

9. Exploratory data analysis

10. Exploratory data analysis
The scatterplots suggest the following:
There are some outlying values, and drinking is not normally distributed, but there are not any values that are obviously fabricated or miskeyed.
There appears to be a positive association between drinking and negative urgency at baseline, which strengthens over time as those higher in negative urgency seem to be drinking more in later years.
The latter impression is also conveyed by the plot of means and standard errors.

11. A first linear mixed model
We will log-transform drinking before fitting any linear mixed models, since linear mixed modeling assumes approximate normality of the outcome variable at fixed values of the predictor variables.
Hereafter let Yjk denote subject j’s log-transformed drinking score at time k. Consider these three equations:
Yjk = a0 + a1 k + error, if subject j is low
Yjk = b0 + b1 k + error, if subject j is average
Yjk = c0 + c1 k + error, if subject j is high
on negative urgency.

12. A first linear mixed model
Three comments are in order:
First, we are in essence regressing (log-transformed) drinking on time but allowing each subject to have one of three intercepts and one of three slopes, according to his/her negative urgency.
Second, our research questions amount to asking whether a0 , b0 , c0 differ from each other, whether a1 , b1 , c1 differ from zero, and
whether a1 , b1 , c1 differ from each other.

13. A first linear mixed model
Third, the linear mixed model defined by the three equations can be expressed as a linear regression model. Let X1 and X2 respectively be dummy variables for low and high negative urgency.
Then we may write
Yjk = b0 + (a0 – b0) X1j + (c0 – b0) X2j +
( b1 + (a1 – b1) X1j + (c1 – b1) X2j ) k + error.

14. A first linear mixed model
Now let us examine the results from fitting the linear mixed model using PROC MIXED.
We see that PROC MIXED used all available observations ( 540 ), including observations from subjects who dropped out early ( 60 ). Along with accommodating a continuous covariate ( time ), this is why linear mixed modeling goes beyond a standard repeated measures ANOVA.
Number of Observations
Number of Observations Read
540
Number of Observations Used
Number of Observations Not Used

15. A first linear mixed model
The variance of the error term is estimated to be The estimates of the intercepts a0 , b0 , c0 are 0.87, 1.05, and The estimates of the slopes a1 , b1 , c1 are , 0.08, and
Covariance Parameter Estimates
Cov Parm
Estimate
Standard Error
Z Value
Pr > Z
Residual
0.4126
16.34
<.0001
Solution for Fixed Effects
Effect
negurgstratum
Estimate
Standard Error DF
t Value
Pr > |t|
0.8726
534
9.90
<.0001 1
1.0536
17.95 2
1.1605
14.25
Time*negurgstratum
-1.83
0.0683
1.70
0.0902
0.2688
4.08

16. A first linear mixed model
We can also use PROC MIXED to estimate any linear combinations of a0 , b0 , c0 , a1 , b1 , c 1. For example, below are estimates of
c0 – a0
( high vs. low negative urgency freshmen ),
( c0 + c1 ) – ( a0 + a1 )
( high vs. low negative urgency sophomores ), and
( c0 + 2c1 ) – ( a0 + 2a1 )
( high vs. low negative urgency juniors ).
Estimates
Label
Estimate
Standard Error DF
t Value
Pr > |t|
High vs low freshman
0.2879
0.1200
534
2.40
0.0168
High vs low sophomore
0.6876
8.67
<.0001
High vs low junior
1.0873
0.1308
8.31

17. A second linear mixed model
As noted earlier, our first linear mixed model can be expressed as a linear regression model. How, then, does linear mixed modeling go beyond linear regression ?
The answer is that we may also allow each subject to have his/her own personal intercept and slope, not merely choose from among three intercepts and three slopes. The personal intercept and slope may be related to negative urgency and to unmeasured or unmeasurable characteristics.

18. A second linear mixed model
More specifically, we may propose the following:
Yjk = b0 + (a0 – b0) X1j + (c0 – b0) X2j + P1j
( b1 + (a1 – b1) X1j + (c1 – b1) X2j + P2j ) k + error.
Above, P1j and P2j are unobserved zero-mean variables that adjust the intercept and slope for subject j. Thus, the interpretations of a0 , b0 , c0 , a1 , b1 , c1 are subtly altered. They are now the average intercepts and slopes for subjects who are low, average, and high on negative urgency.
Even so, our research questions are still addressed by estimating a0 , b0 , c0 , a1 , b1 , c1.

19. A second linear mixed model
While we can “predict” P1j and P2j from the data, in practice this is rarely done. However, their variances and covariance are routinely estimated.
Covariance Parameter Estimates
Cov Parm
Subject
Estimate
Standard Error
Z Value
Pr Z
UN(1,1) ID
0.67
0.2505
UN(2,1)
0.32
0.7509
UN(2,2)
2.71
0.0033
Residual
0.2621
9.14
<.0001
Solution for Fixed Effects
Effect
negurgstratum
Estimate
Standard Error DF
t Value
Pr > |t|
0.8689
160
11.74
<.0001 1
1.0573
21.47 2
1.1633
17.04
Time*negurgstratum
-1.71
0.0892
1.41
0.1610
0.2580
3.85
0.0002

20. A second linear mixed model
Which model is better: the first or second ?
Conceptually, the second model is appealing because P1j and P2j induce correlations among the repeated observations on subject j. Thus, we avoid the unrealistic assumption, present in linear regression, that observations are independent.
Empirically, we may examine a model selection criterion such as the BIC; a smaller value is better. Here are results for the first and second models.
Fit Statistics
-2 Res Log Likelihood
1072.2
AIC (smaller is better)
1074.2
AICC (smaller is better)
BIC (smaller is better)
1078.5
Fit Statistics
-2 Res Log Likelihood
1014.2
AIC (smaller is better)
1022.2
AICC (smaller is better)
1022.3
BIC (smaller is better)
1035.4

21. A third linear mixed model
So far we have treated negative urgency as categorical, but this is not necessary and perhaps not optimal. Let us now consider the following:
Yjk = ( d0 + e0 Nj + P1j ) + ( d1 + e1 Nj + P2j ) k + error.
Above, Nj denotes the continuous negative urgency variable, while P1j and P2j are, as before, adjustments to the intercept and slope.

22. A third linear mixed model
Since negative urgency was expressed as a Z score, d0 and d1 are the average intercept and slope among those average on negative urgency.
Likewise, d0 + e0 and d1 + e1 are the average intercept and slope among those one standard deviation above average on negative urgency.
And, d0 – e0 and d1 – e1 are the average intercept and slope among those one standard deviation below average on negative urgency.

23. A third linear mixed model
We estimate the variances and covariance of P1j and P2j as well as estimating d0 , e0 , d1 , e1 .
Covariance Parameter Estimates
Cov Parm
Subject
Estimate
Standard Error
Z Value
Pr Z
UN(1,1) ID
0.57
0.2859
UN(2,1)
0.24
0.8101
UN(2,2)
2.62
0.0043
Residual
0.2633
9.13
<.0001
Solution for Fixed Effects
Effect
Estimate
Standard Error DF
t Value
Pr > |t|
Intercept
1.0414
198
29.79
<.0001
NegUrg
0.1152
160
3.19
0.0017
Time
178
2.10
0.0369
NegUrg*Time
0.1446
4.09

24. A third linear mixed model
In addition, we may estimate linear combinations of
d0 , e0 , d1 , e1 . For example, 2e0 compares freshmen one standard deviation above to freshmen one standard deviation below, 2e0 + 2e1 compares sophomores, and 2e0 + 4e1 compares juniors. Moreover, the BIC prefers this model over either of the first two.
Estimates
Label
Estimate
Standard Error DF
t Value
Pr > |t|
High vs low freshman
0.2304
160
3.19
0.0017
High vs low sophomore
0.5196
7.70
<.0001
High vs low junior
0.8087
0.1178
6.87
Fit Statistics
-2 Res Log Likelihood
1004.5
AIC (smaller is better)
1012.5
AICC (smaller is better)
1012.6
BIC (smaller is better)
1025.7

25. What’s next ?
After a short break, we will launch SAS and examine the PROC MIXED implementations of the three linear mixed models as well as the exploratory data analyses that preceded them.
In addition to replicating the results shown in this presentation, we will discuss some potentially useful modifications of and additions to the SAS code ( e.g., how to estimate other linear combinations of coefficients in PROC MIXED).