1. Multilevel Linear Modeling aka HLM

2. The Design We have data at two different levels In this case, 7,185 students (Level 1) Nested within 160 Schools (Level 2) We want to investigate effects at both levels.

3. Get the Data Download data file in XLS formatdata file in XLS format Print the Cheat-SheetCheat-Sheet Boot up SAS and import the data name the imported member “HLM” The first analysis will be have no predictors, only means.

4. Level 1 Equation The math achievement of the i th student at the j th school = the intercept for the j th school (the mean at that school) + error. Note: We are using “  0 ” instead of “a” to stand for intercept.

5. Level 2 Equation The intercept (mean match achievement) at the j th school = The average intercept (mean) across schools + (group j intercept) – (overall intercept) –the effect of being in the j th school.

6. Combine the Two Equations Substitute (  00 +  0j ), from the Level 2 equation, for  0j in the Level 1 equation A student’s score = The average intercept across schools + the effect of being at the j th school + other stuff

7. SAS title 'Model 1: Unconditional Means Model, Intercepts Only'; options formdlim='-' pageno=min nodate; proc mixed data = covtest noclprint; class School; model MathAch = / solution; random intercept / subject = School; run;

8. Fixed Effects Effects that are constant across schools. model MathAch =/ No effect follows "=," the only parameter estimated will be mean across schools. Solution for Fixed Effects EffectEstimateStandard Error DFt ValuePr > |t| Intercept12.63700.244315951.72<.0001

9. Random Effects Effects that vary across schools. random intercept / subject = School; There is significant variance across schools (8.6097) in intercepts. And among students within schools (39.1487) Covariance Parameter Estimates Cov ParmSubjectEstimateStandard Error Z ValuePr > Z InterceptSchool8.60971.07787.99<.0001 Residual 39.14870.660759.26<.0001

10. Intraclass Correlation the proportion of the variance in MathAch that is due to differences among schools = (that due to schools) / (total variance) = 8.6097 / (8.6097 + 39.1487) = 18%.  Next, we are going to add a Level 2 predictor, MeanSES. Do note that this variable has been centered to mean 0.

11. Add a Level 2 Predictor The intercept (mean math achievement) at the j th school = The average intercept across schools when all predictors have value 0 (the mean, since we centered MeanSES). + the effect of being in a school with the MeanSES of school j + the effect of everything else on which j differs from the other schools.

12. title 'Model 2: Including Effects of School (Level 2) Predictors'; title2 '-- predicting MathAch from MeanSES'; run; proc mixed covtest noclprint; class school; model MathAch = MeanSES / solution ddfm = bw; random intercept / subject = school; run; “bw” specifies between/within partitioning of df

13. Fixed Effects MeanSES was centered about zero. That is, transformed to mean zero. Math Achievement = 12.6495 + 5.8635(School MeanSES – GrandMean SES) Each 1 point increase in School’s mean SES increases achievement by 5.86 points. Solution for Fixed Effects EffectEstimateStandard Error DFt ValuePr > |t| Intercept12.64950.149215884.77<.0001 MeanSES5.86350.361315816.23<.0001

14. Random Effects Including school mean SES in model reduced variance in intercepts from 8.6097 to 2.6357 = a drop of 5.974. School mean SES accounts for 5.974/8.6097 = 69% of the variance among schools. Covariance Parameter Estimates Cov ParmSubjectEstimateStandard Error Z ValuePr > Z InterceptSchool2.63570.40366.53<.0001 Residual 39.15780.660859.26<.0001

15. Unexplained Variance After accounting for SES, MathAch intercepts (means) still differ significantly across schools (z = 6.53) Residual variance = 2.6357 (among schools) + 39.1578 (within schools) = 41.7935. 2.6357/41.7935 = 6.3% remains to be explained by some other Level 2 predictor.

16. Use a Level 1 Predictor Score for the ij th student = the intercept for the j th school + the effect of this student’s SES + other things involving that student. Student SES will be centered by subtracting from it the mean SES at the student’s school

17. Level 2 Equations Have dropped the MeanSAS predictor Need a random intercept and a random slope. Intercept for School j = Grand intercept + effect of being in School j

18. Slope (for relating student’s SES to MathAch) at School j = The grand slope + the effect (on slope) of being at School j

19. Combined Equation The fixed effects The random effects

20. title 'Model 3: Including Effects of Student-Level Predictors'; title2 '--predicting MathAch from cSES'; data HLM2; set HLM; cSES = SES - MeanSES; run; proc mixed data = hsbc noclprint covtest noitprint; class School; model MathAch = cSES / solution ddfm = bw notest; random intercept cSES / subject = School type = un; run;

21. Fixed Effects estimated MathAch for a student whose SES is average for his or her school is 12.6493 average slope, across schools, for predicting MathAch from student SES is 2.1932, which is significantly different from zero Solution for Fixed Effects EffectEstimateStandard Error DFt ValuePr > |t| Intercept12.64930.244515951.75<.0001 cSES2.19320.1283702417.10<.0001

22. Random Effects The estimated variance in intercepts, across schools, is a significant 8.6769, even after controlling for student SES. Covariance Parameter Estimates Cov ParmSubjectEstimateStandard Error Z ValuePr Z UN(1,1)School8.67691.07868.04<.0001

23. The correlation between School Intercept and School Slope is a nonsignificant.051. Covariance Parameter Estimates Cov ParmSubjectEstimateStandard Error Z ValuePr Z UN(2,1)School0.050750.40620.120.9006

24. The variance in slopes (for predicting MathAch from student SES) is a significant.694. The slopes differ significantly across schools. Covariance Parameter Estimates Cov ParmSubjectEstimateStandard Error Z ValuePr Z UN(2,2)School0.69400.28082.470.0067

25. There remains significant within-school variance, 36.7, after controlling for student SES. SES accounted for 39.1487-36.7006 – 2.4481 units of variance, or 2.4881/39.1487* = 6.25% of the within-school variance. *See Slide 9 Covariance Parameter Estimates Cov ParmSubjectEstimateStandard Error Z ValuePr Z Residual 36.70060.625858.65<.0001

26. Predictors at Both Levels Level 1: Student’s SES Level 2: School mean SES And a new Level 2 predictor, whether the school is in the public sector (0) or is Catholic (1).

27. title 'Model 4: Model with Predictors From Both Levels and Interactions'; proc mixed noclprint covtest noitprint; class School; model mathach = MeanSES sector cSES MeanSES*Sector MeanSES*cSES Sector*cSES MeanSES*Sector*cSES / solution ddfm = bw notest; random intercept cSES / subject = School type = un; run;

28. Nonsignificant Fixed Effects Without further comment, I shall drop these two nonsignificant interactions from the model Solution for Fixed Effects EffectEstimateStandard Error DFt ValuePr > |t| MeanSES*Sector-1.12760.7355156-1.530.1273 MeanSES*Sector*cSES-0.18880.59977021-0.310.7528

29. Our Reduced Model Level 1 Level 2, intercepts Level 2, slopes.

30. Combined Equation Fixed Effects –Grand intercept –Overall slope for School SES (Predictor 1) –Overall slope School Sector (Predictor 2) –Overall slope for Student SES –Interaction between School SES and Student SES –Interaction between School Sector and Student SES

31. Random Effects Effect on intercept of being at School j Effect on slope being at School j Everything else affecting Student i at School j

32. title 'Model 5: Model with Two Interactions Deleted'; title2 '--predicting mathach from meanses, sector, cses and '; title3 'cross level interaction of meanses and sector with cses'; run; proc mixed noclprint covtest noitprint; class School; model MathAch = MeanSES Sector cSES MeanSES*cSES Sector*cSES / solution ddfm = bw notest; random intercept cSES / subject = School type = un; proc means mean q1 q3 min max skewness kurtosis; var MeanSES Sector cSES; run;

33. Fixed Effects Solution for Fixed Effects EffectEstimateStandar d Error DFt ValuePr > |t| Intercept12.11360.198815760.93<.0001 MeanSES5.33910.369315714.46<.0001 Sector1.21670.30641573.970.0001 cSES2.93880.1551702218.95<.0001 MeanSES*cSES1.03890.298970223.480.0005 Sector*cSES-1.64260.23987022-6.85<.0001

34. Interpret New Effects Sector: Math achievement is higher at Catholic Schools MeanSES x cSES: the slopes for predicting MathAch from cSES differ across levels of MeanSES. Sector x cSES: the slopes for predicting MathAch from cSES differ between public and Catholic schools

35. Sector x cSES In the combined equation, substitute 0 for value of sector to get equation for public schools And 1 to get equation for Catholic schools Public: 12.11 + 5.34(MeanSES) + 2.94(cSES) + 1.04(MeanSES)(cSES) Catholic: 13.33 + 5.34(MeanSES) + 1.30(cSES) + 1.04(MeanSES)(cSES)

36. Intercept higher for Catholic than for public – MathAch higher at Catholic schools. Slope for student SES higher at public schools than at Catholic schools.

37. MeanSES x cSES Find Q 1, Q 2, and Q 3 for School SES Substitute the quartile values into the combined equation to get one equation for each quartile. For each of two values (-3, +3) of cSES, predict MathAch at each value with each equation. Prepare table and plot of predicted values.

38. Notice that the slope increases as MeanSES increases MeanSES cSES Diffrnc -3+3 Q1Q1 5.5716.4310.86 Q2Q2 6.2319.2513.02 Q3Q3 7.0621.8814.82

40. Random Effects UN(1,1): The intercepts still differ significantly across schools. Covariance Parameter Estimates Cov ParmSubjectEstimateStandard Error Z ValuePr Z UN(1,1)School2.38170.37176.41<.0001 UN(2,1)School0.19260.20450.940.3464 UN(2,2)School0.10140.21380.470.3177 Residual 36.72120.626158.65<.0001

41. UN(2,1): No significant correlation between intercepts and slopes. UN(2,2): The slopes (predicting MathAch from cSES) do not differ significantly across schools. I shall drop cSES from the random effects.

42. Trimmed Model title 'Model 6: Simpler Model Without cSES Slopes'; proc mixed noclprint covtest noitprint; class School; model MathAch = MeanSES Sector cSES MeanSES*cSES Sector*cSES / solution ddfm = bw notest; random intercept / subject = School; run;

43. Effects of Trimming All of the fixed effects are still significant. Intercepts still differ significantly across schools. The Log Likelihood statistic has increased from 46503.7 to 46504.8, indicating slightly poorer fit. We can evaluate the difference in Log Likelihood statistics via Chi-square on 2 df. p =.58.