1. Multilevel and Random Coefficients Models School of Nursing “Multi-Level Models: What Are They and How Do They Work? ” Melinda K. Higgins, Ph.D. 30 March 2009

2. School of Nursing Multilevel and Random Coefficients Models Why are MLM’s important? Ordinary Least Squares (OLS) and all regression models under the General Linear Model (including logistic and poisson regression) assumes that all observations are independent of one another. Likewise, repeated measures (time1, time2, …) assume that all time points for a given individual are independent of one another (all pairs of time points), including serial dependency (“Durbin–Watson statistic”). When data are “clustered,” OLS may lead to inaccuracies in inference – standard errors may be too small – leads to overestimation of significance (“alpha-inflation”). “Clusters” may include individuals within: dyads; twins; families; counseling groups; households; neighborhoods; schools; nursing units; hospitals (and repeated measures over time within an individual – the individual becomes the “cluster”).

3. School of Nursing Multilevel and Random Coefficients Models An example Subjects: 386 women across 40 women’s groups with a focus on diet and weight control Group sizes range from 5-15 Independent variable (measured for each individual) is “motivation.” Outcome – weight loss The groups meet regularly and may have some level of cohesion – we may expect some correlation between the individuals with a given group. In fact, there is substantial “clustering” indicated by the fact that the groups differ substantially in average weight loss (9.75lbs-24.43lbs)

4. School of Nursing Multilevel and Random Coefficients Models Average Weight Loss – 40 Groups

5. School of Nursing Multilevel and Random Coefficients Models Weight Loss by Motivation (individual)

6. School of Nursing Multilevel and Random Coefficients Models Regression – Motivation(X) vs Wt Loss (Y) (no consideration of groups)

7. Regression – 40 separate groups

8. School of Nursing Multilevel and Random Coefficients Models Regression of Group Averages (avg motivation & avg weight loss) WARNING: “ECOLOGICAL FALLACY” – cannot make generalizations from the group down to the individual level. This approach only tells us the relationship between the group mean motivation and the group’s average weight loss – NOT about individuals performance.

9. School of Nursing Multilevel and Random Coefficients Models Regression – control for groups 39 (g-1) dummy variables

10. School of Nursing Multilevel and Random Coefficients Models ANCOVA – same as (g-1) dummy vars

11. School of Nursing Multilevel and Random Coefficients Models How to estimate the “degree” of clustering? Intraclass Correlation (ICC) The ICC measures the proportion of the total variance of an outcome variable that is accounted for by the clustering of the cases. OLS regression (and GLM) assumes that there is no amount of variance accounted for by these “clusters”/groups; ICC=0. ICC varies from 0 to 1 (normally). [NOTE: ICC can be negative, but it is rare – this would mean that individuals within a given “cluster” were more different from one another within the group than with individuals outside the group.] The ICC may be initially estimated from a one-way (“Fixed Effects”) ANOVA on the groups or “clusters.” Even for ICC’s as small as 0.01, for n=25 subjects per group, the actual alpha=0.11 (which is more than 2 times the nominal alpha=0.05). “Alpha-inflation” increases as ICC and number of subjects per group increases.

12. School of Nursing Multilevel and Random Coefficients Models Intraclass Correlation Where M n mean number of cases per group; sd 2 (n j ) is the variance of the number of cases per group; g=number of groups Estimated ICC=0.22 indicates substantial clustering ** [for ICC=0.20, n=10, actual alpha=0.28 not 0.05 (Kreft, deLeeuw 1998)] One-Way Fixed Effects

13. School of Nursing Multilevel and Random Coefficients Models “Fixed vs. Random Coefficients” OLS regression – only the outcomes and errors/residuals are considered to be random (i.e. samples from a larger population) Random Fixed “Random Coefficient” regression – now the intercepts and slopes are also considered to be RANDOM (i.e. samples from a larger population)

14. School of Nursing Multilevel and Random Coefficients Models “Multi-Level / Hierarchical Linear Models” Level 1: Individual Level; “micro-level” Level 2: Group Level; “macro-level” Combine Level 1 and Level 2 & “Additional New Parameters” – Variance Components – help to capture the impact of the group structure on the relationship of the predictors on the dependent variable. i=individualj=group

15. School of Nursing Multilevel and Random Coefficients Models Unconditional Cell Means Model MIXED pounds BY group /CRITERIA=CIN(95) MXITER(100) MXSTEP(5) SCORING(1) SINGULAR(0.000000000001) HCONVERGE(0, ABSOLUTE) LCONVERGE(0, ABSOLUTE) PCONVERGE(0.000001, ABSOLUTE) /FIXED=| SSTYPE(3) /METHOD=REML /PRINT=CPS CORB COVB DESCRIPTIVES G LMATRIX R SOLUTION TESTCOV /RANDOM=INTERCEPT | SUBJECT(group) COVTYPE(UN) /EMMEANS=TABLES(OVERALL). Calculate ICC from the RC model using only the group membership (no level 1 predictor) – equivalent to a “One-way Random Effects ANOVA” [for ICC=0.20, n=10, actual alpha=0.28 not 0.05 (Kreft, deLeeuw 1998)] =

16. School of Nursing Multilevel and Random Coefficients Models Unconditional Cell Means Model = 16.069 is significant (pval<0.001) – 16.069 units of within class variance in pounds lost that might be significantly accounted for by motivation predictors. = 4.906 is also significant (pval=0.002) – 4.906 units of between class variance in pounds lost (reflects differences between groups in mean pounds lost); and since this is significantly > 0, indicates that there is random variation among the intercepts of the various groups – WE SHOULD NOT IGNORE CLUSTERING. =

17. School of Nursing Multilevel and Random Coefficients Models Random Coefficient Model MIXED pounds BY group WITH motivatc /CRITERIA=CIN(95) MXITER(100) MXSTEP(5) SCORING(1) SINGULAR(0.000000000001) HCONVERGE(0, ABSOLUTE) LCONVERGE(0, ABSOLUTE) PCONVERGE(0.000001, ABSOLUTE) /FIXED=motivatc | SSTYPE(3) /METHOD=REML /PRINT=CPS CORB COVB DESCRIPTIVES G LMATRIX R SOLUTION TESTCOV /RANDOM=INTERCEPT motivatc | SUBJECT(group) COVTYPE(UN) /EMMEANS=TABLES(OVERALL). Level 2 Level 1 Mixed Level 1 & 2

18. School of Nursing Multilevel and Random Coefficients Models Consider the Fixed Effects Portion of the Model Weight Loss (Y ij ) = 15.137701 lbs + 3.118 units_motivation/lbs *(motivation_ctr (x ij )) Both the intercept and slope are significant (both pval<0.001). So, for every unit increase in motivation, 3.118 more lbs are lost; with an average weight loss per group of 15.14 lbs. Fixed Random

19. School of Nursing Multilevel and Random Coefficients Models Consider the Random Effect Portion of the Model variance of the slopes across groups is significant (again do not ignore clustering) the covariance between slopes and intercepts was positive indicating the higher the average pounds lost in a group (intercept), the stronger the positive relationship of motivation on weight loss; but this was not significant. the addition of motivation as a predictor reduced the within group variance in pounds lost by 63% (from 16.069 to 5.933) the addition of motivation as a predictor also reduced the between group differences in avg. pounds lost (intercepts) by 51% (4.906 to 2.397) SO, overall there were fluctuations between the groups in terms of mean pounds lost – these fluctuations were systematically related to the mean motivation level of individuals within these groups. Cannot test this via ANCOVA nor (g-1) dummy variables in regression.

20. School of Nursing Multilevel and Random Coefficients Models MLM with Predictor at Level 2 (group variable) “treatment” = diet program vs control MIXED pounds BY group WITH motivatc treatc /CRITERIA=CIN(95) MXITER(100) MXSTEP(5) SCORING(1) SINGULAR(0.000000000001) HCONVERGE(0, ABSOLUTE) LCONVERGE(0, ABSOLUTE) PCONVERGE(0.000001, ABSOLUTE) /FIXED=motivatc treatc motivatc*treatc | SSTYPE(3) /METHOD=REML /PRINT=CPS CORB COVB DESCRIPTIVES G LMATRIX R SOLUTION TESTCOV /RANDOM=INTERCEPT motivatc | SUBJECT(group) COVTYPE(UN) /EMMEANS=TABLES(OVERALL). Fixed Random

21. School of Nursing Multilevel and Random Coefficients Models Repeated Measures Investigate the following hypothetical data set. YA is the outcome measures over 5 time points for 60 individuals

22. School of Nursing Multilevel and Random Coefficients Models MIXED YA BY ID /CRITERIA=CIN(95) MXITER(100) MXSTEP(5) SCORING(1) SINGULAR(0.000000000001) HCONVERGE(0, ABSOLUTE) LCONVERGE(0, ABSOLUTE) PCONVERGE(0.000001, ABSOLUTE) /FIXED=| SSTYPE(3) /METHOD=REML /PRINT=CPS CORB COVB DESCRIPTIVES G LMATRIX R SOLUTION TESTCOV /RANDOM=INTERCEPT | SUBJECT(ID) COVTYPE(UN) /EMMEANS=TABLES(OVERALL). YA-Model 1 (random intercept – cell means model) DO NOT IGNORE CLUSTERING =

23. School of Nursing Multilevel and Random Coefficients Models MIXED YA BY ID WITH TIME /CRITERIA=CIN(95) MXITER(100) MXSTEP(5) SCORING(1) SINGULAR(0.000000000001) HCONVERGE(0, ABSOLUTE) LCONVERGE(0, ABSOLUTE) PCONVERGE(0.000001, ABSOLUTE) /FIXED=| SSTYPE(3) /METHOD=REML /PRINT=CPS CORB COVB DESCRIPTIVES G LMATRIX R SOLUTION TESTCOV /RANDOM=INTERCEPT TIME | SUBJECT(ID) COVTYPE(UN) /EMMEANS=TABLES(OVERALL). YA-Model 2 (random intercept and slope)

24. School of Nursing Multilevel and Random Coefficients Models MIXED YA BY ID WITH TIME /CRITERIA=CIN(95) MXITER(100) MXSTEP(5) SCORING(1) SINGULAR(0.000000000001) HCONVERGE(0, ABSOLUTE) LCONVERGE(0, ABSOLUTE) PCONVERGE(0.000001, ABSOLUTE) /FIXED=TIME | SSTYPE(3) /METHOD=REML /PRINT=CPS CORB COVB DESCRIPTIVES G LMATRIX R SOLUTION TESTCOV /RANDOM=INTERCEPT TIME | SUBJECT(ID) COVTYPE(UN) /EMMEANS=TABLES(OVERALL). YA-Model 3 (random intercept and slope & fixed time) i=time; j=individual

25. School of Nursing Multilevel and Random Coefficients Models Notes on the MLM approach to RM-ANOVA for YA – Model 3 Model 3 added “time” as a fixed predictor to the model to determine whether there was a significant “linear” effect of time. There is a significant effect of adding time to the model ( residual error dropped from 3.02 down to 0.228), which is still significant. Adding time to the model also accounted for all the significant variance in the slopes of YA on time, e.g. differences between slopes were no longer significant (pval = 0.140). And the covariance (between slopes and intercepts) is also not significant. Individual mean differences (intercepts) (pval<0.001) remain a significant source of variance.

26. School of Nursing Multilevel and Random Coefficients Models Summary If you suspect group (or cluster) effects, i.e. subjects within your groups may exhibit some level of “cohesion” or “alikeness,” leading you to think they are more alike within groups than between groups, the ICC should be calculated and evaluated. If ICC is close to 0, run both OLS regression and a Random Coefficient (“mixed”) model and compare. Evaluate the “random variance components.” Try comparing RM-ANOVA with the “mixed” multi-level model approach where “time” is level 1 and the individual is level 2. Evaluate the impact of adding time to the model.

27. School of Nursing Multilevel and Random Coefficients Models References Cohen, Jacob; Cohen, Patricia; West, Stephen; Aiken, Leona. “Applied Multiple Regression/Correlation Analysis for the Behavioral Sciences” 3 rd edition, Lawrence Erlbaum Associates Inc., 2003. [All data/examples in this lecture came from this book – mostly Chapter 14 (Randon Coefficient Regression and Multilevel Models) and Section 15.4 (Multilevel Regression of Individual Changes Over Time)] Raudenbush; Stephen W.; Bryk, Anthony S. “Hierarchical Linear Models: Applications and Data Analysis Methods” 2 nd edition, SAGE publications, 2002. Kreft, Ita; de Leeuw, Jan. “Introducing Multilevel Modeling,” SAGE publications, 1998. Tabachnick, Barbara G.; Fidell, Linda S. “Using Multivariate Statistics,” 5 th edition, Pearson Education Inc., 2007. [Chapter 15 focuses on Multilevel Linear Modeling.] *

28. School of Nursing Multilevel and Random Coefficients Models VIII. Statistical Resources and Contact Info SON S:\Shared\Statistics_MKHiggins\website2\index.htm [updates in process] Working to include tip sheets (for SPSS, SAS, and other software), lectures (PPTs and handouts), datasets, other resources and references Statistics At Nursing Website: [website being updated] http://www.nursing.emory.edu/pulse/statistics/ http://www.nursing.emory.edu/pulse/statistics/ And Blackboard Site (in development) for “Organization: Statistics at School of Nursing” Contact Dr. Melinda Higgins Melinda.higgins@emory.edu Office: 404-727-5180 / Mobile: 404-434-1785