1. October 20, 2009 Session 8Slide 1 PSC 5940: Estimating the Fit of Multi-Level Models Session 8 Fall, 2009

2. October 20, 2009 Session 8Slide 2 Log-likelihood - GLMs Given a linear model: With possibly correlated errors: The log-likelihood is defined as: Given a linear model, differentiating the Generalized Sum of Squares wrt β, setting partial derivative to zero, and solving for β, produces: Allows errors to be correlated Look familiar?

3. October 20, 2009 Session 8Slide 3 Understanding MLE For OLS, the the MLE is the SSE when β=(X’X) -1 (X’Y) For log models, the MLS is the product of the (logged) errors given the formula for the Generalized Sum of Squares The latter differs from OLS as various assumptions are relaxed (nonlinearity, correlated errors, etc) The results of the likelihood function can be viewed topographically, as a “hill” showing the effect on the LE as you vary the estimated coefficients. The maximum-LE will be the peak of the hill. The “fit” measures (adj R 2, AIC, BIC) are telling you how high (or low) the peak is for a given model. Comparisons of the fit measures across models can assist in model selection

4. October 20, 2009 Session 8Slide 4 Measures of Model Fit R 2 and adj R 2 : AIC: Where is the maximized log- likelihood for the model AIC penalizes for the complexity of the model BIC: Where n is the number of observations BIC penalizes for increased model complexity and sample size (results in preference for simpler models)

5. October 20, 2009 Session 8Slide 5 BIC Test for “Improvement in Model Fit” BIC Difference (M 2 -M 1 ) Evidence for “More Complex” Model 0-2“Weak” 2-6“Positive” 6-10“Strong” >10“Conclusive”

6. October 20, 2009 Session 8Slide 6 Example in R: LM1 Predicting votes in referendum on alternative energy tax (erdf100<-e63_erdf) by price and region: Explanatory variables: Randomly assigned values: ($6 to $2400 p/y) price<-random_p Region (already so named) ML1<-lmer(erdf100~1+(1|price)+(1|region))

7. October 20, 2009 Session 8Slide 7 Example in R: LM2 Predicting votes in referendum on alternative energy tax (erdf100<-e63_erdf) by price, region and perceived risks posed by GCC: Explanatory variables: Randomly assigned values: ($6 to $2400 p/y) price<-random_p Region (already so named) Risk_GCC (0-10 scale) ML2<- lmer(erdf100~1+risk_GCC+(1|price)+(1|region))

8. October 20, 2009 Session 8Slide 8 LM1 Result: > summary(ML1) Linear mixed model fit by REML Formula: erdf100 ~ 1 + (1 | price) + (1 | region) AIC BIC logLik deviance REMLdev 15275 15296 -7634 15271 15267 Random effects: Groups Name Variance Std.Dev. price (Intercept) 97.5853 9.8785 region (Intercept) 8.7641 2.9604 Residual 1113.6147 33.3709 Number of obs: 1546, groups: price, 15; region, 5 Fixed effects: Estimate Std. Error t value (Intercept) 59.026 3.064 19.26

9. October 20, 2009 Session 8Slide 9 LM2 Result: Linear mixed model fit by REML Formula: erdf100 ~ 1 + risk_gcc + (1 | price) + (1 |region) AIC BIC logLik deviance REMLdev 14960 14987 -7475 14954 14950 Random effects: Groups Name Variance Std.Dev. price (Intercept) 99.8298 9.9915 region (Intercept) 6.6622 2.5811 Residual 990.3544 31.4699 Number of obs: 1532, groups: price, 15; region, 5 Fixed effects: Estimate Std. Error t value (Intercept) 30.9259 3.6408 8.494 risk_gcc 4.1735 0.3068 13.604

10. October 20, 2009 Session 8Slide 10 BIC Test Results: BIC for ML1 – BIC for ML2 = 15296 – 14987 = 309; “Conclusive” Note: You can have R calculate the difference: BIC(logLik(ML2))-BIC(logLik(ML1))

11. October 20, 2009 Session 8Slide 11 Model fit: BIC and Thinking Use of BIC is often (ill)used like a statistical “idiot light” Depends on sample employed Maximizes predictive capacity of model rather than model explanation When you face a decision of whether to add an explanatory variable: use a 2-step process Does the variable make theoretical sense? Does BIC show improved model fit? If answers to both are “yes”, then add the variable

12. October 20, 2009 Session 8Slide 12 BREAK

13. October 20, 2009 Session 8Slide 13 R Coding When modeling with only part of your data, use “subset” lmer(y~x…, subset=state==“NY”)

14. October 20, 2009 Session 8Slide 14 Workshop New developments on Models? Progress on Papers Research question motivated by literature reviews

15. October 20, 2009 Session 8Slide 15 Next Week Focus is on paper progress Build timelines for completion Focus on challenges, and what we need to do to surmount them Hone your research question: motivated by literature reviews Need 1-page progress reports, including task assignments