1. Likelihood Ratio Tests Restricted Maximum Likelihood
Stat 414 – Day 4
Likelihood Ratio Tests
Restricted Maximum Likelihood

2. HW 1

3. Leftovers Weighted least squares/Modelling heterogeneity
R2 vs. R2 adjusted

4. Weighted least squares

5. Formulas for R2
1 – SSE/SST
1 – (SSE/n)/(SST/n)
SSmodel /SST

6. Formulas for adjusted R2
R2 – p/(n – p  – 1)(1 – R2)
1 – (1 – R2)(n – 1)/(n – p – 1)
1 – (SSE/(n – p – 1))/(SST/(n – 1))
1 – MSE/”MST”

7. Last Time: MLEs

8. Last Time: Measures of Fit
Likelihood (want large)
Null: -12/2*ln(2*pi)-12/2*ln(193125/12)-12/2 = -75.1
Dist: -12/2*ln(2*pi)-12/2*ln( /12)-12/2 = -69.0
-2 log likelihood (want small)
AIC (want small)
Null: -2(-75.1) + 2(2) = 154.2
Dist: -2(-69.0) + 2(3) = 144
BIC
tends to “gravitate less quickly toward more complex models as n increases.”
“Pseudo R2” = 1 – ((Lintercept)/(Lmodel))2/n
or 1 – (log Lmodel)/(log Lintercept)

9. Likelihood Ratio Test Test statistic = 2 ln(Lfull/ Lreduced)
= - 2 x log likelihood red - (-2 log likelihood full)
= -2 x (L0 – L1)
Reference distribution?
Chi-square with df = difference in number of parameters
Drop in deviance

10. Likelihood ratio test
Intuitively, when the likelihood for the larger model is much greater than it is for the reduced model, we have evidence that the larger model is more closely aligned with the observed data. This isn’t really a fair comparison on the face of it. We need to account for the fact that more parameters were estimated and used for the larger model.
That is accomplished by taking into account the degrees of freedom for the χ2 distribution. The expected value of the χ2 distribution is its degrees of freedom. Thus when the difference in the number of parameters is large, the test statistic will need to be much larger to convince us that it is not simply chance variation with two identical models.

11. Residual standard error?
ML estimators have a downward bias in estimating the variance
Standard errors tend to be too small

12. Restricted Maximum Likelihood

13. REML vs. ML
To compare fixed effects in models with the same random structure,
Use REML standard errors for t/F tests
Use ML for nested LRTests
(Variable selection)
To compare models with different random structures (same fixed component)
Can use either (REML)
For final model estimates
Use REML
Use REML standard errors for t-tests and F-tests of fixed effects

14. To Do
Review HW 1
Bring HW 2 questions to class
Be reading Ch. 2