1 G Lect 11W Logistic Regression Review Maximum Likelihood Estimates Probit Regression and Example Model Fit G Multiple Regression Week 11 (Wednesday)

2 G Lect 11W Logistic Regression Review Additive probability models are often not ideal. »p is bounded by [0,1]. Log odds of p [  (p)] has nice properties. »Unbounded, with  (p)=0 for p=.5. Important equations

3 G Lect 11W Logistic Example: Clinic Only

4 G Lect 11W Maximum Likelihood Estimates Strategy: Choose guesses for regression weights that makes the outcome seem likely under the model. We use simple binomial probability theory to calculate the likelihood. »Choose some probability starting values »Calculate how likely each observation is under the initial probabilities »Summarize the overall likelihood »Modify the model to improve the likelihood

5 G Lect 11W What is the likelihood of rolling a "1" on a single die? What is the likelihood of rolling "snake eyes" with two dice? Review of Binomial Probability 1/6

6 G Lect 11W What is the likelihood of rolling one "1" and one value other than "1"? What is the probability of Y (where Y is 1 when die is "1" and 0 otherwise)? Review of Binomial Probability

7 G Lect 11W Suppose Suppose Y is a binary random variable with probability p. What is the probability of Y? Suppose that r=  Y, where n observations are summed. What is probability of  Y =r? Review of Binomial Probability

8 G Lect 11W Summarizing Likelihood Probabilities of rare events get small. The probability that a particular sample will have a specific pattern of values is very small. Products of probabilities get exponentially smaller Keeping track of these small numbers is facilitated by taking logs. Ln likelihoods sum (whereas likelihoods need to be multiplied)

9 G Lect 11W Example This Excel spreadsheet shows how the regression coefficients change the fitted probabilities.

10 G Lect 11W Probit as an Alternative to Logit Instead of using p=(1+e -  ) -1, some statisticians have suggested using the cumulative normal function p=  (z). »z = 0 for p=.5 »z=> -∞ for p=>0 »z=> ∞ for p=>1 The linear model is written for the assumed standard normal variable underlying the binary process. »The process is made binary by applying some discrete threshold.

11 G Lect 11W Probit Example: Clinic Only * * * * * * * * * * * * P R O B I T A N A L Y S I S * * * * * Parameter estimates converged after 13 iterations. Optimal solution found. Parameter Estimates (PROBIT model: (PROBIT(p)) = Intercept + BX): Regression Coeff. Standard Error Coeff./S.E. FEMALE AGEC Intercept Standard Error Intercept/S.E

12 G Lect 11W The relation of Probit to Logit

13 G Lect 11W Fit Statistics in Logistic Regression Likelihood ratio test of k predictors »Compare LL(model) to LL(null) LL(null) has only intercept in model -2*[LL(model-LL(null)] is distributed as Chi Square on k df Quasi R-square

14 G Lect 11W Other Fit Measures Cox & Snell Index Nagelkerke Index »Scales R 2 CS to have maximum of 1.0 Akaike's Information Criterion

15 G Lect 11W Hosmer-Lemeshow test Hosmer and Lemeshow argue that if the linear logistic model is correct, then we should be able to »Break the sample into g groups »Compute the average probability of the outcome in each group »Compare the observed frequencies to the expected using a Pearson Chi Square »Evaluate the Chi Square statistic to a central Chi Square on g-2 df. Test is useful when sample size is large.