2. Logistic Regression Saed Sayad 1www.ismartsoft.com

3. Definition Logistic Regression is a type of regression model where the dependent variable (target) has just two values, such as: 0, 1 Y, N F, T Logistic Regression is a type of regression model where the dependent variable (target) has just two values, such as: 0, 1 Y, N F, T 2www.ismartsoft.com

4. Sample Dataset www.ismartsoft.com3 Months n BusinessBalanceDefault 189$429,9160 170$240,3191 166$231,3270 423$196,1050 145$193,9071 60$190,9440 97$184,3330 354$152,1260 99$151,0611 80$135,8850 25$119,7511 118$116,5781 74$123,8640...

5. Linear Regression ( Continuous Dependent Variable ) www.ismartsoft.com4 Months in Business Balance

6. Linear Regression ( Binary Dependent Variable ) www.ismartsoft.com5 Default Months in Business

7. Linear Regression Model – Binary Target If the actual Y is a binary variable then the predicted Y can be less than zero or greater than 1 If the actual Y is a binary variable then error is not normally distributed. If the actual Y is a binary variable then the predicted Y can be less than zero or greater than 1 If the actual Y is a binary variable then error is not normally distributed. 6www.ismartsoft.com

8. Linear Regression Model 0 1 Y Y X X 7www.ismartsoft.com

9. Frequency Table www.ismartsoft.com8 Months in BusinessCount Default Count Default Frequency <50400 50-1001210.083 100-150410.25 150-200420.5 200-250430.75 250-300111 >300441

10. Frequency Plot www.ismartsoft.com 9 Months in Business - Bins Default Probability

11. Logistic Function www.ismartsoft.com10

12. Logistic Regression  The logistic distribution constrains the estimated probabilities to lie between 0 and 1.  Maximum Likelihood Estimation is a statistical method for estimating the coefficients of a model.  The logistic distribution constrains the estimated probabilities to lie between 0 and 1.  Maximum Likelihood Estimation is a statistical method for estimating the coefficients of a model. 11www.ismartsoft.com

13. Logistic Regression Model 0 1 Linear Model Logistic Model Y Y X X 12www.ismartsoft.com

14. Maximum Likelihood Estimation (MLE) MLE maximizes the log likelihood (LL) which reflects how likely it is that the dependent variable will be predicted from the independent variables. MLE is an iterative algorithm which starts with initial arbitrary numbers of what the coefficients should be. After this initial function is estimated, the process is repeated until LL does not change significantly. MLE maximizes the log likelihood (LL) which reflects how likely it is that the dependent variable will be predicted from the independent variables. MLE is an iterative algorithm which starts with initial arbitrary numbers of what the coefficients should be. After this initial function is estimated, the process is repeated until LL does not change significantly. 13www.ismartsoft.comCopyright iSmartsoft Inc. 2008

15. Log Likelihood (LL) www.ismartsoft.com14 Likelihood is the probability that the dependent variable may be predicted from the independent variables. LL is calculated through iteration, using maximum likelihood estimation (MLE). Log likelihood is the basis for tests of a logistic model. Likelihood is the probability that the dependent variable may be predicted from the independent variables. LL is calculated through iteration, using maximum likelihood estimation (MLE). Log likelihood is the basis for tests of a logistic model.

16. Log Likelihood Test (-2LL) The log likelihood test is a test of the significance of the difference between the likelihood ratio for the baseline model minus the likelihood ratio for a reduced model. This difference is called "model chi-square“. Also called Likelihood Ratio test. The log likelihood test is a test of the significance of the difference between the likelihood ratio for the baseline model minus the likelihood ratio for a reduced model. This difference is called "model chi-square“. Also called Likelihood Ratio test. www.ismartsoft.com15

17. Wald Test A Wald test is used to test the statistical significance of each coefficient (  ) in the model. A Wald test calculates a Z statistic, which is: This Z value is then squared, yielding a Wald statistic with a chi-square distribution. A Wald test is used to test the statistical significance of each coefficient (  ) in the model. A Wald test calculates a Z statistic, which is: This Z value is then squared, yielding a Wald statistic with a chi-square distribution. www.ismartsoft.com16

18. Summary Logistic Regression is a classification method. It returns the probability that the binary dependent variable may be predicted from the independent variables. Maximum Likelihood Estimation is a statistical method for estimating the coefficients of the model. The Likelihood Ratio test is used to test the statistical significance between the full model and the simpler model. The Wald test is used to test the statistical significance of each coefficient in the model. Logistic Regression is a classification method. It returns the probability that the binary dependent variable may be predicted from the independent variables. Maximum Likelihood Estimation is a statistical method for estimating the coefficients of the model. The Likelihood Ratio test is used to test the statistical significance between the full model and the simpler model. The Wald test is used to test the statistical significance of each coefficient in the model. www.ismartsoft.com17

19. 18www.ismartsoft.com Questions?