Logistic Regression

Categorical versus Continuous Data Analysis Response INTERNAL USE FIG. 05S01F01

Overview INTERNAL USE FIG. 05S03F01

Types of Logistic Regression INTERNAL USE FIG. 05S03F02

Logistic regression Logistic regression investigates the relationship between a response variable and one or more predictors. Minitab provides three logistic regression procedures that you can use to assess the relationship between one or more predictor variables and a categorical response variable of the following types:

What Does Logistic Regression Do? The logistic regression model uses the predictor variables, which can be categorical or continuous, to predict the probability of specific outcomes. In other words, logistic regression is designed to describe probabilities associated with the values of the response variable.

Logistic Regression Curve 1.0 0.9 0.8 0.7 Probability 0.6 0.5 0.4 0.3 0.2 0.1 0.0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 x

Logit Transformation Logistic regression models transform probabilities called logits. Where i indexes all cases (observations). pi is the probability the event (a sale, for example) occurs in the ith case. log is the natural log (to the base e).

Assumption pi (pi ) INTERNAL USE FIG. 05S03F03

Logistic Regression Model logit (pi) = 0 + 1Xi where logit(pi) logit transformation of the probability of the event 0 intercept of the regression line 1 slope of the regression line.

Binary Logistic Regression This demonstration illustrates fitting a binary logistic regression model in PROC LOGISTIC.

Odds Ratio from a Logistic Regression Model Estimated logistic regression model: Estimated odds ratio (Females to Males): odds ratio = (e-.7566+.4373)/(e-.7566) odds ratio = e.4373 = 1.549

Multiple Logistic Regression logit (pi) = 0 + 1X1 + 2X2 + 3X3 INTERNAL USE FIG. 05S03F10

Binary Logistic Regression Use binary logistic regression to perform logistic regression on a binary response variable. A binary variable only has two possible values, such as presence or absence of a particular disease. A model with one or more predictors is fit using an iterative reweighted least squares algorithm to obtain maximum likelihood estimates of the parameters . Binary logistic regression has also been used to classify observations into one of two categories, and it may give fewer classification errors than discriminant analysis for some cases.

RestingPulse Smokes Weight Example: You are a researcher who is interested in understanding the effect of smoking and weight upon resting pulse rate. Because you have categorized the response-pulse rate-into low and high, a binary logistic regression analysis is appropriate to investigate the effects of smoking and weight upon pulse rate.

SAS code proc logistic desc; class Smokes/desc param=glm; model RestingPulse = Smokes Weight; run;

Output Logistic Regression Table Odds 95% CI                                                    Odds    95% CI Predictor        Coef    SE Coef       Z      P  Ratio  Lower  Upper Constant    -1.98717    1.67930   -1.18  0.237 Smokes  Yes         -1.19297   0.552980   -2.16  0.031   0.30   0.10   0.90 Weight     0.0250226  0.0122551   2.04  0.041   1.03   1.00   1.05 Test that all slopes are zero: G = 7.574, DF = 2, P-Value = 0.023

Logistic Regression Table - shows the estimated coefficients, standard error of the coefficients, z-values, and p-values, odds ratio and a 95% confidence interval for the odds ratio. From the output, you can see that the estimated coefficients for both Smokes (z = -2.16, p = 0.031) and Weight (z = 2.04, p = 0.041) have p-values less than 0.05, indicating that there is sufficient evidence that the coefficients are not zero using an a-level of 0.05. The estimated coefficient of -1.193 for Smokes represents the change in the log of P(low pulse)/P(high pulse) when the subject smokes compared to when he/she does not smoke, with the covariate Weight held constant. The estimated coefficient of 0.0250 for Weight is the change in the log of P(low pulse)/P(high pulse) with a 1 unit (1 pound) increase in Weight, with the factor Smokes held constant.

Although there is evidence that the estimated coefficient for Weight is not zero, the odds ratio is very close to one  (1.03), indicating that a one pound increase in weight minimally effects a person's resting pulse rate. A more meaningful difference would be found if you compared subjects with a larger weight difference (for example, if the weight unit is 10 pounds, the odds ratio becomes 1.28, indicating that the odds of a subject having a low pulse increases by 1.28 times with each 10 pound increase in weight). For Smokes, the negative coefficient of -1.193 and the odds ratio of 0.30 indicate that subjects who smoke tend to have a higher resting pulse rate than subjects who do not smoke. Given that subjects have the same weight, the odds ratio can be interpreted as the odds of smokers in the sample having a low pulse being 30% of the odds of non-smokers having a low pulse.

Ordinal Logistic Regression Use ordinal logistic regression to perform logistic regression on an ordinal response variable. Ordinal variables are categorical variables that have three or more possible levels with a natural ordering, such as strongly disagree, disagree, neutral, agree, and strongly agree. A model with one or more predictors is fit using an iterative-reweighted least squares algorithm to obtain maximum likelihood estimates of the parameters. Parallel regression lines are assumed, and therefore, a single slope is calculated for each covariate. In situations where this assumption is not valid, nominal logistic regression, which generates separate logit functions, is more appropriate.

Survival Region ToxicLevel Example Suppose you are a field biologist and you believe that adult population of salamanders in the Northeast has gotten smaller over the past few years. You would like to determine whether any association exists between the length of time a hatched salamander survives and level of water toxicity, as well as whether there is a regional effect. Survival time is coded as 1 if < 10 days, 2 = 10 to 30 days, and 3 = 31 to 60 days.

SAS code proc logistic; class Region/desc param=glm; model Survival = Region ToxicLevel; run;

Logistic Regression Table                                                  Odds     95% CI Predictor       Coef     SE Coef      Z      P    Ratio  Lower  Upper Const(1)    -7.04343    1.68017   -4.19  0.000 Const(2)    -3.52273    1.47108   -2.39  0.017 Region  2           0.201456   0.496153    0.41  0.685   1.22   0.46   3.23 ToxicLevel  0.121289  0.034051   3.56  0.000   1.13   1.06   1.21   Test that all slopes are zero: G = 14.713, DF = 2, P-Value = 0.001

Interpreting the results The values labeled Const(1) and Const(2) are estimated intercepts for the logits of the cumulative probabilities of survival for <10 days, and for 10-30 days, respectively. Because the cumulative probability for the last response value is 1, there is not need to estimate an intercept for 31-60 days. The coefficient of 0.2015 for Region is the estimated change in the logit of the cumulative survival time probability when the region is 2 compared to region being 1, with the covariate ToxicLevel held constant. Because the p-value for estimated coefficient is 0.685, there is insufficient evidence to conclude that region has an effect upon survival time.

Interpreting the results There is one estimated coefficient for each covariate, which gives parallel lines for the factor levels. Here, the estimated coefficient for the single covariate, ToxicLevel, is 0.121, with a p-value of < 0.0005. The p-value indicates that for most a-levels, there is sufficient evidence to conclude that the toxic level affects survival. The positive coefficient, and an odds ratio that is greater than one indicates that higher toxic levels tend to be associated with lower values of survival. Specifically, a one-unit increase in water toxicity results in a 13% increase in the odds that a salamander lives less than or equal to 10 days versus greater than 30 days and that the salamander lives less than or equal to 30 days versus greater than 30 days. Next displayed is the last Log-Likelihood from the maximum likelihood iterations along with the statistic G. This statistic tests the null hypothesis that all the coefficients associated with predictors equal zero versus at least one coefficient is not zero. In this example, G = 14.713 with a p-value of 0.001, indicating that there is sufficient evidence to conclude that at least one of the estimated coefficients is different from zero.

Nominal Logistic Regression Use nominal logistic regression performs logistic regression on a nominal response variable using an iterative-reweighted least squares algorithm to obtain maximum likelihood estimates of the parameters. Nominal variables are categorical variables that have three or more possible levels with no natural ordering. For example, the levels in a food tasting study may include crunchy, mushy, and crispy.

TeachingMethod Age Subject Suppose you are a grade school curriculum director interested in what children identify as their favorite subject and how this is associated with their age or the teaching method employed. Thirty children, 10 to 13 years old, had classroom instruction in science, math, and language arts that employed either lecture or discussion techniques. At the end of the school year, they were asked to identify their favorite subject. We use nominal logistic regression because the response is categorical and possesses no implicit categorical ordering.

SAS code proc logistic; class TeachingMethod/desc param=glm; model Subject = TeachingMethod Age / link = glogit; run;

Logistic Regression Table Odds 95% CI Predictor Coef SE Coef Z P Ratio Lower Upper Logit 1: (math/science) Constant -1.12266 4.56425 -0.25 0.806 Age 0.124674 0.401079 0.31 0.756 1.13 0.52 2.49 TeachingMethod lecture -0.563115 0.937591 -0.60 0.548 0.57 0.09 3.58 Logit 2: (arts/science) Constant -13.8485 7.24256 -1.91 0.056 Age 1.01354 0.584494 1.73 0.083 2.76 0.88 8.66 lecture 2.76992 1.37209 2.02 0.044 15.96 1.08 234.91

Interpreting the results If there are k response distinct values, Minitab estimates k-1 sets of parameter estimates, here labeled as Logit(1) and Logit(2). These are the estimated differences in log odds or logits of math and language arts, respectively, compared to science as the reference event. Each set contains a constant and coefficients for the factor(s), here teaching method, and the covariate(s), here age. The TeachingMethod coefficient is the estimated change in the logit when TeachingMethod is lecture compared to the teaching method being discussion, with Age held constant. The Age coefficient is the estimated change in the logit with a one year increase in age with teaching method held constant. These sets of parameter estimates gives nonparallel lines for the response values.

Interpreting the results The first set of estimated logits, labeled Logit(1), are the parameter estimates of the change in logits of math relative to the reference event, science. The p-values of 0.548 and 0.756 for TeachingMethod and Age, respectively, indicate that there is insufficient evidence to conclude that a change in teaching method from discussion to lecture or in age affected the choice of math as favorite subject as compared to science.

Interpreting the results The second set of estimated logits, labeled Logit(2), are the parameter estimates of the change in logits of language arts relative to the reference event, science. The p-values of 0.044 and 0.083 for TeachingMethod and Age, respectively, indicate that there is sufficient evidence, if the p-values are less than your acceptable a-level, to conclude that a change in teaching method from discussion to lecture or in age affected the choice of language arts as favorite subject compared to science. The positive coefficient for teaching method indicates students given a lecture style of teaching tend to prefer language arts over science compared to students given a discussion style of teaching. The estimated odds ratio of 15.96 implies that the odds of choosing language arts over science is about 16 times higher for these students when the teaching method changes from discussion to lecture. The positive coefficient associated with age indicates that students tend to like language arts over science as they become older.