Logistic Regression

Example: Survival of Titanic passengers  We want to know if the probability of survival is higher among children  Outcome (y) = survived/not  Explanatory (x) = age at death > titan<- read.table("D:/RShortcourse/titanic.txt",sep=",",header=T, na=".") > titan2<-na.omit(titan) > log1<- glm(titan2$Survived~titan2$Age,family=binomial("logit")) Since “logit” is the default, you can actually use: > log1<-glm(titan2$Survived~titan2$Age,binomial) > summary(log1)

Example in R Call: glm(formula = titan$Survived ~ titan$Age, family = binomial("logit")) Coefficients: Estimate Std. Error z value Pr(>|z|) (Intercept) titan$Age Signif. codes: 0 ‘***’ ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 (Dispersion parameter for binomial family taken to be 1) Null deviance: on 755 degrees of freedom Residual deviance: on 754 degrees of freedom AIC:

Plot > plot(titan2$Age~fitted(log1), xlab="Age", ylab="P(Survival)", pch=15)

Example: Categorical x  Sometimes, it’s easier to interpret logistic regression output if the x variables are categorical  Suppose we categorize maternal age into 3 categories: > titan$agecat4 35, c(1),c(0)) > titan$agecat3 =18 & titan$Age<=35, c(1),c(0)) > titan$agecat2 =6 & titan$Age<=17, c(1),c(0)) > titan$agecat1 <- ifelse(titan$Age<6, c(1),c(0)) Age at Death Survival< 6 years6-17 years18-35 years> 35 years No Yes

Example in R > log2<-glm(titan2$Survived~titan2$agecat2 + titan2$agecat3 + titan2$agecat4, binomial)  summary(log2)  Remember, with a set of dummy variables, you always put in one less variable than category

Example in R Call: glm(formula = titan2$Survived ~ titan2$agecat1 + titan2$agecat2 + titan2$agecat3, family = binomial) Coefficients: Estimate Std. Error z value Pr(>|z|) (Intercept) * titan2$agecat *** titan2$agecat titan2$agecat Signif. codes: 0 ‘***’ ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 Null deviance: on 755 degrees of freedom Residual deviance: on 752 degrees of freedom AIC:

Interpretation: odds ratios > exp(cbind(OR=coef(log2),confint(log2))) Waiting for profiling to be done... OR 2.5 % 97.5 % (Intercept) titan2$agecat titan2$agecat titan2$agecat  This tells us that:  Passengers 35 years  Passengers in the other two age groups have no significant difference in odds of survival from passengers >35 years

Now let’s make it more complicated  Do sex and passenger class predict survival?  Let’s try a saturated model: > log3<-glm(titan$Survived~titan$Sex + titan$PClass + titan$Sex:titan$PClass, binomial) > summary(log3)

Example in R Call: glm(formula = titan$Survived ~ titan$Sex + titan$PClass + titan$Sex:titan$PClass, family = binomial) Coefficients: Estimate Std. Error z value Pr(>|z|) (Intercept) e-15 *** titan$Sexmale < 2e-16 *** titan$PClass2nd titan$PClass3rd < 2e-16 *** titan$Sexmale:titan$PClass2nd titan$Sexmale:titan$PClass3rd e-05 *** --- Signif. codes: 0 ‘***’ ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 Null deviance: on 1312 degrees of freedom Residual deviance: on 1307 degrees of freedom AIC:

What variables can we consider dropping? > anova(log3,test="Chisq") Analysis of Deviance Table Response: titan$Survived Terms added sequentially (first to last) Df Deviance Resid. Df Resid. Dev P(>|Chi|) NULL titan$Sex < 2.2e-16 *** titan$PClass < 2.2e-16 *** titan$Sex:titan$PClass e-10 *** --- Signif. codes: 0 ‘***’ ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 Small p-values indicate that all variables are needed to explain the variation in y

Goodness of fit statistics Coefficients: Estimate Std. Error z value Pr(>|z|) (Intercept) e-15 *** titan$Sexmale < 2e-16 *** titan$PClass2nd titan$PClass3rd < 2e-16 *** titan$Sexmale:titan$PClass2nd titan$Sexmale:titan$PClass3rd e-05 *** --- Signif. codes: 0 ‘***’ ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 Null deviance: on 1312 degrees of freedom Residual deviance: on 1307 degrees of freedom AIC: log L AIC

Binned residual plot install.packages("arm") library(arm) x<-predict(log3) y<-resid(log3) binnedplot(x,y) Category is based on the fitted values 95% of all values should fall within the dotted line  Plots the average residual and the average fitted (predicted) value for each bin, or category

Poisson Regression Using count data

What is a Poisson distribution?

Example: children ever born  The dataset has 70 rows representing group-level data on the number of children ever born to women in Fiji:  Number of children ever born  Number of women in the group  Duration of marriage  1=0-4, 2=5-9, 3=10-14, 4=15-19, 5=20-24, 6=25-29  Residence  1=Suva (capital city), 2=Urban, 3=Rural  Education  1=none, 2=lower primary, 3=upper primary, 4=secondary+ > ceb<-read.table("D:/RShortcourse/ceb.dat",header=T)

Poisson regression in R > ceb1<-glm(y ~ educ + res, offset=log(n), family = "poisson", data = ceb) Coefficients: Estimate Std. Error z value Pr(>|z|) (Intercept) <2e-16 *** educnone <2e-16 *** educsec <2e-16 *** educupper <2e-16 *** resSuva * resurban * --- Null deviance: on 69 degrees of freedom Residual deviance: on 64 degrees of freedom AIC: Inf Need to account for different population sizes in each area/group unless data are from same-size populations

Assessing model fit 1.Examine AIC score – smaller is better 2.Examine the deviance as an approximate goodness of fit test  Expect the residual deviance/degrees of freedom to be approximately 1 > ceb1$deviance/ceb1$df.residual [1] Compare residual deviance to a  2 distribution > pchisq(2646.5, 64, lower=F) [1] 0

Model fitting: analysis of deviance  Similar to logistic regression, we want to compare the differences in the size of residuals between models > ceb1<-glm(y~educ, family=“poisson", offset=log(n), data= ceb) > ceb2<-glm(y~educ+res, family=“poisson", offset=log(n), data= ceb) > 1-pchisq(deviance(ceb1)-deviance(ceb2), df.residual(ceb1)-df.residual(ceb2)) [1]  Since the p-value is small, there is evidence that the addition of res explains a significant amount (more) of the deviance

Overdispersion in Poission models  A characteristic of the Poisson distribution is that its mean is equal to its variance  Sometimes the observed variance is greater than the mean  Known as overdispersion  Another common problem with Poisson regression is excess zeros  Are more zeros than a Poisson regression would predict

Overdispersion  Use family=“quasipoisson” instead of “poisson” to estimate the dispersion parameter  Doesn't change the estimates for the coefficients, but may change their standard errors > ceb2<-glm(y~educ+res, family="quasipoisson", offset=log(n), data=ceb)

Poisson vs. quasipoisson Family = “poisson” Family = “quasipoisson” Coefficients: Estimate Std. Error z value Pr(>|z|) (Intercept) <2e-16 *** educnone <2e-16 *** educsec <2e-16 *** educupper <2e-16 *** resSuva * resurban * --- (Dispersion parameter for poisson family taken to be 1) Null deviance: on 69 degrees of freedom Residual deviance: on 64 degrees of freedom Coefficients: Estimate Std. Error t value Pr(>|t|) (Intercept) < 2e-16 *** educnone educsec ** educupper * resSuva resurban (Dispersion parameter for quasipoisson taken to be ) Null deviance: on 69 degrees of freedom Residual deviance: on 64 degrees of freedom

Models for overdispersion  When overdispersion is a problem, use a negative binomial model  Will adjust  estimates and standard errors > install.packages(“MASS”) > library(MASS) > ceb.nb <- glm.nb(y~educ+res+offset(log(n)), data= ceb) OR > ceb.nb<-glm.nb(ceb2) > summary(ceb.nb)

NB model in R glm.nb(formula = ceb2, x = T, init.theta = , link = log) Coefficients: Estimate Std. Error z value Pr(>|z|) (Intercept) < 2e-16 *** educnone educsec ** educupper resSuva resurban (Dispersion parameter for Negative Binomial(3.3872) family taken to be 1) Null deviance: on 69 degrees of freedom Residual deviance: on 64 degrees of freedom AIC: Theta: Std. Err.: x log-likelihood: > ceb.nb$deviance/ceb.nb$df.residual [1]

What if your data looked like…

Zero-inflated Poisson model (ZIP)  If you have a large number of 0 counts… > install.packages(“pscl”) > library(pscl) > ceb.zip <- zeroinfl(y~educ+res, offset=log(n), data= ceb)