1 Experimental design and analyses of experimental data Lesson 6 Logistic regression Generalized Linear Models (GENMOD)

2 Logistic regression Used when data are dichotomous. Used when data are fractions between 0 and 1

3 Example: The distance from the nest to the nearest nest of Herring gull? On the vegetation surrounding the nest? On the number of eggs in the nest? Does predation of eggs in nests of Oyster catcher depend on

4 OBS DIST EGGS VEG KILLED B C B A C A A 3 Data:

5 Analysis of dichotomous data : Nests are categorized according to whether predation has occurred or not. No predation is scored as 0 Predation is scored as 1

6 Plus/minus predator visit to Oyster catcher nest

7 The purpose is to fit a model to the data – a model that predicts the probability of a nest being predated

8 The logistic regression model: where and ε BIN(0, π(1-π)) The logit-transformation The odds (the ratio between the probability of a positive and a negative event)

9 y =0 So that

10 How to do it in SAS

11 DATA logist; OPTIONS LINESIZE = 90; /* Example on logistic regression */ /* The example is inspirered by Dorthe Lahrmann's investigations of Oyster catchers (strandskader) on Langli in Ho Bugt */ INFILE 'h:\lin-mod\logist.prn' FIRSTOBS=2; INPUT dist eggs veg $ killed; /* dist = Distance to the nearest nest of Herring gull (sølvmåge)*/ /* eggs = Number of Oyster catcher eggs in a nest */ /* veg = vegetation type surrounding an Oyster catcher nest*/ IF killed > 0 THEN visit= 1; IF killed = 0 THEN visit = 0; /* If killed > 0 then the nest has been visited by a predator at least once */

12 /* Eksempel A: Analysis of a nest has been visited or not-visited by predators, i.e. visit = 1 or 0 */ PROC GENMOD; /* The procedure is Generalized Linear Models */ TITLE 'Eksempel A'; CLASS veg; /* veg is a class variable */ MODEL visit = dist veg /DIST=binomial LINK=logit TYPE3 DSCALE OBSTATS; /* DIST = distribution function (here chosen as binomial) */ /* LINK = the model uses a logit-transformation of data */ /* TYPE3 = type 3 is used in order to evaluate the relative contribution of the different factors on the independent variable */ /* DSCALE = an option which tells SAS to scale the error in order to meet the demands of the model. If DSCALE is approximately 1, scaling is not needed. */ /* OBSTATS = gives the predicted values as well as their confidence limits */ RUN;

13 Eksempel A 10:19 Thursday, November 22, The GENMOD Procedure Model Information Description Value Data Set WORK.LOGIST Distribution BINOMIAL Link Function LOGIT Dependent Variable VISIT Observations Used 57 Number Of Events 52 Number Of Trials 57 Class Level Information Class Levels Values VEG 3 A B C

14 Criteria For Assessing Goodness Of Fit Criterion DF Value Value/DF Deviance Scaled Deviance Pearson Chi-Square Scaled Pearson X Log Likelihood These values indicate the fit of the model. Low values (for a given DF) indicate a good fit These values should be close to unity if the model’s assumptions are met Values less than unity indicate underdispersion (variance less than expected) Values greater than unity indicate overdispersion (variance greater than expected) Values after scaling with DSCALE

15 Analysis Of Parameter Estimates Parameter DF Estimate Std Err ChiSquare Pr>Chi INTERCEPT DIST VEG A VEG B VEG C SCALE NOTE: The scale parameter was estimated by the square root of DEVIANCE/DOF. LR Statistics For Type 3 Analysis Source NDF DDF F Pr>F ChiSquare Pr>Chi DIST VEG

16 Criteria For Assessing Goodness Of Fit Criterion DF Value Value/DF Deviance Scaled Deviance Pearson Chi-Square Scaled Pearson X Log Likelihood Analysis Of Parameter Estimates Parameter DF Estimate Std Err ChiSquare Pr>Chi INTERCEPT DIST SCALE NOTE: The scale parameter was estimated by the square root of DEVIANCE/DOF. LR Statistics For Type 3 Analysis Source NDF DDF F Pr>F ChiSquare Pr>Chi DIST

17 Observation Statistics VISIT Pred Xbeta Std HessWgt Lower Upper Resraw

18 Predicted values and 95% confidence limits

19 /* Example B: Analysis of the fraction of eggs in a nest that are lost */ PROC GENMOD; /* procedure is Generalized Linear Models */ TITLE 'Eksempel B'; CLASS veg; /* veg is a class variable */ MODEL killed/eggs = dist veg eggs/DIST=binomial LINK=logit TYPE3 DSCALE OBSTATS; /* DIST = distribution function (here chosen as binomial) */ /* LINK = the model uses a logit-transformation of data */ /* TYPE3 = SS3 is used to determine the contribution of the individual factors to the dependent variable */ /* DSCALE = option that can be used if Deviance/DF is different from 1. It reduces the risk of Type 1 errors if the scale parameter is > 1 og the risk of a Type II errors, if the scale parameter is < 1 */ /* OBSTATS = gives the predicted values, and the confidence limits */ RUN; Note that this procedure takes the absolute number of eggs killed out of the total number of eggs into consideration, and not merely the proportion of killed eggs

20 Eksempel B 12:26 Thursday, November 22, The GENMOD Procedure Model Information Description Value Data Set WORK.LOGIST Distribution BINOMIAL Link Function LOGIT Dependent Variable KILLED Dependent Variable EGGS Observations Used 57 Number Of Events 183 Number Of Trials 336 Class Level Information Class Levels Values VEG 3 A B C

21 Criteria For Assessing Goodness Of Fit Criterion DF Value Value/DF Deviance Scaled Deviance Pearson Chi-Square Scaled Pearson X Log Likelihood

22 Analysis Of Parameter Estimates Parameter DF Estimate Std Err ChiSquare Pr>Chi INTERCEPT DIST VEG A VEG B VEG C EGGS SCALE NOTE: The scale parameter was estimated by the square root of DEVIANCE/DOF. LR Statistics For Type 3 Analysis Source NDF DDF F Pr>F ChiSquare Pr>Chi DIST VEG EGGS

23 Criteria For Assessing Goodness Of Fit Criterion DF Value Value/DF Deviance Scaled Deviance Pearson Chi-Square Scaled Pearson X Log Likelihood Analysis Of Parameter Estimates Parameter DF Estimate Std Err ChiSquare Pr>Chi INTERCEPT DIST SCALE NOTE: The scale parameter was estimated by the square root of DEVIANCE/DOF. LR Statistics For Type 3 Analysis Source NDF DDF F Pr>F ChiSquare Pr>Chi DIST

24 Predicted values and 95% confidence limits

25 Criteria For Assessing Goodness Of Fit Criterion DF Value Value/DF Deviance Scaled Deviance Pearson Chi-Square Scaled Pearson X Log Likelihood What is this?

26 The likelihood function

27 A nest contains n eggs of which r are eaten by predators. The probability that a given egg is eaten is denoted π. The probability that exactly r of the eggs are killed is The binomial distribution where

28 r 1 = number of killed eggs out of n 1 eggs in the first nest r 2 = number of killed eggs out of n 2 eggs in the second nest r i = number of killed eggs out of n i eggs in the ith nest The probability of observing exactly r 1, r 2,...,r i events is times L = P(r 1 ) P(r 2 ) P(r 3 ) P(r i ) P(r k ) = ln L = ln P(r 1 ) + ln P(r 2 ) + ln P(r 3 ) ln P(r i ) ln P(r k ) = Log-likelihood function

29 Maximum likelihood The parameters of are found as the values that maximize the likelihood of observing exactly r 1, r 2,....,r i.... positive events out of n 1, n 2,....,n i.... events The maximum value of L can be found by differentiation of L with respect to β 0, β 1,...., β p, and setting the derivative equal to 0. This is the same as differentiation with respect to ln L......