University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 11/06/2014 3:11 AM 1 Goodness of fit, contingency tables and log-linear models Appropriate questions The null hypothesis Tests of independence Subdividing tables Multiway tables and log-linear models Power analysis in goodness of fit and contingency tables

University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 11/06/2014 3:11 AM 2 Concepts map

University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 11/06/2014 3:11 AM 3 Goodness of fit measures the extent to which some empirical distribution fits the distribution expected under the null hypothesis Fork length Frequency Observed Expected

University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 11/06/2014 3:11 AM 4 Goodness of fit: the underlying principle If the match between observed and expected is poorer than would be expected on the basis of measurement precision, then we should reject the null hypothesis. Fork length Observed Expected Frequency Reject H 0 Accept H 0

University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 11/06/2014 3:11 AM 5 Testing goodness of fit : the Chi- square statistic ( Testing goodness of fit : the Chi- square statistic ( Used for frequency data, i.e. the number of observations/results in each of n categories compared to the number expected under the null hypothesis. Frequency Category/class Observed Expected

University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 11/06/2014 3:11 AM 6 How to translate 2 into p? Compare to the 2 distribution with n - 1 degrees of freedom. If p is less than the desired level, reject the null hypothesis (df = 5) Probability 2 = 8.5, p = 0.31 accept p = = 0.05

University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 11/06/2014 3:11 AM 7 Testing goodness of fit: the log likelihood- ratio Chi-square statistic (G) Similar to 2, and usually gives similar results. In some cases, G is more conservative (i.e. will give higher p values). Frequency Category/class Observed Expected

University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 11/06/2014 3:11 AM 8 2 versus the distribution of 2 or G 2 versus the distribution of 2 or G For both 2 and G, p values are calculated assuming a 2 distribution......but as n decreases, both deviate more and more from / 2 /G (df = 5) Probability 2 /G, very small n 2 /G, small n

University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 11/06/2014 3:11 AM 9 Assumptions ( 2 and G) n is larger than 30. Expected frequencies are all larger than 5. Test is quite robust except when there are only 2 categories (df = 1). For 2 categories, both X 2 and G overestimate 2, leading to rejection of null hypothesis with probability greater than i.e. the test is liberal.

University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 11/06/2014 3:11 AM 10 What if n is too small, there are only 2 categories, etc.? Collect more data, thereby increasing n. If n > 2, combine categories. Use a correction factor. Use another test. More data Classes combined

University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 11/06/2014 3:11 AM 11 Corrections for 2 categories For 2 categories, both X 2 and G overestimate 2, leading to rejection of null hypothesis with probability greater than i.e. test is liberal Continuity correction: add 0.5 to observed frequencies. Williams correction: divide test statistic (G or 2 ) by:

University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 11/06/2014 3:11 AM 12 Contingency analysis: types of questions Involves two (or more) categorical variables, each with 2 or more categories. Considers the number of observations (observed frequencies) in each category of the variables. Test is for lack of independence. Results of tests on the efficacy of two sprays (1, 2) in reducing apple blight infection in orchards

University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 11/06/2014 3:11 AM 13 Contingency analysis: types of questions Does the species composition of bird communities differ among habitats? 2 categorical variables: species, habitat type H 0 : the proportion of individuals of each species is independent of (i.e. more or less the same in each) habitat.

University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 11/06/2014 3:11 AM 14 Components of the test Null hypothesis Observations (observed frequencies) Statistic (Chi-square or G) Assumptions

University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 11/06/2014 3:11 AM 15 Null hypothesis In contingency analysis, the null hypothesis is that the distribution of observed frequencies among categories of one variable (e.g. A) is independent of the category of the other variables (B, C,...), i.e. that there is no interaction. The null hypothesis is always intrinsic!

University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 11/06/2014 3:11 AM 16 Testing H 0 : goodness-of-fit In contingency analysis (as in all statistical procedures) we fit a model to the data. H 0 specifies particular values for particular terms (coefficients) in the model… …and is evaluated by assessing how well the fitted model, with parameter values as specified by H 0, fits the data, i.e. by evaluating goodness-of- fit.

University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 11/06/2014 3:11 AM 17 Reminder: goodness of fit. Measures the extent to which some empirical distribution fits the distribution expected under the null hypothesis. Observed Expected Fork length Frequency

University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 11/06/2014 3:11 AM 18 Testing goodness of fit : the Chi- square statistic ( Testing goodness of fit : the Chi- square statistic ( Used for frequency data, i.e. the number of observations/results in each of n categories compared to the number expected under the null hypothesis. Frequency Category/class Observed Expected

University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 11/06/2014 3:11 AM 19 Two way tables: H 0 accepted H 0 : proportion of infected versus non- infected trees is the same for both sprays. In this case, we accept H 0. Proportion infected Spray 2 Spray 1

University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 11/06/2014 3:11 AM 20 Two way tables: H 0 rejected H 0 : proportion of infected versus non- infected trees is the same for both sprays. In this case, we reject H 0. Proportion infected Spray 2 Spray 1

University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 11/06/2014 3:11 AM 21 Two-way tables: the general model-fitting procedure Fit 2 models: one in which the interaction is included, the other with it removed. Evaluate GOF for each model. Evaluate the reduction in GOF associated with dropping the interaction, i.e. under H 0 that the interaction is zero. Model 1 (interaction in) Model 2 (interaction out) GOF (e.g. 2 ) Accept H 0 ( small) Reject H 0 ( large)

University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 11/06/2014 3:11 AM 22 Two-way tables : H 0 and model fit For two way tables, the general model includes a constant, two main effects, and an interaction. Thus, independence implies that the goodness- of-fit of a model with the interaction deleted is not significantly different from a model with the interaction included. Interaction out Interaction in Accept H 0 Goodness of fit (e.g. G) Reject H 0 GOF

University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 11/06/2014 3:11 AM 23 Two-way tables : what does the general model mean anyway? The model attempts to predict the observed frequencies in each category. So, if all frequencies are equal, then the appropriate model is: N = 80, = 80/4 = 20

University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 11/06/2014 3:11 AM 24 Two-way tables : what does the general model mean anyway? If N varies between the two sprays, then there will be a main effect due to spray. So the appropriate model includes a main effect due to spray (row, i). N = 80, = 80/4 = 20 f 1_ /2 = 30 = 1.5 f 2_ /2 = 10 = = 1.5, 2 = 0.5

University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 11/06/2014 3:11 AM 25 Two-way tables : what does the general model mean anyway? If the total number of trees infected is different than the number not infected, then there will be a main effect due to infection level. So the appropriate model includes a main effect due to both spray type and infection level. N = 80, = 80/4 = 20 f 1_ /2 = 30 = 1.5 f 2_ /2 = 10 = = 1.5, 2 = 0.5 f _1 /2 = 16 = 0.8 f _2 /2 = 24 = = 0.8, 2 = 1.2

University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 11/06/2014 3:11 AM 26 Two-way tables : what does the general model mean anyway? Since the expected frequency in cell (i,j) under H 0 is:... we can calculate the interaction by: N = 90, = 90/4 = 22.5

University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 11/06/2014 3:11 AM 27 Tests of independence: the Chi-square statistic ( Tests of independence: the Chi-square statistic ( Calculate expected frequency for each cell in the table. Calculate squared difference between observed and expected frequencies and sum over all cells. Observed Expected Frequency Cell

University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 11/06/2014 3:11 AM 28 Testing independence: the log likelihood-ratio Chi-square statistic (G) Similar to 2, and usually gives very similar results. In some cases, G is more conservative (i.e. will give higher p values). Observed Expected Frequency Cell

University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 11/06/2014 3:11 AM 29 An example: sex-ratios of eider ducks in different habitats in Hudsons Bay Cell counts are observed numbers (raw frequencies) of males and females in different habitats.

University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 11/06/2014 3:11 AM 30 Computing expected frequencies Use intrinsic null hypothesis and compute the probability of an observation falling into a cell in the table under this hypothesis. Partition the total number of observations according to these probabilities. p(A) = 64/160 =.40; p(male) = 97/160 = p(A, male) under H 0 = p(A)p(male) =.2425 f(A, male) = p(A, male) X 160 = 38.8

University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 11/06/2014 3:11 AM 31 Assumptions ( 2 and G) n is larger than 30. Expected frequencies are all larger than 5. Test is quite robust except when there are only 2 categories (df = 1). For 2 categories, both X 2 and G overestimate 2, leading to rejection of null hypothesis with probability greater than i.e. the test is liberal.

University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 11/06/2014 3:11 AM 32 What if n is too small, there are only 2 categories, etc.? Increase n. If n > 2, combine categories. Use a correction factor. Use another test.

University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 11/06/2014 3:11 AM 33 An example: Combining categories With three habitat categories, expected frequencies are too small in 2 cells. Therefore, combine habitats B and C.

University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 11/06/2014 3:11 AM 34 Corrections for 2 categories For 2 categories, both X 2 and G overestimate 2, leading to rejection of null hypothesis with probability greater than i.e. test is liberal Continuity correction: add 0.5 to observed frequencies. Williams correction: divide test statistic (G or 2 ) by: q = 1 + (k 2 - 1)/(6n(k-1))

University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 11/06/2014 3:11 AM 35 Subdividing tables When null hypothesis is rejected, you may wish to determine which categories are contributing substantially to the overall significant test statistic. General procedure: find set of largest homogeneous subtables. Start with smallest homogeneous table, then add rows or columns until the null hypothesis is rejected.

University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 11/06/2014 3:11 AM 36 Subdividing tables Significant interaction

University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 11/06/2014 3:11 AM 37 Subdividing tables Significant interaction

University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 11/06/2014 3:11 AM 38 Subdividing tables No significant interaction

University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 11/06/2014 3:11 AM 39 Subdividing tables Conclusion: B and C are homogeneous, with both differing significantly from A.

University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 11/06/2014 3:11 AM 40 Conclusion Contingency tables are one of the most common methods of analyzing biological data. They provide robust tests (chi-square or G) of independence for categorical data......if sample sizes are adequate and expected frequencies are not too small.

University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 11/06/2014 3:11 AM 41 Multiway tables and log-linear models Notion of interaction extended to consideration of the effects of several different variables (factors) simultaneously… … exactly as in multiple-classification ANOVA.

University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 11/06/2014 3:11 AM 42 Two-way tables : H 0 and model fit For two way tables, the general model includes a constant, two main effects, and an interaction. Thus, independence implies that the goodness- of-fit of a model with the interaction deleted is not significantly different from a model with the interaction included. Interaction out Interaction in Accept H 0 Goodness of fit (e.g. G) Reject H 0 GOF

University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 11/06/2014 3:11 AM 43 Multiway-way tables and log-linear models For 3- way tables, the general model includes a constant, 3 main effects, 3 2-way interactions, and 1 3- way interaction. Thus, independence implies that the goodness-of-fit of a model with the interaction deleted is not significantly different from a model with the interaction included. Interaction out Interaction in Accept H 0 Goodness of fit (e.g. G) Reject H 0 GOF

University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 11/06/2014 3:11 AM 44 Multi-way tables and log-linear models Effects of temperature (H,L) and humidity (H, L) on plant yield (H, L) No 3-way interaction, as interaction between yield and temperature does not depend on humidity. Frequency Yield class Humidity Temperature H L H L Low yield High yield

University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 11/06/2014 3:11 AM 45 Multi-way tables and log- linear models Effects of temperature (H,L) and humidity (H, L) on plant yield (H, L) 3-way interaction, since effect of temperature on yield depends on humidity. Frequency Humidity Temperature H L H L Low yield High yield

University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 11/06/2014 3:11 AM 46 The procedure Test highest order interaction by comparing goodness of fit of full model and model with interaction removed. If non-significant, test next-lowest interactions individually (i.e. with the others included). Where interactions are significant, do separate tests within each category of the factor(s) involved.

University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 11/06/2014 3:11 AM 47 An example: sex-ratio of sturgeon in the lower Saskatchewan River What is the best model that can be fitted to these data? Does sex-ratio depend on location? On year? On location*year?

University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 11/06/2014 3:11 AM 48 Questions/null hypotheses Does the sex ratio vary among years? H 0 : ( ) ij = 0 Does the sex ratio vary between locations? H 0 : ( ) ik = 0 Does the sex ratio vary among (year, location) combinations? H 0 : ( ) ijk = 0

University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 11/06/2014 3:11 AM 49 Fitting log-linear models with SYSTAT Test 3-way interaction by specifying model with 7 terms. Conclusion: accept H 0 Analysis of Deviance Table Poisson model Response: FREQUENC Terms added sequentially (first to last) Df Deviance Resid. Df Resid. Dev Pr(Chi) NULL TEMP LIGHT INFECTED TEMP:LIGHT TEMP:INFECTED LIGHT:INFECTED TEMP:LIGHT:INFECTED

University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 11/06/2014 3:11 AM 50 Residuals (in contingency tables and log-linear models) The difference between observed and expected cell frequencies. There is one residual for each cell in the table. If the fitted model is good, all residuals should be relatively small and there should be no obvious pattern in the table.

University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 11/06/2014 3:11 AM 51 Power and sample size in goodness of fit An external null hypothesis is specified, which specifies a set of expected frequencies, or, alternatively, a set of expected proportions: The effect size is given by: Observed Expected Frequency Cell

University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 11/06/2014 3:11 AM 52 Calculating power given w Given and w and N, we can read 1- from suitable tables or curves (e.g. Cohen (1988), Tables 7-3). 1- Decreasing N = =.01 w

University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 11/06/2014 3:11 AM 53 Power in goodness of fit: an example Biological hypothesis: plumage colour in snow geese controlled by a single autosomal locus with 2 alleles, aa = white, Aa, AA = blue. So Aa X Aa cross should yield segregation ratios: 1 (AA): 2(Aa): 1(aa).

University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 11/06/2014 3:11 AM 54 Power in goodness of fit: an example (contd) H 0 accepted, and effect size given by: From table, So, > 84% chance of Type II error, i.e. probability of detecting a true effect size of.076 is very small.

University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 11/06/2014 3:11 AM 55 Power and sample size in contingency tables Calculate expected cell proportions p 0,ij under H 0 of independence given by marginal proportions: The effect size is given by: Df = (R-1)(C-1)=

University of Ottawa - Bio 4158 – Applied Biostatistics © Antoine Morin and Scott Findlay 11/06/2014 3:11 AM 56 Power and sample size in contingency tables: an example Age structure of two different field mice populations So, about 75% chance of Type II error. Cell proportions, N = 120