1. 1-1 Copyright © 2015, 2010, 2007 Pearson Education, Inc. Chapter 25, Slide 1 Chapter 26 Comparing Counts

2. 1-2 Copyright © 2015, 2010, 2007 Pearson Education, Inc. Chapter 25, Slide 2 Chapter 26 Introduction Does everyone know what a Zodiac sign is? After that is clarified, let’s look at some data to see if we can answer this question: Is your Zodiac sign a good predictor of how successful you will be later in life? Fortune magazine randomly collected the Zodiac signs for 256 company heads, and the data is in the table on the next slide….

3. 1-3 Copyright © 2015, 2010, 2007 Pearson Education, Inc. Chapter 25, Slide 3 Chapter 26 Introduction Birth Counts Zodiac Sign 23Aries 20Taurus 18Gemini 23Cancer 20Leo 19Virgo 18Libra 21Scorpio 19Sagittarius 22Capricorn 24Aquarius 29Pisces The answer to the question will basically end up being that the sign does or does not have the capability to predict success. What would have to happen (or what would you expect) for the conclusion to be that it does not? That it does? What would the hypotheses look like?

4. 1-4 Copyright © 2015, 2010, 2007 Pearson Education, Inc. Chapter 25, Slide 4 Goodness-of-Fit A test of whether the distribution of counts in one categorical variable matches the distribution predicted by a model is called a goodness-of-fit test. More technically worded, this test sees if there is a pattern or nawh. As usual, there are assumptions and conditions to consider, but we have basically already established them.

5. 1-5 Copyright © 2015, 2010, 2007 Pearson Education, Inc. Chapter 25, Slide 5 Assumptions and Conditions categorical variable Counted Data Condition: Check that the data are counts for the categories of a categorical variable. Independence Assumption: The counts in the cells should be independent of each other. Randomization Condition: The individuals who have been counted and whose counts are available for analysis should be a random sample from some population. 10% Condition: Sample is less than 10% of population.

6. 1-6 Copyright © 2015, 2010, 2007 Pearson Education, Inc. Chapter 25, Slide 6 Assumptions and Conditions (cont.) Sample Size Assumption: We must have enough data for the methods to work. Expected Cell Frequency Condition: We should expect to see at least 5 individuals in each cell. This is similar to the condition that np and nq be at least 10 when we tested proportions.

7. 1-7 Copyright © 2015, 2010, 2007 Pearson Education, Inc. Chapter 25, Slide 7 No Confidence Intervals Here The reaction in the hallways: Person #1: Yay! Person #2: No manches! Bueno, por que? Person #1: No importa. Person #2: Si, neta. Orale.

8. 1-8 Copyright © 2015, 2010, 2007 Pearson Education, Inc. Chapter 25, Slide 8 New Distribution!!! Called the Chi-Square Distribution This is a family of curves, just the t-distribution. Which curve depends on degrees of freedom, which is still n-1, but here the n is the NUMBER OF CATERGORIES!!! Not symmetrical- skewed to the right.

9. 1-9 Copyright © 2015, 2010, 2007 Pearson Education, Inc. Chapter 25, Slide 9 Calculations Since we want to examine how well the observed data reflect what would be expected, it is natural to look at the differences between the observed and expected counts (Obs – Exp). These differences are actually residuals, so we know that adding all of the differences will result in a sum of 0. That’s not very helpful.

10. 1-10 Copyright © 2015, 2010, 2007 Pearson Education, Inc. Chapter 25, Slide 10 Calculations (cont.) We’ll handle the residuals as we did in regression, by squaring them. To get an idea of the relative sizes of the differences, we will divide each squared difference by the expected count from that cell.

11. 1-11 Copyright © 2015, 2010, 2007 Pearson Education, Inc. Chapter 25, Slide 11 Calculations (cont.) The test statistic, called the chi-square (or chi- squared) statistic, is found by adding up the sum of the squares of the deviations between the observed and expected counts divided by the expected counts: This is done in the calculator, but understand what is going on in that magical device and know this equation. This is done in the calculator, but understand what is going on in that magical device and know this equation.

12. 1-12 Copyright © 2015, 2010, 2007 Pearson Education, Inc. Chapter 25, Slide 12 Calculations (cont.) The chi-square models are actually a family of distributions indexed by degrees of freedom (much like the t-distribution). where n is the number of categories The number of degrees of freedom for a goodness-of-fit test is n – 1, where n is the number of categories.

13. 1-13 Copyright © 2015, 2010, 2007 Pearson Education, Inc. Chapter 25, Slide 13 One-Sided or Two-Sided? If the observed counts don’t match the expected, the statistic will be large—it can’t be “too small.” Why not??? So the chi-square test is always one-sided. If the calculated statistic value is large enough, this will result in a low P-value, and thus we’ll reject the null hypothesis. The process of the test determines if the difference is “large enough.”

14. 1-14 Copyright © 2015, 2010, 2007 Pearson Education, Inc. Chapter 25, Slide 14 One-Sided or Two-Sided? (cont.) The mechanics may work like a one-sided test, but the interpretation of a chi-square test is in some ways many-sided. There are many ways the null hypothesis could be wrong. There’s no direction to the rejection of the null model—all we know is that it doesn’t fit. Interpretation- the test doesn’t say where the difference is, just that the null model is not correct.

15. 1-15 Copyright © 2015, 2010, 2007 Pearson Education, Inc. Chapter 25, Slide 15 The Chi-Square Calculation 1.Find the expected values: Every model gives a hypothesized proportion for each cell. The expected value is the product of the total number of observations times this proportion. 2.Compute the residuals: Once you have expected values for each cell, find the residuals, Observed – Expected. 3.Square the residuals.

16. 1-16 Copyright © 2015, 2010, 2007 Pearson Education, Inc. Chapter 25, Slide 16 The Chi-Square Calculation (cont.) 4.Compute the components. Now find the components for each cell. 5.Find the sum of the components (that’s the chi- square statistic).

17. 1-17 Copyright © 2015, 2010, 2007 Pearson Education, Inc. Chapter 25, Slide 17 The Chi-Square Calculation (cont.) 6.Find the degrees of freedom. It’s equal to the number of cells minus one. 7.Test the hypothesis.  Use your chi-square statistic to find the P- value. (Remember, you’ll always have a one- sided test.)  Large chi-square values mean lots of deviation from the hypothesized model, so they give small P-values.

18. 1-18 Copyright © 2015, 2010, 2007 Pearson Education, Inc. Chapter 25, Slide 18 For-Goodness-of-Fit Test Give the hypotheses. Check the four conditions. Say you can use a chi-square model with d.f.=n-1 to perform a chi-square goodness-of-fit test. Write = # (calculator). Draw the curve with this number and the right tail shaded in. Give the P-value of this statistic. Conclude the test- reject/don’t reject the null based on the P-value and alpha level, apply to alternative.

19. 1-19 Copyright © 2015, 2010, 2007 Pearson Education, Inc. Chapter 25, Slide 19 But I Believe the Model… Goodness-of-fit tests are likely to be performed by people who have a theory of what the proportions should be, and who believe their theory to be true. Unfortunately, the only null hypothesis available for a goodness-of-fit test is that the theory is true. As we know, the hypothesis testing procedure allows us only to reject or fail to reject the null.

20. 1-20 Copyright © 2015, 2010, 2007 Pearson Education, Inc. Chapter 25, Slide 20 But I Believe the Model… (cont.) We can never confirm that a theory is in fact true. At best, we can point out only that the data are consistent with the proposed theory. Remember, it’s that idea of “not guilty” versus “innocent.” It’s like if you tried to assume that there is a difference/preference and then prove that there is none, the difference can be anywhere, so it wouldn’t work. So….let’s think about a different type of test.

21. 1-21 Copyright © 2015, 2010, 2007 Pearson Education, Inc. Chapter 25, Slide 21 Teaching Tips A box of Froot Loops (or other colored food or candy) is a great way to introduce Goodness-of- Fit. Not all calculators have the Goodness-of-fit test installed. But as it is on a “legal” calculator, it can be added to any other calculator for class and AP test use.

22. 1-22 Copyright © 2015, 2010, 2007 Pearson Education, Inc. Chapter 25, Slide 22 Example Suppose a university wants to know what their graduates are planning on doing in all of their different colleges. You could take the count from each college that is planning on doing each post-graduate path, and then see what a test says about this information. So, this is different groups, seeing if there is a difference among them or nawh. GOF looks at one group broken up.

23. 1-23 Copyright © 2015, 2010, 2007 Pearson Education, Inc. Chapter 25, Slide 23 Comparing Observed Distributions Homogeneity- things are the same. A test comparing the distribution of counts for two or more groups on the same categorical variable is called a chi-square test of homogeneity. A test of homogeneity is actually the generalization of the two-proportion z-test. It’s very similar to the GOF, but slightly different.

24. 1-24 Copyright © 2015, 2010, 2007 Pearson Education, Inc. Chapter 25, Slide 24 Comparing Observed Distributions (cont.) The statistic that we calculate for this test is identical to the chi-square statistic for goodness- of-fit. In this test, however, we ask whether choices are the same among different groups (i.e., there is no model). The expected counts are found directly from the data and we have different degrees of freedom.

25. 1-25 Copyright © 2015, 2010, 2007 Pearson Education, Inc. Chapter 25, Slide 25 GOF VS. HOMOGENEITY GOF: Compares distribution of obs. outcomes of for a single variable to the expected outcomes predicted by a model (given in null) to see if the model is viable. HOMO: Compares distribution of obs. outcomes for several groups to each other to see if there is evidence of differences among the populations (null is that there differences among categories for different groups).

26. 1-26 Copyright © 2015, 2010, 2007 Pearson Education, Inc. Chapter 25, Slide 26 Assumptions and Conditions The assumptions and conditions are the same as for the chi-square goodness-of-fit test: Counted Data Condition: The data must be counts. Randomization Condition and 10% Condition: Same as before. Expected Cell Frequency Condition: The expected count in each cell must be at least 5. This is given in the residual matrix (B).

27. 1-27 Copyright © 2015, 2010, 2007 Pearson Education, Inc. Chapter 25, Slide 27 Calculations To find the expected counts, we multiply the row total by the column total and divide by the grand total. We calculated the chi-square statistic as we did in the goodness-of-fit test: In this situation we have (R – 1)(C – 1) degrees of freedom, where R is the number of rows and C is the number of columns. We’ll need the degrees of freedom to find a P-value for the chi-square statistic.

28. 1-28 Copyright © 2015, 2010, 2007 Pearson Education, Inc. Chapter 25, Slide 28 Chi- Square Test of Homogeneity Give the hypotheses. Check the four conditions. Say you can use a chi-square model with d.f.= (R- 1)(C-1) to perform a chi-square test of homogeneity. Write = # (calculator). Enter table in a matrix to perform the testing in the calc. Draw the curve with this number and the right tail shaded in. Give the P-value of this statistic. Conclude the test- reject/don’t reject the null based on the P-value and alpha level, apply to alternative.

29. 1-29 Copyright © 2015, 2010, 2007 Pearson Education, Inc. Chapter 25, Slide 29 Examining the Residuals When we reject the null hypothesis, it’s always a good idea to examine residuals. For chi-square tests, we want to work with standardized residuals, since we want to compare residuals for cells that may have very different counts. To standardize a cell’s residual, we just divide by the square root of its expected value:

30. 1-30 Copyright © 2015, 2010, 2007 Pearson Education, Inc. Chapter 25, Slide 30 Examining the Residuals (cont.) These standardized residuals are just the square roots of the components we calculated for each cell, with the + or the – sign indicating whether we observed more cases than we expected, or fewer. The standardized residuals give us a chance to think about the underlying patterns and to consider the ways in which the distribution might not match what we hypothesized to be true.

31. 1-31 Copyright © 2015, 2010, 2007 Pearson Education, Inc. Chapter 25, Slide 31 Independence Contingency tables categorize counts on two (or more) variables so that we can see whether the distribution of counts on one variable is contingent on the other. A test of whether the two categorical variables are independent examines the distribution of counts for one group of individuals classified according to both variables in a contingency table. A chi-square test of independence uses the same calculation as a test of homogeneity.

32. 1-32 Copyright © 2015, 2010, 2007 Pearson Education, Inc. Chapter 25, Slide 32 In other words… A test of independence cross-categorizes one group on two variables to see if there is an association between them. One sample, two variables. Null: there is no association between the two variables.

33. 1-33 Copyright © 2015, 2010, 2007 Pearson Education, Inc. Chapter 25, Slide 33 Assumptions and Conditions Same ol same ol. Check the residuals that the calculator will store in matrix (B) to see if you should expect at least 5 counts in each category.

34. 1-34 Copyright © 2015, 2010, 2007 Pearson Education, Inc. Chapter 25, Slide 34 Examine the Residuals Each cell of a contingency table contributes a term to the chi-square sum. It helps to examine the standardized residuals, just like we did for tests of homogeneity.

35. 1-35 Copyright © 2015, 2010, 2007 Pearson Education, Inc. Chapter 25, Slide 35 Chi- Square Test of Independence Give the hypotheses. Check the four conditions. Say you can use a chi-square model with d.f.= (R- 1)(C-1) to perform a chi-square test of independence. Write = # (calculator). Enter table in a matrix to perform the testing in the calc. Draw the curve with this number and the right tail shaded in. Give the P-value of this statistic. Conclude the test- reject/don’t reject the null based on the P-value and alpha level, apply to alternative.

36. 1-36 Copyright © 2015, 2010, 2007 Pearson Education, Inc. Chapter 25, Slide 36 Chi-Square and Causation Chi-square tests are common, and tests for independence are especially widespread. We need to remember that a small P-value is not proof of causation. Since the chi-square test for independence treats the two variables symmetrically, we cannot differentiate the direction of any possible causation even if it existed. And, there’s never any way to eliminate the possibility that a lurking variable is responsible for the lack of independence.

37. 1-37 Copyright © 2015, 2010, 2007 Pearson Education, Inc. Chapter 25, Slide 37 Chi-Square and Causation (cont.) In some ways, a failure of independence between two categorical variables is less impressive than a strong, consistent, linear association between quantitative variables. Two categorical variables can fail the test of independence in many ways. Examining the standardized residuals can help you think about the underlying patterns.

38. 1-38 Copyright © 2015, 2010, 2007 Pearson Education, Inc. Chapter 25, Slide 38 What Can Go Wrong? Don’t use chi-square methods unless you have counts. Just because numbers are in a two-way table doesn’t make them suitable for chi-square analysis. Beware large samples. With a sufficiently large sample size, a chi-square test can always reject the null hypothesis. Don’t say that one variable “depends” on the other just because they’re not independent. Association is not causation.

39. 1-39 Copyright © 2015, 2010, 2007 Pearson Education, Inc. Chapter 25, Slide 39 What have we learned? We’ve learned how to test hypotheses about categorical variables. All three methods we examined look at counts of data in categories and rely on chi-square models. Goodness-of-fit tests compare the observed distribution of a single categorical variable to an expected distribution based on theory or model. Tests of homogeneity compare the distribution of several groups for the same categorical variable. Tests of independence examine counts from a single group for evidence of an association between two categorical variables.

40. 1-40 Copyright © 2015, 2010, 2007 Pearson Education, Inc. Chapter 25, Slide 40 What have we learned? (cont.) Mechanically, these tests are almost identical. While the tests appear to be one-sided, conceptually they are many-sided, because there are many ways that the data can deviate significantly from what we hypothesize. When we reject the null hypothesis, we know to examine standardized residuals to better understand the patterns in the data.

41. 1-41 Copyright © 2015, 2010, 2007 Pearson Education, Inc. Chapter 25, Slide 41 AP Tips You have to check that the expecteds are greater than 5. That means you need to write all the expecteds on your paper to show that you actually checked this. Writing the correct hypotheses is one of the biggest challenges. Make sure you can do this correctly. And use the wording given to you in the problem to write them.