1. Contingency tables and goodness of fit
Chapter 13
Contingency tables and goodness of fit

2. Part 1 - Contingency Tables
We’ll begin with a simple example and then extend it to create a very general test.

3. Example Xi ~ i.i.d. BER(p) H0: p = p0 H1: p ≠ p0
Derive an asymptotic test and show that it is of the form:
sum of (observed – expected)2/expected
Identify the distribution of the test statistic (under the null).
Find the p-value with the following data (p0 = ½):
Outcome of test statistic = 5/2.
P-value = 1 - pchisq(5/2, df = 1) = 0.11
Observed (expected)
Successes
Failures
15 (20)
25 (20)

4. Example
A 6-sided die is rolled 60 times, and we want to test if it is fair.
Find the test statistic and its distribution under the null.
Discuss the term ‘degrees of freedom.’
Compute the p-value.
Number
Observed (expected) 1
8 (10) 2
11 (10) 3
5 (10) 4
12 (10) 5
15 (10) 6
9 (10)
Outcome of test statistic = 6
P-value = 1-pchisq(6,df=5) = 0.31

5. Example 13.3.1 A certain characteristic is believed to be present in
20% of Race 1
20% of Race 2
20% of Race 3
From a sample we find the characteristic present in
20/50 from Race 1
25/100 from Race 2
15/50 from Race 3
Write the 2-way contingency table (race by characteristic status) with expected an actual counts.
Identify the test statistic and its distribution under the null.
Calculate the outcome of the test statistic and the p-value.
Outcome of test statistic is 17.8.
P-value is 1-pchisq(17.8, df = 3) =

6. Example
Repeat the previous example with the null hypothesis that the presence of the characteristic is that same in each race (but not any value in particular).
H0: p1 = p2 = p3
Demonstrate how to estimate p and use it to obtain expected counts.
Demonstrate that both rows and columns add to zero, so df = 2.
You can also think about this as losing a degree of freedom for each parameter estimated.
Outcome of test statistic = 3.57.
P-value = 1 - pchisq(3.57, df=2) = 0.17

7. Example
Is political affiliation associated with the space program?
A survey was given asking political affiliation and when we should increase, decrease or maintain current support for the space program.
This data is fake.
Increase
Same
Decrease
Total
Republican 8 12 10 30
Democrat 17 13 40
Independent 6 35

8. Example
Is political affiliation associated with the space program?
A survey was given asking political affiliation and when we should increase, decrease or maintain current support for the space program.
This data is fake.
Increase
Same
Decrease
Total
Republican
8 (9)
12 (10.5)
10 (10.5) 30
Democrat
10 (12)
17 (14)
13 (14) 40
Independent
12 (9)
6 (10.5) 35
Demonstrate that rows and columns sum to zero, giving 4 degrees of freedom.
Outcome of test statistic = 4.54
P-value = 1 - pchisq(4.54, df=4) = 0.34

9. A real example
Remind the class that the chi-squared test statistic has a distribution under the null that is only APPROXIMATELY chi-squared.
Provide the rule of thumb that the expected cell count in 75% or more of the cells should be 5 or more.

10. Fisher’s exact test
When the chi-squared test is appropriate, it will give results very similar to Fisher’s exact test.

11. Goodness of fit
We can test whether observed data came from a particular distribution.
Until now, we have assumed we knew the family that the population distribution belonged to. For example, to find the MLE, we must know the likelihood as a function of the unknown parameter. That assumes that the only thing we don’t know about the distribution is the parameter.

12. Example 13.7.1 Let Xi be the repair times for an airplane part.
H0: Xi ~ i.i.d. POI(3)
How can we use a contingency table to test this hypothesis?
How do we determine expected cell counts?
Repair time (Days) 1 2 3 4 5 6 7+ 7 10

13. Example
Expected counts can be determined based on the null hypothesis and the associated mass function.
H0: Xi ~ i.i.d. POI(3)
There’s a problem. What is it? How do we fix it?
Repair time (Days) 1 2 3 4 5 6 7+
1 (2.00)
3 (5.96)
7 (8.96)
6 (8.96)
10 (6.72)
7 (4.04)
6 (2.00)
0 (1.36)
Rule of thumb for approximation not met. Too many cells with low expected cell count.

14. Example
Cells can be combined to create cells with larger expected counts.
There’s a problem. What is it? How do we fix it?
Compute the p-value.
Repair time (Days)
0 - 1 2 3 4 5+
4 (7.96)
7 (8.96)
6 (8.96)
10 (6.72)
13 (7.40)
By arbitrarily choosing which bins to combine, I can get different p-values. I could choose bins in order to for the p-value to be lower or higher depending on what I want to show. Fix this by specifying the bins a priori.
P-value = 1-pchisq(9.22, df = 4) = Note that observed - expected sum to zero, because there are 40 observations and each one will be in a bin.

15. Example
Repeat the previous example with a composite null hypothesis.
H0: Xi ~ i.i.d. POI(μ)
How do we determine expected cell counts?
Don’t forget to combine cells as needed.
(How and when to choose?)
Compute the p-value.
What conclusions can we make about the distribution at α = 0.05?
Repair time (Days)
0 - 1 2 3 4 5+
4 (4.84)
7 (6.92)
6 (8.44)
10 (7.68)
13 (12.12)
By arbitrarily choosing which bins to combine, I can get different p-values. I could choose bins in order to for the p-value to be lower or higher depending on what I want to show. Fix this by specifying the bins a priori.
P-value = 1-pchisq(1.62, df = 3) = Note that a degree of freedom is lost for the parameter we estimated.
Technically the approximate p-value is between 1-pchisq(1.62, df = 3) = and 1-pchisq(1.62, df = 4) = , since the expected counts are based on the regular MLE rather than the MLE for binned data.

16. MLE for binned data The MLE for binned data is a bit different.
In Example , the likelihood function for binned data (in 5 categories) is
L(μ) = n!/(4!7!6!10!13!) ⋅ [p1(μ)]4[p2(μ)]7[p3(μ)]6[p4(μ)]10[p5(μ)]13
Log[L(μ)] ∝ 4log(p1(μ)) + …
= 4log(e-μ(1+μ)) + ...
The derivatives may not be quite as bad as they look, but with more than a few bins, it would be unpleasant to maximize this.
You will not be asked to do this on quizzes or exams.
On Hw 13.15, assume that observed data are always at the median of the bin, and on Hw 13.16, assume that the observation that is ≥5 is actually 5.

17. Other goodness of fit tests
Kolmogorov-Smirnov
See image to the right
Shapiro-Wilk
For testing normality