1. Statistical Methods Lecture 25

2. Goodness-of-Fit Tests
QTM1310/ Sharpe
Goodness-of-Fit Tests
Given the following…
1) Counts of items in each of several categories
2) A model that predicts the distribution of the relative frequencies
…this question naturally arises:
“Does the actual distribution differ from the model because of random error, or do the differences mean that the model does not fit the data?”
In other words, “How good is the fit?”

3. Goodness-of-Fit Tests
QTM1310/ Sharpe
Goodness-of-Fit Tests
Example : Credit Cards
At a major credit card bank, the percentages of people who historically apply for the Silver, Gold, and Platinum cards are 60%, 30%, and 10% respectively. In a recent sample of customers, 110 applied for Silver, 55 for Gold, and 35 for Platinum. Is there evidence to suggest the percentages have changed?
Null Hypothesis: The distribution of types of credit card applications is no different from the historic distribution.
Test the hypothesis with a chi-square goodness-of-fit test. 3

4. Goodness-of-Fit Tests
QTM1310/ Sharpe
Goodness-of-Fit Tests
Assumptions and Condition
Counted Data Condition – The data must be counts for the categories of a categorical variable.
Independence Assumption – The counts should be independent of each other. Think about whether this is reasonable.
Randomization Condition – The counted individuals should be a random sample of the population. Guard against auto-correlated samples.
auto-correlated—just means any sample where one count going up or down, due to sample size typically requires another of the counts to go up or down in response 4

5. Goodness-of-Fit Tests
QTM1310/ Sharpe
Goodness-of-Fit Tests
Sample Size Assumption
There must be enough data so check the following condition:
Expected Cell Frequency Condition – must be at least 5 individuals per cell. 5

6. Goodness-of-Fit Tests
QTM1310/ Sharpe
Goodness-of-Fit Tests
Chi-Square Model
To decide if the null model is plausible, look at the differences between the observed values and the values expected if the model were true.
Note that c2 “accumulates” the relative squared deviation of each cell from its expected value.
So, c2 gets “big” when
i) the data set is large and/or
ii) the model is a poor fit. 6

7. Goodness-of-Fit Tests
QTM1310/ Sharpe
Goodness-of-Fit Tests
The Chi-Square Calculation
Find the expected values. These come from the null hypothesis value.
Compute the residuals,
Square the residuals,
Compute the components. Find for each cell.
Find the sum of the components,
Find the degrees of freedom (no. of cells – 1)
Test the hypothesis, finding the p-value or comparing the test statistic from 5 to the appropriate critical value. 7

8. Goodness-of-Fit Tests
QTM1310/ Sharpe
Goodness-of-Fit Tests
Example : Credit Cards
At a major credit card bank, the percentages of people who historically apply for the Silver, Gold, and Platinum cards are 60%, 30%, and 10% respectively. In a recent sample of customers, 110 applied for Silver, 55 for Gold, and 35 for Platinum. Is there evidence to suggest the percentages have changed?
What type of test do you conduct?
What are the expected values?
Find the test statistic and p-value.
State conclusions. 8

9. Goodness-of-Fit Tests
QTM1310/ Sharpe
Goodness-of-Fit Tests
Example : Credit Cards
At a major credit card bank, the percentages of people who historically apply for the Silver, Gold, and Platinum cards are 60%, 30%, and 10% respectively. In a recent sample of customers, 110 applied for Silver, 55 for Gold, and 35 for Platinum. Is there evidence to suggest the percentages have changed?
What type of test do you conduct?
This is a goodness-of-fit test comparing a single sample to previous information (the null model).
What are the expected values?
Silver
Gold
Platinum
Observed
110 55 35
Expected
120 60 20 9

10. Goodness-of-Fit Tests
QTM1310/ Sharpe
Goodness-of-Fit Tests
Example : Credit Cards
At a major credit card bank, the percentages of people who historically apply for the Silver, Gold, and Platinum cards are 60%, 30%, and 10% respectively. In a recent sample of customers, 110 applied for Silver, 55 for Gold, and 35 for Platinum. Is there evidence to suggest the percentages have changed?
Find the test statistic
and p-value. ??????? 10

11. Interpreting Chi-Square Values
QTM1310/ Sharpe
Interpreting Chi-Square Values
The Chi-Square Distribution
The c2 distribution is right-skewed and becomes broader with increasing degrees of freedom:
mode at df – 2
mean is df
The c2 test is a one-sided test. 11

12. Goodness-of-Fit Tests
QTM1310/ Sharpe
Goodness-of-Fit Tests
Example : Credit Cards
Is there evidence to suggest the percentages have changed?
With the test statistic c2 = , find the p-value:
Using df = 2 and technology (Excel: “=1 - CHISQ.DIST(12.499, 2, TRUE)”, the p-value =
State conclusions.
Reject the null hypothesis. There is sufficient evidence customers are not applying for cards in the traditional proportions.
=CHIDIST(12.499,2) 12

13. Examining the Residuals
QTM1310/ Sharpe
Examining the Residuals
When we reject a null hypothesis, we can examine the residuals in each cell to discover which values are extraordinary.
Because we might compare residuals for cells with very different counts, we should examine standardized residuals:
Note that standardized residuals from goodness-of-fit tests are distributed as z-scores (which we already know how to interpret and analyze). 13

14. Examining the Residuals
QTM1310/ Sharpe
Examining the Residuals
Standardized residuals for the credit card data:
Card Type
Standardized
Residual
Silver
Gold
Platinum
Neither of the Silver nor Gold values is remarkable.
The largest, Platinum, at 3.35, is where the difference from historic values lies. 14

15. The Chi-Square Test for Homogeneity
QTM1310/ Sharpe
The Chi-Square Test for Homogeneity
Assumptions and Conditions
Counted Data Condition – Data must be counts
Independence Assumption – Counts need to be independent from each other. Check for randomization
Randomization Condition – Random samples /stratified sample needed
Sample Size Assumption – There must be enough data so check the following condition.
Expected Cell Frequency Condition – Expect at least 5 individuals per cell. 15

16. The Chi-Square Test for Homogeneity
QTM1310/ Sharpe
The Chi-Square Test for Homogeneity
Following the pattern of the goodness-of-fit test, compute the component for each cell:
Then, sum the components:
The degrees of freedom are 16

17. The Chi-Square Test for Homogeneity
QTM1310/ Sharpe
The Chi-Square Test for Homogeneity
Example: More Credit Cards
A market researcher for the credit card bank wants to know if the distribution of applications by card is the same for the past 3 mailings. She takes a random sample of 200 from each mailing and counts the number of applications for each type of card.
Type of Card
Silver
Gold
Platinum
Total
Mailing 1
120 50 30
200
Mailing 2
115 35
Mailing 3
105 55 40
340
155
600 17

18. The Chi-Square Test for Homogeneity
QTM1310/ Sharpe
The Chi-Square Test for Homogeneity
Example: More Credit Cards
A market researcher for the credit card bank wants to know if the distribution of applications by card is the same for the past 3 mailings.
But, are the differences real or just natural sampling variation?
Our null hypothesis is that the relative frequency distributions are the same (homogeneous) for each country.
Test the hypothesis with a chi-square test for homogeneity. 18

19. The Chi-Square Test for Homogeneity
QTM1310/ Sharpe
The Chi-Square Test for Homogeneity
Example: More Credit Cards
A market researcher for the credit card bank wants to know if the distribution of applications by card is the same for the past 3 mailings.
Use the total % to determine the expected counts for each table column (type of card):
Type of Card
Silver
Gold
Platinum
Total
Mailing 1
113.33
51.67 35
200
Mailing 2
Mailing 3
340
155
105
600
= 200*(340/600)
51.67 = 200*(155/600)
35 = 200*(1/5/600) 19

20. The Chi-Square Test for Homogeneity
QTM1310/ Sharpe
Example : More Credit Cards
A market researcher for the credit card bank wants to know if the distribution of applications by card is the same for the past 3 mailings. She takes a random sample of 200 from each mailing and counts the number of applications for each type of card.
Find the test statistic.
Given p-value = ,state conclusions.
Fail to reject the null. There is insufficient evidence to suggest that the distributions are different for the three mailings. 20

21. Looking back
Recognize when a chi-square test of goodness of fit or homogeneity is appropriate.
For each test, find the expected cell frequencies.
For each test, check the assumptions and corresponding conditions and know how to complete the test.
Interpret a chi-square test.
Examine the standardized residuals