Hypothesis Testing Part III – Applying the Concepts

This video is designed to accompany pages in Making Sense of Uncertainty Activities for Teaching Statistical Reasoning Van-Griner Publishing Company

Multivitamins and Cancer

Article Analysis When you see “statistically significant” or “not statistically significant” you have to: 1.Establish what was being compared. 2.Express this comparison appropriately as a choice between a null and an alternative hypothesis. 3.Determine how the comparison turned out. 4.Articulate the risk associated with the decision made.

Multivitamins and Cancer 1.Number of cancers seen in group taking multivitamins and the number of cancers seen in group taking placebo. 2.In terms of H0 and HA: a)H0: Number of cancers in vitamin group is same as in placebo group. b)HA: Number of cancers in vitamin group is less than in placebo group. 3.Results were statistically significant so you know that the decision was to accept HA, that multivitamin usage reduces cancer risk. 4.There is always a chance that this was the wrong decision, that the multivitamins are not effective at reducing cancer risk. We know that in this case the “false positive” risk is less than 5 chances in 100.

Subway versus McDonalds

Subway Versus McDonald’s 1.Calories consumed by teens at Subway and those consumed by teens at McDonald’s. 2.In terms of H0 and HA: a)H0: Number of calories consumed by teens at Subway is the same as at McDonald’s. b)HA: Number of calories consumed by teens at Subway is less than at McDonald’s. 3.Results were NOT statistically significant so you know that the decision was to not accept HA, so no evidence that Subway is a practically healthier food choice for teens. 4.There is always a chance that this was the wrong decision, that in the whole population of teens fewer calories are consumed at Subway. We don’t monitor this “False Negative Rate” explicitly, but assume the method of comparison had robust sensitivity.

Mental Illness and Obesity

1.Weight before and after patients complete fitness program for mentally ill. 2.In terms of H0 and HA: a)H0: Weight before fitness program is no different than weight after fitness program b)HA: Weight before fitness program is less than weight after fitness program. 3.Results were statistically significant so you know that the decision was to accept HA, that the fitness programs to lead to weight loss. 4.There is always a chance that this was the wrong decision, that the fitness programs are not effective at reducing weight in the larger population. We know that in this case the “false positive” risk is less than 5 chances in 100.

Presidential Payment

Presidential Pay and Performance 1.Presidential pay at low achieving universities and Presidential pay at high achieving universities 2.In terms of H0 and HA: a)H0: There is no correlation between Presidential pay and Performance of a University. b)HA: There is a positive correlation between Presidential pay and the Performance of a University. 3.Results were NOT statistically significant so you know that the decision was to not accept HA, so no evidence that universities that pay their Presidents more get better results in terms of performance. 4.There is always a chance that this was the wrong decision, that in the whole population of universities (not just those 145 studied) there is a correlation that is significant. We don’t monitor this “False Negative Rate” explicitly, but assume the method of comparison had robust sensitivity.

Streaks and Basketball

1.Streaks of free throw successes versus what player would make at random. 2.In terms of H0 and HA: a)H0: Streaks (“Runs”) are no longer than would be expected by random chance. b)HA: Streaks (“Runs”) are longer than would be expected by random chance. 3.Results were statistically significant so you know that the decision was to accept HA, that there is evidence streaks are longer than expected by random chance. 4.There is always a chance that this was the wrong decision, that the streaks are not longer than random chance in the larger population. We know that in this case the “false positive” risk is less than 5 chances in 100.

One-Sentence Reflection To understand the usage of “statistical significance” in the media, always ask what is being compared, what the null and alternatives are, how the comparison turned out, and what risks were involved in the decision that was made.