1. Should You Believe a Statistical Study?
Chapter 10 Unit B
Should You Believe a Statistical Study?

2. Essential Questions How do we know if a study is reliable?
What are the eight guidelines for evaluating a statistical study?

3. Guideline 1: Big Picture
What was the goal of the study?
What was the population under study?
Was the population clearly and appropriately defined?
Was the study observational or an experiment. If an experiment, was it single or double blind and were the treatment groups properly randomized?

4. Guideline 1: Big Picture - EXAMPLE
Researchers gave 100 participants their astrological horoscopes and asked whether the horoscopes appeared to be accurate; 85% said yes. The researchers concluded that horoscopes are valid most of the time.

5. Guideline 2: Consider the Source
Consider the source of a study
Evaluate the potential for biases that may invalidate its conclusions

6. Guideline 2: Consider the Source - EXAMPLE
By 1963, enough research on the health dangers of smoking so the Surgeon General publicly announced smoking is bad for health. Research mostly shows this, but there were some studies that found no dangers from smoking, and perhaps has health benefits. Those studies were carried out by the Tobacco Research Institute, funded by Tobacco companies

7. Guideline 3: Look for Bias in the Sample
Selection bias: reserachers select their sample in a way that tends to make it unrepresentative of the population (ex: poll only Republicans)
Participation bias: people choose to participate

8. Guideline 3: Look for Bias in the Sample
Literary Digest chose a sample of 10 million from different lists like the telephone book and country club rosters. Then, they mailed out ballots to the 10 million people and 2.4 million returned. They predicted that Alf Landon would win (got 57%) but actually Roosevelt won with 62% of the vote

9. Variable: item or quantity that can vary or take different values
Guideline 4: Look for Problems in Defining or Measuring the Variables of Interest
Variable: item or quantity that can vary or take different values

10. Guideline 4: Look for Problems in Defining or Measuring the Variables of Interest
Survey found that people would pay an average of $487,000 for “true love,” $407,000 for “great intellect,” $285,000 for “talent,” and $259,000 for “eternal youth”

11. Guideline 5: Beware of Confounding Variables
Confounding variables: they confuse a study’s results
May be difficult to spot and sometimes are not discovered at all

12. Guideline 5: Beware of Confounding Variables
Radon is a radioactive gas caused by the decay of uranium. Imagine a study that seeks to determine whether radon gas cause lung cancer by comparing the lung cancer rate in Colorado (radon is fairly common) and lung cancer rate in Hong Kong. If the study finds their rates are nearly the same, is it fair to say radon is not a significant cause of lung cancer?

13. Guideline 6: Consider the Setting and Wording in Surveys
Watch out for problems in the setting or wording that might produce inaccurate or dishonest responses
Be more wary when survey concerns sensitive subjects like personal habits or income

14. Guideline 6: Consider the Setting and Wording in Surveys
The Republican National Committee commissioned a poll to find out whtether Americans supported their proposed tax cuts. Asked “Do you favor a tax cut?” a large majority said yes. Should we conclude that Americans supported the proposal?

15. Guideline 7:Check That Results Are Presented Fairly
May misrepresented in graph or concluding statements
Should look for inconsistencies between the interpretation of a study (pictures and words) and any actual data given

16. Guideline 7:Check That Results Are Presented Fairly
A school board created a hubbub when it announced that 28% of the students were reading “below grade level” so concluded that methods of teaching reading needed to be changed. Based on results that 28% of children scored below national average for their grade. Does the data support the conclusion?

17. Guideline 8: Stand Back and Consider the Conclusions
Did the study achieve its goals?
Do the conclusions make sense?
Can you rule out alternative explanations for the results?
If the conclusions make sense, do they have any practical significance?

18. Guideline 8: Stand Back and Consider the Conclusions
People who tried “Fast Diet Supplement” are compared to weight losses of a control group who try to lose weight in other ways. After 8 weeks, the results show that the treatment group lost an average of ½ lb. more than the control group. Assuming it has no dangerous side effects, does the study suggest “Fast Diet Supplement” is a good treatment for people to lose weight?