Section 1-4 Critical Thinking

Misuses of Statistics 1. Evil intent on the part of dishonest people. 2. Unintentional errors on the part of people who don’t know any better. We should learn to distinguish between statistical conclusions that are likely to be valid and those that are seriously flawed.

Graphs To correctly interpret a graph, you must analyze the numerical information given in the graph, so as not to be misled by the graph’s shape. READ labels and units on the axes!

Pictographs Part (b) is designed to exaggerate the difference by increasing each dimension in proportion to the actual amounts of oil consumption.

Bad Samples Voluntary response sample (or self-selected sample) one in which the respondents themselves decide whether to be included In this case, valid conclusions can be made only about the specific group of people who agree to participate and not about the population.

Correlation and Causality Concluding that one variable causes the other variable when in fact the variables are linked Two variables may seemed linked, smoking and pulse rate, this relationship is called correlation. Cannot conclude the one causes the other. Correlation does not imply causality.

Small Samples Conclusions should not be based on samples that are far too small. Example: Basing a school suspension rate on a sample of only three students

Percentages Misleading or unclear percentages are sometimes used. For example, if you take 100% of a quantity, you take it all. If you have improved 100%, then are you perfect?! 110% of an effort does not make sense.

Loaded Questions If survey questions are not worded carefully, the results of a study can be misleading. Survey questions can be “loaded” or intentionally worded to elicit a desired response. Too little money is being spent on “welfare” versus too little money is being spent on “assistance to the poor.” Results: 19% versus 63%

Order of Questions Questions are unintentionally loaded by such factors as the order of the items being considered. Would you say traffic contributes more or less to air pollution than industry? Results: traffic - 45%; industry - 27% When order reversed. Results: industry - 57%; traffic - 24%

Nonresponse Occurs when someone either refuses to respond to a survey question or is unavailable. People who refuse to talk to pollsters have a view of the world around them that is markedly different than those who will let poll-takers into their homes.

Missing Data *Can dramatically affect results. *Subjects may drop out for reasons unrelated to the study. *People with low incomes are less likely to report their incomes. *US Census suffers from missing people (tend to be homeless or low income).

Self-Interest Study *Some parties with interest to promote will sponsor studies. *Be wary of a survey in which the sponsor can enjoy monetary gain from the results. *When assessing validity of a study, always consider whether the sponsor might influence the results.

Precise Numbers Because a figure is precise, many people incorrectly assume that it is also accurate. A precise number can be an estimate, and it should be referred to that way.

Deliberate Distortion Some studies or surveys are distorted on purpose. The distortion can occur within the context of the data, the source of the data, the sampling method, or the conclusions.

Example 1: Use critical thinking to develop an alternative or correct conclusion: Based on a study showing that college graduates tend to live longer than those who do not graduate from college, a researcher concludes that studying causes people to live longer. College graduates tend to earn more money than non-graduates, and people who have more money are able to purchase better health care. Having more money (whether or not it resulted from having a college degree) and not studying more is the primary contributing factor toward longer life.

Example 2: Use critical thinking to develop an alternative or correct conclusion: Data published in USA Today were used to show that there is a correlation between the number of times songs are played on radio stations and the numbers of times the songs are purchased. Conclusion: Increasing the times that songs are played on radio stations causes sales to increase.

In general, radio stations provide the programming that will attract the highest number of listeners – and thus command the highest advertising rates. In order to please their listeners, it is reasonable to believe that radio stations play most often the songs that were the most popular – i.e., that were being purchased the most often. When it comes to the success of a new song, the radio is more of a lagging indicator than a leading indicator.

Example 3: Use critical thinking to develop an alternative or correct conclusion: A study showed that in Orange County, more speeding tickets were issued to minorities than to whites. Conclusion: In Orange County, minorities speed more than whites.

If the population of Orange County includes significantly more minority drivers than white drivers, one would expect more speeding tickets to be issued to minorities than to whites – even if the percentage of white drivers who violated the speed limit was greater than the percentage of minority drivers who did so. It is also possible that police tend to target minority drivers – so that the numbers of tickets issued to the various racial/ethnic groups does not correspond to the actual amount of speed limit violations occurring. The fact that more speeding tickets are given to minorities does not warrant the conclusion that minority persons are more likely to speed.

Example 4: Use critical thinking to develop an alternative or correct conclusion: In the judicial case United States v. City of Chicago, a minority group failed the Fire Captain Examination at a much higher rate than the majority group. Conclusion: The exam is biased and causes members of the minority group to fail at a much higher rate.

It could be that the exam is a fair instrument to measure the aptitude for the job, but that larger percentages of the members of the minority group fail to test reasons other than the test being biased. There may be fewer opportunities for the minorities, causing a larger number of them (including those who are lesser qualified) to unrealistically attempt the exam.

Example 5: Use critical thinking to address the key issue: In an ABC Nightline poll, 186,000 viewers each paid 50 cents to call a “900” telephone number with their opinion about keeping the United Nations in the United States. The results showed that 67% of those who called were in favor of moving the United Nations out of the United States. Interpret the results by identifying what we can conclude about the way the general population feels about keeping the United Nations in the United Sates.

There are at least two reasons why the results cannot be used to reach any conclusion about how the general population feels about keeping the United Nations in the Unites States. First, only viewers of the ABC “Nightline” program, which are not necessarily representative of the general population, were aware of the poll. Secondly, the 186,000 persons who responded (and there is always a possibility that some people called more than once in order to give their opinion more weight) are voluntary response sample – which means that the responses will include an over representation of those with strong feelings and/or personal interest in the survey.

Example 6: Use critical thinking to address the key issue: The Hawaii State Senate held hearing while considering a law requiring that motorcyclists wear helmets. Some motorcyclists testified that they had been in crashes in which helmets would not have been helpful. Which important group was not able to testify? People who were killed in motorcycle crashes, when a helmet may have saved their lives, were not present to testify.

Example 7: Use critical thinking to address the key issue: Good Housekeeping magazine invited women to visit its Web site to complete a survey, and 1,500 responses were recorded. When asked whether they would rather have more money or more sleep, 88% chose more money and 11% chose more sleep. Based on these results, what can we conclude about the population of all women?

Nothing. Because the information comes from a voluntary response sample, the opinions expressed are more likely to be representative of those with strong feelings on the topic than of the general population. Even if the information had not come from a voluntary response sample, the conclusion would apply only to women who read Good Housekeeping magazine and not to “all women.”

Example 8: In the New York Times Magazine, a report about the decline of Western investment in Kenya included this: “After years of daily flights, Lufthansa and Air France had halted passenger service. Foreign investment fell 500 percent during the 1990s.” What is wrong with this statement? If something falls 100%, there is none remaining. To reduce plaque by 300% would be to eliminate 3 times as much plaque as there is to begin with – which is not physically possible.

Example 9: In a Gallup poll, 49% of 734 surveyed Internet users said that they shop on the Internet frequently or occasionally. What is the actual number of Internet user who said that they shop on the Internet frequently or occasionally? (49%)(734) = (49/100)(734) = 360