Statistics as Evidence Arguing by the numbers

impressing with statistics  approximately one in five children ages who use the Internet frequently have received a sexual solicitation or approach over the Internet.  the mortality rate for avian flu is more than 50 percent.  The average L.A. driver spends 136 hours per year in rush hour traffic  A typical child has seen 4,286 acts of violence on TV by age 18.  Bill Gates makes more money than the poorest 100 million Americans

But are the numbers really that impressive?  An article in the New England Journal of Medicine stated “a person who purchases a handgun is 57 times more likely to commit suicide within a week of buying the weapon than the general population as a whole.”  The study said suicide is the leading cause of death among gun buyers, in the first year after a weapon is purchased.  The statistic is misleading, however, because buying a handgun doesn’t make one more likely to commit suicide, wanting to commit suicide makes one more likely to buy a handgun.  People who are bent on committing suicide probably buy more knives, rope, sleeping pills, and razor blades, too.

Are the numbers accurate? Kevin & Bean, KROQ 10/9/08: “Tigers have killed kill 300,000 people in India.” Wikipedia: “Although humans are not regular prey for it, the tiger has killed more people than any other cat. Between 1800 and 1900, it is estimated that tigers had killed over 300,000 people in India alone” Actually, about 300,000 people live in the Sundarban area of Bangladesh, where Royal Bengal tigers are found Tigers are responsible for about 10 deaths a year. (Reuters, Jan 23, 2008, reported by Enamul Haque) Valmik Thapar, a leading expert on tigers in India, put deaths due to tiger attacks at about 80 people annually. (The Statesman, New Delhi, July 16, 2001)

Are the numbers accurate? On average men think about sex every 7 seconds. 54% of men think about sex everyday or several times a day, 43% a few times per month or a few times per week, and 4% less than once a month (Kinsey Report from KinseyInstitute.org)

watch out for “unknowable” statistics  example: The average person tells 13 lies per week  example: There are at least 30 million victims of modern day slavery in the world today (US Trafficking in Persons Report).  example: “in up to 98% of the cases, simply brandishing a gun is sufficient to stop a crime.” John Lott Jr., Wall street Journal July,30, 2001

comparing non-comparable units (apples versus oranges) A study conducted by the dean of admission for Harvard University concluded that Harvard students who took prep courses scored lower than students who didn’t take prep courses. The study concluded “the coaching industry is playing on parental anxiety.” –However, the comparison is not based non-equivalent samples of students. Students who didn’t even bother taking a prep course might be much stronger academically than students who opted to take such a course. No coaching coacheddifference VSAT MSA T

watch out for unrepresentative samples  Non-random samples:  voting on the American Idol show  phone calls or s to a politician about a particular bill  testimonials on infomercials  talk radio callers  To be truly valid a sample must be:  random  representative  sufficient

questionable extrapolations  Using a limited sample (limited in time or number) and extrapolating the percentages nationwide or globally  example: According to the National Self Defense Survey in 1994, the rate of Defensive Gun Uses can be projected nationwide to approximately 2.5 million per year -- one Defensive Gun Use every 13 seconds. 

questionable extrapolation Babe Ruth would have hit 1,051 home runs if he had the same number of at bats as Hank Aaron. Ruth hit home runs per at bat, Aaron had 12,364 at bats

keep the numbers in perspective  Compare “scary” statistics to other, well-established health risks.  “mad cow disease” or the “flesh eating virus” sound scary, but the odds are much greater of dying of dog bites, drowning in the tub, or being struck by lightning.  Chicken pox kills 100 people per year in America.  Flu kills about 35,000 people annually in the U.S.

Keep the numbers in perspective  School killings  Total school-related violent deaths, August 1, 2005, July 31, 2006:  Shooting: 15 Suicides: 1 Murder-Suicide: 4  Fight-Related: 0  Stabbing: 3  Other: 4  Total = 27  Compared to drownings and car crashes  In 2003, 782 children ages 0 to 14 years died from drowning (CDC 2005).  In the United States during 2004, 1,638 children ages 14 years and younger died as occupants in motor vehicle crashes

Correct interpretation? Does owning a gun make it more likely you’ll be killed?get you killed? gun owners are 2.7 times more likely to be murdered than non-owners. (Arthur Kellermann et. al., "Gun Ownership as a Risk Factor for Homicide in the Home," The New England Journal of Medicine, October 7, 1993, pp ) Does this prove it is safer not to have guns in the house? It may be true that guns cause murders, but it also might be true that having a greater risk of being murdered causes people to own guns.

bias in gathering and reporting statistics Cathie Adams, president of the Texas Eagle Forum, stated on CNN on April 21, 2005 that: “research shows that children in same-sex couple homes are 11 times more likely to be abused sexually…It is a proven fact and that was a research study done in the state of Illinois” But Illinois doesn’t keep track of whether households are same sex or not. It's possible that much of what the study calls “homosexual abuse” occurred in heterosexual households. The study also confuses pedophilia with homosexuality. If the sexual abuse was male-male or female-female, the study presumed the abuser was gay or lesbian.

not defining the units to be counted clearly example: statistics on “juvenile violence” the legal ages of juveniles vary from state to state, as do the ages at which juveniles may be tried as adults. example: comparing “unemployment” rates in two countries The definitions of unemployed may not be the same. In the U.S., people who are under- employed, or having stopped looking for work aren’t counted. example: statistics on “child abductions” can refer to kidnappings by strangers or parental custody disputes

Not defining the units clearly Report: 1 in 50 U.S. children face homelessness “OVER 1.5 MILLION OF OUR NATION’S CHILDREN GO TO SLEEP WITHOUT A HOME EACH YEAR.” The report, by the National Center on Family Homelessness, analyzed data from and found that more than 1.5 million children were without a home. By “each year,” they mean “at least one night a year.” But what does “without a home” mean? “Sharing the housing of other persons due to loss of housing, economic hardship, or a similar reason (sometimes referred to as doubled-up); Living in motels, hotels, trailer parks, or camping grounds due to lack of alternative accommodations” if you did any of the above for one night out of a year, you are counted by the National Center on Family Homelessness as “homeless.”

carefully considering statistical measures  Failure to report the margin of error (a.k.a “sampling error”)  example: +/- 5% in a national poll of 1,200 respondents  Making conclusions within the margin of error  If candidate A is preferred by 43% of voters, and candidate B is preferred by 46% of voters, all one can say is that they are tied in a statistical dead heat.  Not stating whether the results are statistically significant or what the threshold for statistical significance is.  example: p <.05

misleading statistics Some studies have reported that 90% of students are bullied at school. But the results depend on what the researcher defines as “bullying.” Does bullying included physical and verbal abuse? If bullying means “people being mean to you,” then it is surprising that 10% of students haven’t been bullied!

Misleading statistics Comparing the total number of complaints by airline passengers, without considering the total numbers of passengers per airline. AirlineTotal Complaints Jan- Oct 2008 American1,105 Continental456 Delta1,120 Northwest362 Southwest224 United1,045 US Airways957

misleading graphs and charts This CDC graph, comparing causes of death is misleading, because it leaves out deaths from heart disease, cancer, and stroke. The omission tends to make smoking look much more harmful by comparison.

media slanting: What to watch and listen for  Remember: news media are after ratings, not necessarily the truth.  Watch for bias during the opening or lead-in.  Ask how representative or generalizable case studies are.  Don’t let pictures overwhelm the facts.  Beware of quoted sources with an agenda.  Beware of “black hats” or “white hats.”  When statistics are presented is the underlying methodology explained?  Watch out for rhetorical questions with no follow-up proof.