Chapter 41 Sample Surveys in the Real World

Chapter 42 Thought Question 1 (from Seeing Through Statistics, 2nd Edition, by Jessica M. Utts, p. 14) Nicotine patches are patches attached to the arm that dispense nicotine into the blood of someone who is trying to quit smoking. Suppose you were to read about a study showing that nicotine patches were twice as effective in getting people to quit smoking as “control” patches (made to look like the real thing). Further, suppose you are a smoker trying to quit. What questions would you want answered about the study before you decided whether or not to try the patches yourself?

Chapter 43 Thought Question 2 When surveying students on their opinions of their school’s athletics programs, do you think it matters who conducts the interviews? Explain your answer with an example.

Chapter 44 Questions to ask about a study (determining the soundness of statistical studies) 1. Who funded the study, and who conducted the study (and are they related)? 2. Who were the individuals (or objects) studied, and how were they selected [population, sample size, response rate] ? 3. What was the setting in which the measurements were taken [time, location, method of contact, etc.] ? 4. What was the exact nature of the measurements made or questions asked? 5. Were there any other differences in the groups being compared [any confounding/lurking variables] ? 6. What was the magnitude of any claimed effects or differences?

Chapter 45 Case Study Questions to ask Nicotine Patches for Quitting Smoking from Thought Question 1

Chapter 46 Questions to ask Nicotine Patches for Quitting Smoking 1. Who funded the study, and who conducted the study (and are they related)? Funding Source: Company producing nicotine patch? Independent research organization? Researchers: People from the funding agency, possibly with direct interest in the results? Hired persons who do not know the goals of the research?

Chapter 47 Questions to ask 2. Who were the individuals or objects studied, and how were they selected? What is the population of interest? Were all subjects trying to quit smoking? Age, gender, previous smoking behavior, etc.? Volunteers? Recruited by researchers? 3. What was the setting in which the measurements were taken? Probably was in natural environment. How were subjects contacted and measured? Any measurements done in a lab? Nicotine Patches for Quitting Smoking

Chapter 48 Questions to ask 4. What was the exact nature of the measurements made or questions asked? How was “quitting” measured (for how long)? Group assignments were probably recorded. 5. Were there any other differences in the groups being compared? Could nicotine and “control” groups be different in regards to age, gender, smoking behavior, etc.? Were subjects randomly assigned to groups? Nicotine Patches for Quitting Smoking

Chapter 49 Questions to ask 6. What was the magnitude of any claimed effects or differences? Nicotine patches were “twice as effective” in getting people to quit as control patches. What are the actual percentages that quit?.01% &.02%? 30% & 60%? others? Nicotine Patches for Quitting Smoking

Chapter 410 Questions to ask about a study “University Students are Healthier than you Think” Spring 2002 random undergraduate classroom survey of n=810 students was conducted by the Office of Health Promotion within the University Student Health Services, Division of Student Affairs. Statistics from this survey led to the following conclusions: - most students (67%) have 0-4 drinks when they go out - most (69%) have had 0-1 sex partners in the past year - most (76%) either don’t drink, or use designated drivers if they do What questions should you ask to help you assess the credibility of these results?

Chapter 411 Errors in Sample Surveys u Sampling errors u Random sampling errors –measured by margin of error u Nonsampling errors

Chapter 412 Sampling Errors u Difficulties –Using the wrong sampling frame (next slide) u Disasters –Using voluntary response (volunteer sample) –Using a convenience or haphazard sample v cannot extend results to the population of interest (need a broad cross-section of the population)

Chapter 413 Using the Wrong Sampling Frame Including some units not in the population. Undercoverage: Excluding some units in the population.

Chapter 414 u Difficulties –Processing errors (data entry, calculations) –Wording of questions / Response error u Disasters –Nonresponse (cannot contact subjects or they do not respond) Nonsampling Errors

Chapter 415 Concerns when Asking Survey Questions u Deliberate bias u Unintentional bias u Desire to please u Asking the uninformed u Unnecessary complexity u Ordering of questions u Confidentiality and anonymity

Chapter 416 Deliberate Bias u “If you found a wallet with $20 in it, would you return the money?” u “If you found a wallet with $20 in it, would you do the right thing and return the money?”

Chapter 417 Unintentional Bias u “I have taught several students over the past few years.” –How many students do you think I have taught? –How many years am I referring to? u “Over the past few days, how many servings of fruit have you eaten?” –How many days are you considering? –What constitutes a serving?

Chapter 418 Desire to Please u “Is your instructor doing a good job presenting the course material in a clear and interesting way?” v Yes v No

Chapter 419 u A 1978 poll done in Cincinnati asked people whether they “favored or opposed repealing the 1975 Public Affairs Act.” –There was no such act! –About one-third of those asked expressed an opinion about it. Washington Post National Weekly Edition (April 10-16, 1995, p. 36) Asking the Uninformed: Case Study

Chapter 420 Unnecessary Complexity u “Do you sometimes find that you have arguments with your family members and co-workers?” –Arguments with family members –Arguments with co-workers –“sometimes find” (vague or unclear)

Chapter 421 Ordering of Questions u “How often do you normally go out on a date? About ___ times a month.” u “How happy are you with life in general?” –Strong association between these questions. –If the ordering is reversed, then there would be no strong association between these questions.

Chapter 422 Confidentiality and Anonymity u Confidential answer –respondent is known, but the information is a secret –facilitates follow-up studies u Anonymous answer –the respondent is not known, or cannot be linked to his/her response –usually yields more truthful answers

Chapter 423 Probability Sampling Plans u Simple random sampling (SRS) u Stratified random sampling u Cluster sampling u Systematic sampling u Random digit dialing u Multistage sampling

Chapter 424 Stratified Random Sample u first divide the population into groups of similar individuals, called strata u second, choose a separate simple random sample in each stratum u third, combine these simple random samples to form the full sample –if only certain strata are (randomly) chosen to be used, and all subjects in these strata make up the sample, then we have a cluster sample

Chapter 425 Systematic Sample u randomly select a member of the sampling frame for the sample u using a set procedure or rule, select the rest of the individuals for the sample –for example, randomly select an individual from the sampling frame, and then select every 25 th member of the sampling frame to be in the sample

Chapter 426 Multistage Sample u divide the population of interest into groups u randomly select some of those groups u divide the resulting collection of individuals into smaller groups u randomly select some of those groups u continue dividing the resulting collection of individuals into groups and randomly selecting some of those groups until you can simply list all of the resulting individuals and randomly select n of them for your sample

Chapter 427 Multistage Sample u Example: Selecting 1500 registered U.S. voters [Use multistage sampling since we cannot obtain a sampling frame (list) of all registered U.S. voters.] –randomly select five U.S. states –obtain a list of all counties/cities in those states –randomly select 20 of those counties/cities –obtain a list of all registered voters in those 20 counties/cities –randomly select 1500 voters from that list

Chapter 428 Key Concepts u Ask questions when you read about studies. u Sampling & Nonsampling Errors –Concerns when asking survey questions u Stratified Sampling, Cluster Sampling, Systematic Sampling, Multistage Sampling