Chapter 3 Producing Data

Observational study: observes individuals and measures variables of interest but does not attempt to influence the responses (ex. surveys/polling) Experiment: deliberately imposes some treatment on individuals in order to observe their responses (coke vs pepsi, testing new drugs) In most statistical studies, the objective is to use a small group of units to make an inference about a larger group.

3.1 Designing Samples

voluntary response sample consists of people who choose themselves by responding to a general appeal Example: call in response; internet voting people with negative opinions tend to respond much more often than positive opinions

Convenience sampling chooses individuals easiest to reach Example: in mall Not a wide variety of people for sample

Simple random samples sample chosen so that every individual has an equal chance of being selected To choose a simple random sample: – assign a number to each individual in the population (all numbers used must have the same # of digits) – use table of random digits (table B) or calculator to select random numbers – Pg 173 # 7, 8, 9

stratified random sample divide the population into groups of similar individuals called strata – male & female; urban, suburban, rural; age groups; grades in school choose a separate simple random sample in each strata combine the SRS’s to form the full sample

multistage samples choosing the sample in stages example: divide into grade level, then male & female, then randomly pick

Cautions about samples (bias) Selection bias – sample does not represent the population of interest (undercoverage) Nonresponse bias – cannot be contacted or does not respond Response bias – participants provide incorrect information – respondents may lie – interviewers attitude may suggest one answer is more desirable than others – sex or race may influence answers – answers to questions that ask respondents to recall past events

Possible bias in surveys Wording of questions – Deliberate bias in questions by asking confusing or leading questions – Unnecessary complexity Desire of respondents to please Ordering of questions Confidentiality concerns

Margin of Error A measure of the accuracy of a sample proportion At least 95% of the population would give this answer between...

Margin of error example: For a CNN Gallup Poll conducted on Sept. 2, 2002, a random sample of 1003 adult Americans were asked, “How important would you say religion is in your own life; very important, fairly important, or not very important?” The percentage of the sample that selected very important was 65%. Calculate the margin of error and calculate the interval for which at least 95% of all adult Americans.

Margin of error and sample size: When the sample size is increased, the margin of error decreases When a large sample size is made even larger, the improvement in accuracy is relatively small Sample sizeMargin of error (10%) (5%) (4%) (3.2%) (2.5%) (2%) (1%)

3.2 Designing Experiments

Experiments When we do something to people, animals, or objects in order to observe the response Experimental units – individuals in which the experiment is done Subjects – human experimental units Factors – explanatory variables in an experiment Treatment – specific experimental condition applied to the units

Researchers studying the absorption of a drug into the bloodstream inject the drug into 25 people. Subjects – 25 people Factor – Dosage of the drug Treatment – 1 level / 1 injection of the drug Response variable – Concentration of the drug in a subject’s blood

What are the effects of repeated exposure to an advertising message? An experiment investigated this question using undergraduate students as subjects. All subjects viewed a 40 minute TV program that included ads for a 35 mm camera. Some subjects saw a 30 second commercial; others, a 90 second version. The same commercial was repeated either 1, 3, or 5 times during the program. After viewing, all subjects were asked questions about their recall of the ad, their attitude toward the camera, and their intention to purchase it.

Subjects – Undergraduate students Factors – Length of commercial – 2 levels – Repetitions – 3 levels Treatments – 6 combinations length of commercial with repetitions Response variables – Recall of ad, attitude, intent to purchase

Advantages of experiments over observational studies study the effects of the specific treatments can control the environment of the experimental units (controlled variables) can give good evidence for causation can study the combined effects of several factors simultaneously Pg 187 # 32, 33, 34

Principles of experimental design Control of the effects of lurking variables Randomization Replication of experiment on many units to reduce chance variation in the results

statistically significant an observed effect so large that it would rarely occur by chance

lack of realism the subjects (using rats instead of humans), treatments, or setting of an experiment (in lab instead of at home) may not duplicate the conditions we really want to study Examples pg 195 Pg 196 # 41, 42

Use Comparative experiments! Experiments should compare treatments rather than attempt to assess a single treatment Placebo effect – giving a person a “dummy” drug and they react to it anyway. Probably because they trust that the medicine will work. “The power of positive thinking” The placebo effect and other lurking variables operate on both groups Control group – the group of people receiving a “fake” treatment

Randomized comparative experiments an experiment that uses both comparison and randomization can have one or more than one factor to compare Examples pg 190 & 191 Pg 192 # 37

logic of randomized comparative experiments random assignment of subjects forms groups that should be similar in all respects before the treatments are applied comparative design ensures that influences other than the experimental treatments operate equally in all groups differences in average response must be due either to the treatments or to the play of chance in the random assignment of subjects to the treatments

Advantage of Randomized comparative experiments produces data that gives good evidence for a cause-and-effect relationship between the explanatory and response variables Pg 194 # 39, 40

double blind experiment when neither the patient nor the doctor knows what form of the pill they are getting

Matched pair design compares just two treatments choose subjects as completely matched as possible and each receive one treatment Coke vs Pepsi each subject serves as his or her own control, each subject receives both treatments in random order order of treatments can influence the subjects response so randomize (flip a coin, roll a die) the order

Block design Block separating into groups before you start (male/female) random assignment of units to treatments within the block allows us to draw separate conclusions about each block allows for more precise overall conclusions Pg 198 # 43, 45

Observational study issues to be addressed: What is the population of interest and what is the sampled population? – Careful of undercoverage or overcoverage How were the individuals or objects in the sample actually selected? What are the potential sources of bias – Increasing the sample size does not reduce bias

Experiment issues to be addressed: Should be clear about how random assignment was incorporated into design of experiment Any factors held constant throughout the experiment Was blocking used? If so, how were the blocks created?