Brian Kelly '06 Chapter 13: Experiments

Observational Study n Observational Study: A type of study in which individuals are observed or certain outcomes are measured. n Retrospective Study: A type of observational study in which subjects are selected and then their previous behaviors are determined. n Prospective Study: A type of observational study in which subjects are followed to observe future outcomes. n Experiment: An experiment manipulates factor levels to create treatments, randomly assigns subjects to these treatment levels, and then compares the responses of the subject groups across the treatment levels. n Random Assignment: Assignment of experimental units to treatment groups at random.

Randomized, Comparative Experiments n An experimenter must identify one explanatory variable, called a factor, to manipulate and at least one response variable to measure. n Subjects or participants: Humans that are experimented on. n Experimental units: Individuals experimented on such as rats or plants. n Levels: The specific values that are chosen for a factor. n Treatment: Combination of specific levels from all the factors that an experimental unit receives.

Four Principles of Experimental Design n Control: We can control sources of variation by making conditions as similar as possible for all treatment groups. n Randomization: Allows us to equalize the effects of unknown or uncontrollable sources of variation. n Replication: The need to be able to repeat the experiment. n Blocking: Grouping experimental units that are similar and randomizing from there.

Steps in Designing an Experiment n State what you would like to know. n Specify your response variable, the factor levels and the treatments, and the experimental units. n Observe the principles of your design. –Control any sources of variability. –Randomly assign experimental units to treatments. –Replicate your results for a better figure.

Diagrams n A diagram emphasizes the random allocation of subjects to treatment groups, the separate treatments applied to these groups, and the ultimate comparison.

Control Treatments n Control Group: The experimental units assigned to a baseline treatment level, typically either the default treatment, which is well understood, or a null, placebo treatment.

Blinding n Blinding: Any individual associated with an experiment who is not aware of how subjects have been allocated to treatment groups is said to be blind. n Two main classes who can affect the outcome of an experiment: –Subjects, treatment administrators, or technicians who can influence the results. –Judges, treating physicians, and ect. who evaluate the results. n Single blind: Occurs when every individual in either of the above classes are blinded. n Double blind: Occurs when every individual in both classes above are blinded.

Placebos n Placebo: A fake treatment that looks just like the treatments being tested. n This is the best way to blind subjects from whether or not they are receiving the treatment. n Placebo effect: The tendency of human subjects to show a response even when administered a placebo. Usually more than 20% of the subjects exhibit this quality.

Blocking! n When groups of experimental units are similar, it’s usually a good idea to allocate them together into blocks. n By clocking we isolate the variability due to the difference between the blocks so that we can see the differences due to the treatment more clearly. n Matching subjects that are paired because they are similar in ways not under the study can reduce variation. n Randomized block design: Given this title because the randomization only occurs within blocks during these types of studies.

Confounding n When levels of one factor are associated with levels of another factor, we say that the two factors are confounding.

What Can Go Wrong?? n Beware of confounding: use randomization whenever possible to ensure your factors are not confounded. n Bad things can happen to even good experiments: Collect as much information and even extra information to ensure nothing goes wrong. n Don’t spend your entire budget on the first run. Try to pilot a small experiment before running your large scale experiment.