AP Statistics Section 5.2 B More on Experiments

As a researcher conducts an experiment, he/she is hoping to see a difference in the responses that is so large that the difference is unlikely to be due to mere chance variation. An observed effect so large that it would rarely occur by chance variation is called __________________ statistically significant

When all experimental units are allocated at random among all treatments, the experiments is said to have a completely randomized design.

Completely randomized designs are the simplest statistical designs for experiments. However, completely randomized designs are inferior to more elaborate statistical designs.

A block is a group of experimental units that are known before the experiment to be similar in some way that is expected to systematically affect the response to the treatments.

In a block design, the random assignment of units to treatments is carried out separately within each block.

* Blocks control the effects of some outside variables by bringing those variables into the experiment to form the blocks. * Unless the link between the blocking variable and the response variable is obvious, you must justify the reason for blocking on any given variable.

Why no control group in this experiment? Example: The progress of a type of cancer differs in women and men. Design an experiment to compare three therapies for this cancer. randomly assign Why no control group in this experiment? Unethical to not treat patients with cancer

Blocks allow us to draw separate conclusions about each block, for example, men and women in the previous experiment. Blocking also allows more precise overall conclusions because the systematic differences between men and women can be removed when we study the overall effects of the three therapies.

Example: The soil type and fertility of farmland differ by location (hilly, arid and shaded). Because of this, a test of the effect of 2 tillage types and 3 pesticide application schedules on soybean yields uses small fields as blocks. Design an experiment to test the effects of the six treatments. Randomly assign T1P1, T1P2, T1P3, T2P1, etc. to the plots within each block. Again, no control group because tillage and some type of pesticide are both necessary.

unavoidable randomization A wise experimenter will form blocks based on the most important ____________ sources of variability among the experimental units. ______________will then average out the remaining variation and allow an unbiased comparison of the treatments. unavoidable randomization

Example: Design an experiment to compare old and new waterproofing treatments on the boots of 20 volunteers. Match subjects with similar lifestyles. Randomly assign which subjects gets boots with the new or old treatment. Compare the condition of the boots after a certain length of time. Each subject will get a pair of boots where one boot has the new treatment and the other boot the old treatment. Randomly assign which boot gets which treatment.

compares just two treatments. The simplest use of blocking is a matched pairs design, which _____________________________ Subjects are matched in pairs, essentially forming a block of two units that are _____________________________ compares just two treatments. similar in some important way.

The idea is that matched subjects are more similar than unmatched subjects, so that comparing responses within a number of pairs is more efficient than comparing the responses of groups of randomly assigned subjects. Randomization remains important, however.

Example: An experiment to compare two advertisements for the same product might use pairs of subjects with the same _____, _____ and _______. sex age income

One common variation of the matched pairs design imposes both treatments on the same subjects, so that each subject serves as his or her own control.

Example: Consider our earlier experiment (Section 5 Example: Consider our earlier experiment (Section 5.2 A) concerning the effect of listening to classical music while reading an unfamiliar piece of literature on retention. Previously, we simply randomly divided 40 participants into two groups and had one listen to classical music while reading and the other did not listen to classical music. What lurking variables could affect the response variable? reading ability, IQ, etc.

Design a matched pairs experiment that will better determine the effects of listening to classical music while reading on retention. How does randomization play a part? Each subject will read an unfamiliar piece of literature while not listening to music and a second unfamiliar piece of literature while listening to classical music. Randomly assign which subjects do the music first or second. Compare the results.

Athlete’s weight and conditioning Example: We often see football players on the sidelines inhaling oxygen. Does oxygen help speed recovery from intense exercise? Design a completely randomized experiment. What confounding variables are not accounted for? Athlete’s weight and conditioning

Design a blocked approach for conducting this experiment. Divide the subjects into two blocks by position – linemen and skill position. Within each block, randomly assign subjects to the two treatment groups. Compare the recovery times within each block and overall.

Design a matched pairs approach for conducting this experiment. Each subject will do intense exercise and have oxygen afterward as well as do intense exercise with no oxygen afterwards. Randomly assign which subjects do which first and give sufficient time between exercise periods. Randomly assign which subjects do which first and give sufficient time between exercise periods. Compare the recovery times.

Cautions about Experimentation

Earlier in this section we discussed the role of a placebo in experimental design. What would be one important factor when using a placebo in an experiment? The group receiving the placebo cannot know it is getting the placebo instead of the drug. BLIND experiment

In fact, a researcher may accidentally introduce bias into an experiment if he is aware of which group received the placebo. To avoid this possibility we could use a double-blind experiment: neither the subjects nor the researcher knows which treatment a subject receives.

Many, perhaps most, experiments have some weaknesses in detail Many, perhaps most, experiments have some weaknesses in detail. The environment of an experiment can influence the outcomes in unexpected ways. The most serious potential weakness of experiments is the lack of realism – the subjects treatments or settings do not realistically represent the conditions we wish to study.

CAUTION: Most experimenters want to generalize their conclusions to some setting wider than that of the original experiment. Statistical analysis of an experiment cannot tell us how far the results will generalize to other settings.