Chapter 5.2 Designing Experiments

Experiments Experimental Units – individuals on whom the experiment is performed When an experiment is performed on humans, they are known as subjects. Factors – explanatory variable(s) Need at least one factor for every experiment Levels – specific values that the experimenter chooses for a factor with at least two levels for each factor. Treatments – different levels of a single factor or combinations of levels of two or more factors.

Does taking aspirin regularly help protect people against heart attacks? The subjects for this experiment are 21,996 male physicians. 2 1 3 4 Note the two factors here. With two factors, we end up with 4 possible treatments.

Example – OptiGro Fertilizer An ad for OptiGro plant fertilizer claims that with this product we will grow “juicier, tastier” tomatoes. We'd like to test this claim and see if we can get by with a half-dose. What are the experimental units in this example? Tomato plants (the thing we apply the treatment to) What is the response we are interested in? “Juiciness and Tastiness” of the tomatoes

Example – OptiGro Fertilizer What is the factor and the levels of the factor? What are the treatments? They are the same as the levels of the factors since there is only one factor.

Example – OptiGro Fertilizer The makers of OptiGro want to ensure that their product will work under a wide variety of watering conditions. They have asked that we include a second factor. The CEO has requested that we look at daily watering vs. watering every other day.

Example – OptiGro Fertilizer Notice that we now have 6 treatments based on two factors. (3 times 2 = 6)

Purpose of Experiments Samples and Census can give good evidence of association or correlation but can not provide evidence of causation. In principle, experiments can give good evidence of causation if there is sound experimental design.

Terminology Control Group – experimental units assigned to a baseline treatment level Provides basis for comparison Baseline: either a “gold standard” or a placebo Placebo – a null treatment known to have no effect Placebo Effect – tendency of many human subjects to show response even when administered a placebo

Terminology If there are other factors influencing the subjects, they are called confounding (lurking) variables. Recall the example of the income versus height (i.e. taller people made more money) Gender was a confounding variable We want the only influence on our units to be the treatment.

Diagrams One way to portray an experiment is through a diagram. UNITS  TREATMENT  RESPONSE Random Assignment Group 1 Group 2 Treatment 1 Treatment 2 Compare response

Principles of Experimental Design Randomization Experimental units should be randomly assigned to treatment groups Order of the trials should also be randomized Some uncontrolled sources of variation will be controlled through randomization Experiments without randomization may have biased results

Comparative experiment – Breakfast Study The randomized design should distribute the variability within the rats evenly for each group. The comparative design ensures that influences outside of the diet operate equally on each group.

Principles of Experimental Design Control Make all conditions as similar as possible for all treatment groups The only difference between groups should be the treatments Controlling “outside” influences reduces variation in the responses, making it easier to detect differences among the groups due to the treatments

Principles of Experimental Design Replication In an experiment, each treatment is given to several different experimental units. The larger the number of experimental units, the more accurate the results will be if the experiment is designed appropriately. Entire experiment should be repeated on a different group of experimental units

Statistical Significance If the differences between the treatment groups are big enough, we will attribute the differences to the treatments How big is “big enough”? Answer more precisely later Looking for differences large enough to be due to something other than random variation

Terminology Blinding - the researcher disguises treatments Single Blind - either the subject or the evaluator is unaware of the treatment Double Blind - both the subject AND the evaluator are in the dark.

Lack of realism population Random sampling is meant to gain information about the larger population from which we sample. sample Is the treatment appropriate for the response you want to study? Is studying the effects of eating red meat on cholesterol values in a group of middle-aged men a realistic way to study factors affecting heart disease problem in humans? What about studying the effects of hair spray on rats to determine what will happen to women with big hair?

Lack of Realism Subjects or treatments or setting of an experiment may not realistically duplicate the conditions that we want to study. This can limit the conclusions that we can draw from the experimental results.

Lack of Realism Example Recall the smoking beagles discussion from earlier in the year. Even though smoking causes lung cancer, the experiment provides evidence of a “safe” cigarette. This is a lack of realism because the biology of a beagle has many unique properties that are not found in humans.

Blocking This can be considered another principle of experimental design, but not required in an experimental design Group similar individuals together and then randomize within each of these blocks Reduces the effects of identifiable attributes of the subjects that cannot be controlled

Example – Shingles A research doctor believes a new ointment will be more effective than the current medication in treating shingles (a painful skin rash). Eight patients have volunteered for our study. We will design an experiment to help the doctor verify her claim.

Example – Shingles Experimental Unit = Person (we have 8) Response = Severity of shingles Factor = ointment Levels = current ointment and new ointment Why do we not want a placebo here? Treatments = current ointment and new ointment Treatments are the same as levels here because we only have one factor

Example - Shingles Control Replication Randomization Ointment taken every day for both groups All other care is similar Replication 4 patients in each treatment group Randomization Randomly assign 4 patients to each treatment group How do we do this?

Example - Shingles Patients: Alex, Bethany, Carl, Denise, Evelyn, Fred, George, Hannah Number the patients 1 through 8 Use random numbers to assign treatments 41098 18329 78458 31685 55259 First 4 patients we select get new treatment, the rest get the current treatment

Example - Shingles By one’s: 4 1 0 9 8 1 8 3 2 9 7 8 4 5 8 3 1 6 8 5 5 5 2 5 9 Throw out 0 and 9 (they don’t correspond to patients) Throw out repeats New ointment: 4, 1, 8, 3 Denise, Hannah, Alex, Carl Current ointment: 2, 5, 6, 7 (everyone else) Bethany, Evelyn, Fred, George

Compare severity of shingles Example - Shingles Group 1 4 patients Treatment 1 New Ointment Randomly Assign the 8 patients to groups Compare severity of shingles Group 2 4 patients Treatment 2 Current Ointment

Example - Shingles Could we make this experiment double-blind? Double-blind means that neither the patient nor the person evaluating the results knows who is receiving each treatment. To make this double-blind we would have to assume that the two ointments look alike: same color, unmarked tubes, similar texture, similar odors, etc. If one ointment had a distinctive odor, it would probably not be possible to make the experiment double-blind.

Example – Blocking Cancer treatment (3 treatments) We suspect that men and women respond differently so divide subjects into men and women Randomly assign each block to three groups The three groups in each block receive only one treatment Compare survival rates

Examples – Blocking Group 1 Treatment 1 Compare survival rates Random Assign. Men Group 2 Treatment 2 Group 3 Treatment 3 Subjects Group 1 Treatment 1 Compare survival rates Random Assign. Women Group 2 Treatment 2 Group 3 Treatment 3