1. Chapter 5.2 Designing Experiments

2. Experiments
Experimental Units – individuals on whom the experiment is performed
When an experiment is performed on humans, they are known as subjects.
Factors – explanatory variable(s)
Need at least one factor for every experiment
Levels – specific values that the experimenter chooses for a factor with at least two levels for each factor.
Treatments – different levels of a single factor or combinations of levels of two or more factors.

3. Does taking aspirin regularly help protect people against heart attacks?
The subjects for this experiment are 21,996 male physicians. 2 1 3 4
Note the two factors here.
With two factors, we end up with 4 possible treatments.

4. Example – OptiGro Fertilizer
An ad for OptiGro plant fertilizer claims that with this product we will grow “juicier, tastier” tomatoes. We'd like to test this claim and see if we can get by with a half-dose.
What are the experimental units in this example?
Tomato plants (the thing we apply the treatment to)
What is the response we are interested in?
“Juiciness and Tastiness” of the tomatoes

5. Example – OptiGro Fertilizer
What is the factor and the levels of the factor?
What are the treatments?
They are the same as the levels of the factors since there is only one factor.

6. Example – OptiGro Fertilizer
The makers of OptiGro want to ensure that their product will work under a wide variety of watering conditions.
They have asked that we include a second factor. The CEO has requested that we look at daily watering vs. watering every other day.

7. Example – OptiGro Fertilizer
Notice that we now have 6 treatments based on two factors.
(3 times 2 = 6)

8. Purpose of Experiments
Samples and Census can give good evidence of association or correlation but can not provide evidence of causation.
In principle, experiments can give good evidence of causation if there is sound experimental design.

9. Terminology
Control Group – experimental units assigned to a baseline treatment level
Provides basis for comparison
Baseline: either a “gold standard” or a placebo
Placebo – a null treatment known to have no effect
Placebo Effect – tendency of many human subjects to show response even when administered a placebo

10. Terminology
If there are other factors influencing the subjects, they are called confounding (lurking) variables.
Recall the example of the income versus height (i.e. taller people made more money)
Gender was a confounding variable
We want the only influence on our units to be the treatment.

11. Diagrams
One way to portray an experiment is through a diagram. UNITS  TREATMENT  RESPONSE
Random Assignment
Group 1
Group 2
Treatment 1
Treatment 2
Compare response

12. Principles of Experimental Design
Randomization
Experimental units should be randomly assigned to treatment groups
Order of the trials should also be randomized
Some uncontrolled sources of variation will be controlled through randomization
Experiments without randomization may have biased results

13. Comparative experiment – Breakfast Study
The randomized design should distribute the variability within the rats evenly for each group.
The comparative design ensures that influences outside of the diet operate equally on each group.

14. Principles of Experimental Design
Control
Make all conditions as similar as possible for all treatment groups
The only difference between groups should be the treatments
Controlling “outside” influences reduces variation in the responses, making it easier to detect differences among the groups due to the treatments

15. Principles of Experimental Design
Replication
In an experiment, each treatment is given to several different experimental units.
The larger the number of experimental units, the more accurate the results will be if the experiment is designed appropriately.
Entire experiment should be repeated on a different group of experimental units

16. Statistical Significance
If the differences between the treatment groups are big enough, we will attribute the differences to the treatments
How big is “big enough”?
Answer more precisely later
Looking for differences large enough to be due to something other than random variation

17. Terminology Blinding - the researcher disguises treatments
Single Blind - either the subject or the evaluator is unaware of the treatment
Double Blind - both the subject AND the evaluator are in the dark.

18. Lack of realism
population
Random sampling is meant to gain information about the larger population from which we sample.
sample
Is the treatment appropriate for the response you want to study?
Is studying the effects of eating red meat on cholesterol values in a group of middle-aged men a realistic way to study factors affecting heart disease problem in humans?
What about studying the effects of hair spray on rats to determine what will happen to women with big hair?

19. Lack of Realism
Subjects or treatments or setting of an experiment may not realistically duplicate the conditions that we want to study.
This can limit the conclusions that we can draw from the experimental results.

20. Lack of Realism Example
Recall the smoking beagles discussion from earlier in the year. Even though smoking causes lung cancer, the experiment provides evidence of a “safe” cigarette.
This is a lack of realism because the biology of a beagle has many unique properties that are not found in humans.

21. Blocking
This can be considered another principle of experimental design, but not required in an experimental design
Group similar individuals together and then randomize within each of these blocks
Reduces the effects of identifiable attributes of the subjects that cannot be controlled

22. Example – Shingles
A research doctor believes a new ointment will be more effective than the current medication in treating shingles (a painful skin rash). Eight patients have volunteered for our study. We will design an experiment to help the doctor verify her claim.

23. Example – Shingles Experimental Unit = Person (we have 8)
Response = Severity of shingles
Factor = ointment
Levels = current ointment and new ointment
Why do we not want a placebo here?
Treatments = current ointment and new ointment
Treatments are the same as levels here because we only have one factor

24. Example - Shingles Control Replication Randomization
Ointment taken every day for both groups
All other care is similar
Replication
4 patients in each treatment group
Randomization
Randomly assign 4 patients to each treatment group
How do we do this?

25. Example - Shingles
Patients: Alex, Bethany, Carl, Denise, Evelyn, Fred, George, Hannah
Number the patients 1 through 8
Use random numbers to assign treatments
First 4 patients we select get new treatment, the rest get the current treatment

26. Example - Shingles
By one’s:
Throw out 0 and 9 (they don’t correspond to patients)
Throw out repeats
New ointment: 4, 1, 8, 3
Denise, Hannah, Alex, Carl
Current ointment: 2, 5, 6, 7 (everyone else)
Bethany, Evelyn, Fred, George

27. Compare severity of shingles
Example - Shingles
Group 1
4 patients
Treatment 1
New Ointment
Randomly Assign the
8 patients to groups
Compare severity of shingles
Group 2
4 patients
Treatment 2
Current Ointment

28. Example - Shingles Could we make this experiment double-blind?
Double-blind means that neither the patient nor the person evaluating the results knows who is receiving each treatment. To make this double-blind we would have to assume that the two ointments look alike: same color, unmarked tubes, similar texture, similar odors, etc.
If one ointment had a distinctive odor, it would probably not be possible to make the experiment double-blind.

29. Example – Blocking Cancer treatment (3 treatments)
We suspect that men and women respond differently so divide subjects into men and women
Randomly assign each block to three groups
The three groups in each block receive only one treatment
Compare survival rates

30. Examples – Blocking Group 1 Treatment 1 Compare survival rates
Random Assign.
Men
Group 2
Treatment 2
Group 3
Treatment 3
Subjects
Group 1
Treatment 1
Compare survival rates
Random Assign.
Women
Group 2
Treatment 2
Group 3
Treatment 3