1. Basic Experimental Design
Adapted from the original by
Larry V. Hedges

2. What is Experimental Design?
Experimental design includes both
Strategies for organizing data collection
Data analysis procedures matched to those data collection strategies

3. Why Do We Need Experimental Design?
Because of variability
We wouldn’t need a science of experimental design if
If all units (people or other things we are experimenting with) were identical
and
If all units responded identically to treatments
We need experimental design to control variability so that treatment effects can be identified

4. A Little History
The idea of controlling variability through design has a long history
In 1747 Sir James Lind’s studies of scurvy
Their cases were as similar as I could have them. They all in general had putrid gums, spots and lassitude, with weakness of their knees. They lay together on one place … and had one diet common to all (Lind, 1753, p. 149)
Lind then assigned six different treatments to groups of patients

5. A Little History
The first modern randomized clinical trial in medicine is usually considered to be the trial of streptomycin for treating tuberculosis
It was conducted by the British Medical Research Council in 1946 and reported in 1948

6. A Little History Studies in crop variation I – VI (1921 – 1929)
In 1919 a statistician named Fisher was hired at Rothamsted agricultural station
They had a lot of observational data on crop yields and hoped a statistician could analyze it to find effects of various treatments
All he had to do was sort out the effects of confounding variables

7. Studies in Crop Variation I (1921)
Fisher studied ways to minimise the effects of confounding variables (variables other than the one we are experimenting with, which influence the variable we are measuring).
soil fertility gradients
drainage
effects of rainfall
effects of temperature and weather, etc.
Conclusion
The effects of confounders are typically larger than those of the systematic effects we want to study

8. Studies in Crop Variation II (1923)
Fisher invents
Basic principles of experimental design
Control of variation by randomization
Analysis of variance

9. Our Hero in 1929

10. Principles of Experimental Design
Experimental design controls background variability so that systematic effects of treatments can be observed
Three basic principles
Control by matching
Control by randomisation
Control by statistical adjustment

11. Control by Matching
Matching ensures that groups compared are alike on specific known and observable characteristics (in principle, everything we have thought of)
Wouldn’t it be great if there were a method of making groups alike on not only everything we have thought of, but everything we didn’t think of too?
There is such a method

12. Control by Randomisation
Matching controls for the effects of variation due to specific observable characteristics
Randomisation controls for the effects all (observable or non-observable, known or unknown) characteristics
Randomisation makes groups equivalent (on average) on all variables (known and unknown, observable or not)
Randomisation also gives us a way to assess whether differences after treatment are larger than would be expected due to chance.

13. Control by Randomisation
Random assignment is not assignment with no particular rule. It is a purposeful process
Assignment is made at random. This does not mean that the experimenter writes down the names of the varieties in any order that occurs to him, but that he carries out a physical experimental process of randomisation, using means which shall ensure that each variety will have an equal chance of being tested on any particular plot of ground (Fisher, 1935, p. 51)

14. Using Principles of Experimental Design
You have to know a lot to use matching and statistical control effectively
You do not have to know a lot to use randomisation effectively.
In this course we will use randomisation.

15. Randomisation Procedures
Completely Randomised Design
(2 treatments, 2n individuals)
Make a list of all individuals
For each individual, pick a random number from 1 to 2 (odd or even)
Assign the individual to treatment 1 if even, 2 if odd
When one treatment is assigned n individuals, stop assigning more individuals to that treatment

16. Inference Models Two different kinds of inferences about effects
Unconditional Inference (units are a random sample from the population)
Inference to the whole population
(requires a representative sample)
Conditional Inference (units fixed)
Inference to the part of the population in the experiment
(does not need to be a random sample)

17. Statistical Analysis Procedures
In this course, we do not worry about getting a representative sample.
We make an inference for the people or other units in the experiment.
In the conclusion, we can discuss whether we think the results might be applicable to the whole population (define the population carefully).