Presentation on theme: "Basic Experimental Design"— Presentation transcript:
1 Basic Experimental Design Adapted from the original byLarry V. Hedges
2 What is Experimental Design? Experimental design includes bothStrategies for organizing data collectionData analysis procedures matched to those data collection strategies
3 Why Do We Need Experimental Design? Because of variabilityWe wouldn’t need a science of experimental design ifIf all units (people or other things we are experimenting with) were identicalandIf all units responded identically to treatmentsWe need experimental design to control variability so that treatment effects can be identified
4 A Little HistoryThe idea of controlling variability through design has a long historyIn 1747 Sir James Lind’s studies of scurvyTheir cases were as similar as I could have them. They all in general had putrid gums, spots and lassitude, with weakness of their knees. They lay together on one place … and had one diet common to all (Lind, 1753, p. 149)Lind then assigned six different treatments to groups of patients
5 A Little HistoryThe first modern randomized clinical trial in medicine is usually considered to be the trial of streptomycin for treating tuberculosisIt was conducted by the British Medical Research Council in 1946 and reported in 1948
6 A Little History Studies in crop variation I – VI (1921 – 1929) In 1919 a statistician named Fisher was hired at Rothamsted agricultural stationThey had a lot of observational data on crop yields and hoped a statistician could analyze it to find effects of various treatmentsAll he had to do was sort out the effects of confounding variables
7 Studies in Crop Variation I (1921) Fisher studied ways to minimise the effects of confounding variables (variables other than the one we are experimenting with, which influence the variable we are measuring).soil fertility gradientsdrainageeffects of rainfalleffects of temperature and weather, etc.ConclusionThe effects of confounders are typically larger than those of the systematic effects we want to study
8 Studies in Crop Variation II (1923) Fisher inventsBasic principles of experimental designControl of variation by randomizationAnalysis of variance
10 Principles of Experimental Design Experimental design controls background variability so that systematic effects of treatments can be observedThree basic principlesControl by matchingControl by randomisationControl by statistical adjustment
11 Control by MatchingMatching ensures that groups compared are alike on specific known and observable characteristics (in principle, everything we have thought of)Wouldn’t it be great if there were a method of making groups alike on not only everything we have thought of, but everything we didn’t think of too?There is such a method
12 Control by Randomisation Matching controls for the effects of variation due to specific observable characteristicsRandomisation controls for the effects all (observable or non-observable, known or unknown) characteristicsRandomisation makes groups equivalent (on average) on all variables (known and unknown, observable or not)Randomisation also gives us a way to assess whether differences after treatment are larger than would be expected due to chance.
13 Control by Randomisation Random assignment is not assignment with no particular rule. It is a purposeful processAssignment is made at random. This does not mean that the experimenter writes down the names of the varieties in any order that occurs to him, but that he carries out a physical experimental process of randomisation, using means which shall ensure that each variety will have an equal chance of being tested on any particular plot of ground (Fisher, 1935, p. 51)
14 Using Principles of Experimental Design You have to know a lot to use matching and statistical control effectivelyYou do not have to know a lot to use randomisation effectively.In this course we will use randomisation.
15 Randomisation Procedures Completely Randomised Design(2 treatments, 2n individuals)Make a list of all individualsFor each individual, pick a random number from 1 to 2 (odd or even)Assign the individual to treatment 1 if even, 2 if oddWhen one treatment is assigned n individuals, stop assigning more individuals to that treatment
16 Inference Models Two different kinds of inferences about effects Unconditional Inference (units are a random sample from the population)Inference to the whole population(requires a representative sample)Conditional Inference (units fixed)Inference to the part of the population in the experiment(does not need to be a random sample)
17 Statistical Analysis Procedures In this course, we do not worry about getting a representative sample.We make an inference for the people or other units in the experiment.In the conclusion, we can discuss whether we think the results might be applicable to the whole population (define the population carefully).