1. Observational Studies and Experiments
Chapter 11
Observational Studies and Experiments

2. Observational Studies vs. Experiments
Do not attempt to control anything; simply collect data
Cannot be used to identify cause and effect
Retrospective Studies identify subjects and then collect data from the past.
Prospective Studies identify subjects and then collect data in the future.

3. Observational Studies vs. Experiments
Example – We ask people how much vitamin C and vitamin E they took, and how long their most recent cold lasted. This is a retrospective study.
Example – We ask people to record how much vitamin C and vitamin E they take, and how long their next cold lasts. This is a prospective study.

4. Observational Studies vs. Experiments
Randomized, Comparative Experiments
Manipulate one or more variables called factors
The specific values for a factor are called levels
The combination of levels for all factors is called a treatment
Measure one or more other variables called response variables
Can be used to identify cause and effect, but only if we can eliminate all other possible causes

5. Observational Studies vs. Experiments
Randomized, Comparative Experiments
Example – We manipulate the amount of vitamin C and vitamin E each patient receives, and measure how long the cold lasts. We use two levels of vitamin C ( 0 mg and 500 mg) and three levels of vitamin E (0 mg, 250 mg, and 500 mg). We combine these into 6 treatments:
0 mg vitamin C and 0 mg vitamin E
0 mg vitamin C and 250 mg vitamin E
0 mg vitamin C and 500 mg vitamin E
250 mg vitamin C and 0 mg vitamin E
250 mg vitamin C and 250 mg vitamin E
250 mg vitamin C and 500 mg vitamin E

6. Observational Studies vs. Experiments
When does data happen?
Manipulates variables?
Determines cause and effect?
Retrospective Study
Past No
Prospective Study
Future
Experiment
Yes

7. 3 Principles of Experimental Design
Control – We control as many other variables as we can by making conditions similar between all of the treatments.
Randomization – We randomly assign subjects to treatments to minimize variation in variables we can’t control.
Replicate
We use multiple subjects in each treatment group
We repeat the experiment with other groups

8. Diagrams of Experiments
Random Assignment
Group 1
Treatment 1
Subjects
Compare
Group 2
Treatment 2

9. Statistical Significance
We say a result is statistically significant if we don’t believe it is likely to have occurred by chance.
If we flip a coin 100 times, and get 54 heads does that mean the coin isn’t fair? Probably not. We expect some variation in results. 54 is probably random chance, so it is not statistically significant.
If we flip a coin 100 times, and get 94 heads does that mean the coin isn’t fair? Probably. 94 is way more than we would expect by chance. In this case, the results are statistically significant.

10. Experimental Design
Control Treatment – A treatment that provides a baseline measurement to compare the other treatments to.
Example – one treatment group gets none of the new medicine being tested.
Example – one treatment group gets the standard dose of the old medicine the new medicine is being compared against.

11. Experimental Design
Blinding – Ensuring that the people involved in the experiment do not know which treatment is applied to each individual. Two classes:
Those who could influence the results (subjects, those applying the treatments)
Evaluators (judges, evaluating physicians)
When one class is blind, the experiment is single-blind. When both classes are blind, the experiment is double-blind.

12. Experimental Design
Placebo – a “fake” treatment to disguise which treatment group a subject is in.
example – if one treatment group gets none of the drug, they might be given a sugar pill instead.
Often some of the group getting a placebo will actually get better. This is known as the placebo effect.

13. Experimental Design
Blocking – a strategy to control the effects of a variable by separating the subjects into groups.
Group the subjects into blocks (for example, by age)
Apply each treatment to every block.
Compare results within each block.
Example – We are testing plant food on tomatoes. We can’t get enough tomato plants from one store.

14. Experimental Design Random Assignment Group 1 4 plants Treatment 1
Control
Block A
12 plants from store A
Group 2
4 plants
Treatment 2
½ dose
Compare
Taste
12 plants from store A and 6 plants from store B
Group 3
4 plants
Treatment 3 full dose
Block
Random Assignment
Group 4
2 plants
Treatment 4
Control
Block B
6 plants from store B
Group 5
2 plants
Treatment 5
½ dose
Compare
Taste
Group 6
2 plants
Treatment 6 full dose

15. Comparing to Samples Randomization
Samples use randomization to make the sample as diverse as the population.
In experiments, we often want the subject group to be as similar as possible. Experiments use randomization to make each treatment group as diverse as the others.

16. Comparing to Samples Grouping by a similar characteristic
Samples stratify to reduce variation in a variable.
Studies match to isolate the variable of interest.
Experiments block to separate the effects of a variable from the effects of the manipulated variable.

17. Complicating Issues
Confounding – when the effects of two different variables can’t be separated.
Ethical considerations
Is it OK to give rats cancer to study tumor growth?
Is it OK to kill pigs to research heart disease?
Is it OK to give a placebo to someone with a deadly disease?