Observational Studies and Experiments Chapter 11 Observational Studies and Experiments

Observational Studies vs. Experiments Do not attempt to control anything; simply collect data Cannot be used to identify cause and effect Retrospective Studies identify subjects and then collect data from the past. Prospective Studies identify subjects and then collect data in the future.

Observational Studies vs. Experiments Example – We ask people how much vitamin C and vitamin E they took, and how long their most recent cold lasted. This is a retrospective study. Example – We ask people to record how much vitamin C and vitamin E they take, and how long their next cold lasts. This is a prospective study.

Observational Studies vs. Experiments Randomized, Comparative Experiments Manipulate one or more variables called factors The specific values for a factor are called levels The combination of levels for all factors is called a treatment Measure one or more other variables called response variables Can be used to identify cause and effect, but only if we can eliminate all other possible causes

Observational Studies vs. Experiments Randomized, Comparative Experiments Example – We manipulate the amount of vitamin C and vitamin E each patient receives, and measure how long the cold lasts. We use two levels of vitamin C ( 0 mg and 500 mg) and three levels of vitamin E (0 mg, 250 mg, and 500 mg). We combine these into 6 treatments: 0 mg vitamin C and 0 mg vitamin E 0 mg vitamin C and 250 mg vitamin E 0 mg vitamin C and 500 mg vitamin E 250 mg vitamin C and 0 mg vitamin E 250 mg vitamin C and 250 mg vitamin E 250 mg vitamin C and 500 mg vitamin E

Observational Studies vs. Experiments When does data happen? Manipulates variables? Determines cause and effect? Retrospective Study Past No Prospective Study Future Experiment Yes

3 Principles of Experimental Design Control – We control as many other variables as we can by making conditions similar between all of the treatments. Randomization – We randomly assign subjects to treatments to minimize variation in variables we can’t control. Replicate We use multiple subjects in each treatment group We repeat the experiment with other groups

Diagrams of Experiments Random Assignment Group 1 Treatment 1 Subjects Compare Group 2 Treatment 2

Statistical Significance We say a result is statistically significant if we don’t believe it is likely to have occurred by chance. If we flip a coin 100 times, and get 54 heads does that mean the coin isn’t fair? Probably not. We expect some variation in results. 54 is probably random chance, so it is not statistically significant. If we flip a coin 100 times, and get 94 heads does that mean the coin isn’t fair? Probably. 94 is way more than we would expect by chance. In this case, the results are statistically significant.

Experimental Design Control Treatment – A treatment that provides a baseline measurement to compare the other treatments to. Example – one treatment group gets none of the new medicine being tested. Example – one treatment group gets the standard dose of the old medicine the new medicine is being compared against.

Experimental Design Blinding – Ensuring that the people involved in the experiment do not know which treatment is applied to each individual. Two classes: Those who could influence the results (subjects, those applying the treatments) Evaluators (judges, evaluating physicians) When one class is blind, the experiment is single-blind. When both classes are blind, the experiment is double-blind.

Experimental Design Placebo – a “fake” treatment to disguise which treatment group a subject is in. example – if one treatment group gets none of the drug, they might be given a sugar pill instead. Often some of the group getting a placebo will actually get better. This is known as the placebo effect.

Experimental Design Blocking – a strategy to control the effects of a variable by separating the subjects into groups. Group the subjects into blocks (for example, by age) Apply each treatment to every block. Compare results within each block. Example – We are testing plant food on tomatoes. We can’t get enough tomato plants from one store.

Experimental Design Random Assignment Group 1 4 plants Treatment 1 Control Block A 12 plants from store A Group 2 4 plants Treatment 2 ½ dose Compare Taste 12 plants from store A and 6 plants from store B Group 3 4 plants Treatment 3 full dose Block Random Assignment Group 4 2 plants Treatment 4 Control Block B 6 plants from store B Group 5 2 plants Treatment 5 ½ dose Compare Taste Group 6 2 plants Treatment 6 full dose

Comparing to Samples Randomization Samples use randomization to make the sample as diverse as the population. In experiments, we often want the subject group to be as similar as possible. Experiments use randomization to make each treatment group as diverse as the others.

Comparing to Samples Grouping by a similar characteristic Samples stratify to reduce variation in a variable. Studies match to isolate the variable of interest. Experiments block to separate the effects of a variable from the effects of the manipulated variable.

Complicating Issues Confounding – when the effects of two different variables can’t be separated. Ethical considerations Is it OK to give rats cancer to study tumor growth? Is it OK to kill pigs to research heart disease? Is it OK to give a placebo to someone with a deadly disease?