1. Statistical Thinking Experiments in the Real World
Chapter 6
Experiments in the Real World
Chapter 6
Chapter 6

2. Clinical Trials
Experiments that study the effectiveness of medical treatments on actual patients.
Chapter 6

3. Statistical Thinking
Thought Question 1
Suppose you are interested in determining if drinking a glass of red wine each day helps prevent heartburn. You recruit 40 adults age 50 and older to participate in an experiment. You want half of them to drink a glass of red wine each day and the other half to not do so. You ask them which they would prefer, and 20 say they would like to drink the red wine and the other 20 say they would not. You ask each of them to record how many cases of heartburn they have in the next six months. At the end of that time period, you compare the results reported from the two groups. Give three reasons why this is not a good experiment.
Chapter 6
Chapter 6

4. Randomization in Expts.
Completely Randomized Design
The experimental units are divided into different groups through a process of random selection.
Chapter 6

5. Blocking
Refers to the idea of only making comparisons within relatively similar groups of subjects
In the smoking lung cancer example, we could
“block” age, subjects in a block will have similar
ages.
Completely Randomized Block Design
Chapter 6

6. Blocking and randomization in expts.
Block to ensure fair comparisons with respect
to factors known to be important
Randomize to try to obtain comparability with
respect to unknown factors
Randomization also allows the calculation of
how much the estimates made from the study
data are likely to be in error
Chapter 6

7. Blocking and randomization in expts.
Block to ensure fair comparisons with respect to factors known to be important.
“Block what you can and randomize what you cannot.”
Chapter 6

8. “Blocking” vs “stratification”
word used in describing an experimental design
“Stratification”
used in describing a survey or observational study
Both refer to idea of only making comparisons within relatively similar groups of subjects
Chapter 6

9. Blinding:
Preventing people involved in an experiment from knowing which experimental subjects have received which treatment
One may be able to blind
subjects themselves
people administering the treatments
people measuring the results
Double blind: Both the subjects and those administering the treatments have been blinded.
Chapter 6

10. Statistical Thinking
Chapter 6
Chapter 6

11. Hawthorne, Placebo, and Experimenter Effects
Statistical Thinking
Hawthorne, Placebo, and Experimenter Effects
The problem:
people may respond differently when they know they are part of an experiment.
The solution:
use placebos, control groups, and double-blind studies when possible.
Chapter 6
Chapter 6

12. Hawthorne, Placebo, and Experimenter Effects : Case Study I
Statistical Thinking
Hawthorne, Placebo, and Experimenter Effects : Case Study I
1920’s Experiment by Hawthorne Works of the Western Electric Company
What changes in working conditions improve productivity of workers?
More lighting?
Less lighting?
Other changes?
All changes improved productivity!
Chapter 6
Chapter 6

13. Hawthorne, Placebo, and Experimenter Effects : Case Study II
Statistical Thinking
Hawthorne, Placebo, and Experimenter Effects : Case Study II
Experimenter Effects in Behavioral Research (Rosenthal, 1976, Irvington Pub., p. 410)
Teachers given a list of student names
told these were students “who would show unusual academic development.”
IQ was measured at end of year
first graders on list: 15 points higher
second graders on list: 9.5 points higher
older: no striking difference
Great expectations = self-fulfilling prophecy
students were randomly selected (did not have high IQ)
Chapter 6
Chapter 6

14. Experiments: Some Techniques
Statistical Thinking
Experiments: Some Techniques
Matched Pairs
to reduce a source of variability in responses
the same or similar subjects receive each treatment
Chapter 6
Chapter 6

15. (not) Double-Blinded: Case Study
Statistical Thinking
(not) Double-Blinded: Case Study
Mozart, Relaxation and Performance on Spatial Tasks (Nature, Oct. 14, 1993, p. 611)
Variables:
Explanatory: Relaxation condition assignment
Response: Stanford-Binet IQ measure
Not double-blinded
Participants know their treatment group
Single-blinded
Those measuring the IQ
Chapter 6
Chapter 6

16. Double-Blinded: Case Study
Statistical Thinking
Double-Blinded: Case Study
Quitting Smoking with Nicotine Patches (JAMA, Feb. 23, 1994, pp )
Variables:
Explanatory: Treatment assignment
Response: Cessation of smoking (yes/no)
Double-blinded
Participants do not know which patch they received
Nor do those measuring smoking behavior
Chapter 6
Chapter 6

17. Experiments: Difficulties and Disasters
Statistical Thinking
Experiments: Difficulties and Disasters
Extraneous variables
Confounding variables (in chapter 5)
Interacting variables
Hawthorne, placebo, and experimenter effects
Refusals, non-adherers, dropouts
Extending the results (generalizing)
Chapter 6
Chapter 6

18. Interacting Variables
Statistical Thinking
Interacting Variables
The problem:
effect of explanatory variable on response variable may vary over levels of other variables.
The solution:
measure and study potential interacting variables.
does the relationship between explanatory and response variables change for different levels of these interacting variables?
if so, report results for different groups defined by the levels of the interacting variables.
Chapter 6
Chapter 6

19. Interacting Variables: Case Study
Statistical Thinking
Interacting Variables: Case Study
Quitting Smoking with Nicotine Patches (JAMA, Feb. 23, 1994, pp )
Researchers considered:
smoker at home
found this to be an interacting variable:
Percent quitting Nicotine Placebo
Smoker at home % %
No smoker at home % %
other variables: age, weight, depression
no interactions found
Chapter 6
Chapter 6

20. Extending the Results ( Can We Generalize? )
Statistical Thinking
Extending the Results ( Can We Generalize? )
The problem:
lack of generalizability due to:
unrealistic treatments
unnatural settings
sample that is not representative of population
The solution:
Researchers should use natural settings with a properly chosen sample.
Chapter 6
Chapter 6

21. Extending the Results : Case Study
Statistical Thinking
Extending the Results : Case Study
Does Aspirin Prevent Heart Attacks? (NEJM, Jan. 28, 1988, pp )
Participants were measured in their natural setting (at home)
Only healthy male physicians were participants
Results may not apply to:
male physical labourers
women
Chapter 6
Chapter 6

22. Example: # Page111
The progress of a type of cancer differs in men and
women. A clinical experiment to compare four
therapies for this cancer therefore treats sex as a
blocking variable.
You have 500 male and 300 female patients who
are willing to serve as subjects. Use a diagram to
outline a block design for this experiment.
Chapter 6

23. Example Cont. (b) What are the advantages of a block design over a
completely randomized design using these 800 subjects?
A block design allows the researchers to control for differences between
men and women. If, for example, the treatment is effective for men but not
women (or vice versa), the treatment might be found to be ineffective
overall if both genders are mixed together.
Chapter 6

24. Key Concepts Double-Blind Experiment Difficulties and Disasters
Statistical Thinking
Key Concepts
Double-Blind Experiment
Difficulties and Disasters
Experimental Designs
Completely Randomized Design
Matched Pairs Design
Block Design
Chapter 6
Chapter 6

25. Quiz
In what sense does random allocation make for comparisons that are fair or unbiased?
What is a completely randomized design?
What is blocking?
Why do we use blocking in designing experiments?
What is a control group and why are control groups used?
What is a placebo effect?
Chapter 6