1. 4.2 Experiments

2. Observational Study vs. Experiment
Observes individuals and measures variables of interest but does not attempt to influence the responses.
Deliberately imposes some treatment on individuals to measure their responses
When our goal is to understand CAUSE/EFFECT, experiments are they ONLY source of fully convincing data.
IF YOU DON’T LEARN ANYTHING ELSE FROM MY CLASS, LEARN THIS!!!!!!!

3. Confounding
AP EXAM TIP:
If you are asked to identify a possible confounding variable in a given setting, you are expected to explain how the variable you chose
Is associated with the explanatory variable AND
Affects the response variable
Occurs when two variables are associated in such a way that their effects on a response variable cannot be distinguished from each other

4. Language of Experiments
Treatment: a specific condition applied to the individuals in an experiment. (If an experiment has several explanatory variables, a treatment in a combo of specific values of these variables.)
Experimental Units: the smallest collection of individuals to which treatments are applied.
Subjects: when the units are humans
Factor: the explanatory variables (sometimes called factors)
Level: each treatment is formed by combining a specific value of the factors

5. HW #25:
19, 21, 23, 25
45, 47, 49, 51

6. Have HW #25 out

7. TV Advertising Example (pg. 239)
Factor: Length (2 levels)
Factor: Frequency (3 levels)
So, 6 treatments total

8. How to experiment badly...

9. Example: Are Online SAT Prep Courses Effective?
Pg. 240
Experimental Units
Treatment
Measure Response
WATCH FOR CONFOUNDING!
COMPARISON!

10. How to experiment well...

11. Experimental units are assigned to treatments using a chance process
The remedy for confounding is to perform a comparative experiment in which some units receive one treatment and similar units receive another.
But, comparison isn’t enough! (If the treatments are given to 2 groups that differ greatly, BIAS will result!!!)
Solution:
RANDOM ASSIGNMENT:
Experimental units are assigned to treatments using a chance process
To go along with SAT Prep: Online vs. Classroom
Randomly assign treatment to each student
HAT METHOD: put all names in a hat. Draw names one at a time, until you have 25 names. Those 25 will take the online course. The remaining 25 will take the classroom course.
OR, put 25 “online” and 25 “classroom” slips in a hat and have each student come up one at a time and draw a treatment.
TECHNOLOGY: assign a number to eat subject in alphabetical order by last name. Use ranInt() to generate 25 unique numbers. Those 25 will take the online course, the remaining 25 will take the classroom course.
TABLE D: assign numbers to each subject. Choose a line in Table D and the first 25 (unique) numbers you get to will be in the group that takes the online course. Remaining will take the classroom course.
GOAL: to create groups of experimental units that are roughly equivalent at the beginning of the study. It balances the effects of other variables among treatment groups. (*only systematic difference between the 2 groups should be the treatment assigned)

12. AP Exam Common Error:
Many students lose credit on the AP Exam for failing to adequately describe how they assign the treatments to experimental units in an experiment.
Most important, the method the students use must be random. In addition, the method must be described in sufficient detail so that 2 knowledgeable users of statistics could follow the student’s description and carry out the method in exactly the same way.

13. For example, saying “assign students to the two groups using random digits” isn’t sufficiently detailed because there are many ways to use random digits. If a student chooses to use random digits, he or she must use labels of the same length (e.g., 01-30, not 1-30).
Students must also address how they will deal with repeated numbers that come up when using a random digit table or random number generator. For example, they can say “ignoring repeats” or state that they will generate 25 “different” numbers from 1 to 50.

14. Principles of Experimental Design
Comparison use a design that compares 2+ treatments
Random Assignment use change to assign experimental units to treatments. Doing so helps create roughly equivalent units by balancing the effects of other variables among the treatment groups.
Control keep other variables that might affect the response the same for all groups
Replication use enough experimental units in each group so that any differences in the effects of the treatments can be distinguished from chance differences between the groups
*random assignment doesn’t ELIMINATE the effects of other variables -- it just balances them. So in each group there should be approx the same number of males/females, young/old, etc etc
*control -- yes, could be a control group. But good experimenters should always do whatever is possible to control other variables by making them the same for all treatment groups.
*the more replication, the more balanced the treatment groups will be after the random assignment.

15. Completely Randomized Design - treatments are assigned to all the experimental units completely by chance.
Control Group - receives an inactive treatment or an existing baseline treatment
Kinda like taking an SRS when selecting a sample
There are no restrictions on who can be assigned to each treatment.

16. AP Exam Tip:
Use the hat method whenever possible -- it is easy to describe and helps you avoid making the mistakes that are possible when using random digits or coin flips.
Just make sure that the slips of paper are the same size and that they are well mixed!
(See “Think About It” on pg. 245 for an example of bad assignment)

17. HW #26:
57, 59, 61, 63, 65

18. Have HW #26 out

19. What could go wrong??
Placebo Effect: the response to a dummy treatment
Double-Blind Experiment: neither the subjects nor those who interact with them/measure the response variable know which treatment a subject received.
Placebo: like a mother kissing her baby’s “boo-boo”
**read “more expensive placebo” on pg. 247
Experiments can still be single blind!
For example: bottom of pg. 248

20. Triple Blind Study Participant doesn’t know what he is taking.
Physician doesn’t know what the participant is taking.
Statistician doesn’t know what he is doing.

21. Inference for Experiments
Researchers usually hope to see a difference in the responses so large that it is unlikely to happen just because of chance variation
We can use laws of probability to learn whether the treatments effects are larger than we would expect to see if only chance were operating. If they are, we call them statistically significant
An observed effect so large that it would rarely occur by chance is called statistically significant
A statistically significant association in data from a well-designed experiment DOES imply causation.
Only statistically significant IF the difference in the response is bigger than what would be expected due to chance variation in the random assignment
Can only imply causation in an EXPERIMENT -- not in an observational study

22. Blocking:
Grouping similar individuals together (similar to stratifying)

23. Blocking:
Block: a group of experimental units that are known before the experiment to be similar in some way that is expected to affect the response to the treatments
In a RANDOMIZED BLOCK DESIGN the random assignment of experimental units to treatments is carried out separately within each block.
Students are not expected to analyze the results of a blocked experiment on the AP exam, except when the blocks are matched pairs (coming soon).

24. Matched Pairs Design:
A common type of randomized block design for comparing two treatments
Basically, a special case of a randomized block design that uses blocks of size 2

25. Standing and Sitting Pulse Rate
Matched Pairs Design:
Standing and Sitting Pulse Rate
Mean pulse rate STANDING = 74.83
Mean pulse rate SITTING = 68.33
So average pulse rate is 6.5 bpm higher in standing group
HOWEVER, variability in dot plots creates a lot of overlap
*these data don’t provide convincing evidence

26. Standing and Sitting Pulse Rate
Matched Pairs Design:
Standing and Sitting Pulse Rate
*MATCHED PAIRS EXPERIMENT
Shows the difference (standing - sitting)
Mean difference = 6.8 bpm
21/24 students recorded a positive difference (meaning the standing pulse rate was higher)
*PROVIDE CONVINCING EVIDENCE

27. HW #27:
67, 69, 71, 73

28. Have HW #27 out

29. FRQs: 1997 #2 (Fishtanks) 2002 #2 (Waterproofing Boots)