1. + Section 4.2 Experiments After this section, you should be able to… DISTINGUISH observational studies from experiments DESCRIBE the language of experiments APPLY the three principles of experimental design DESIGN comparative experiments utilizing completely randomized designs and randomized block designs, including matched pairs design Learning Objectives

2. + Observational Study versus Experiment In contrast to observational studies, experiments don’t just observe individuals or ask them questions. They activelyimpose some treatment in order to measure the response. Definition: An observational study observes individuals and measures variables of interest but does not attempt to influence the responses. An experiment deliberately imposes some treatment on individuals to measure their responses. When our goal is to understand cause and effect, experiments are the only source of fully convincing data.

3. + Observational Study versus Experiment Observational studies of the effect of one variable on another often fail because of confounding between the explanatory variable and one or more lurking variables. Definition: A LURKING VARIABLE is a variable that is not among the explanatory or response variables in a study but that may influence the response variable. CONFOUNDING occurs when two variables are associated in such a way that their effects on a response variable cannot be distinguished from each other. Well-designed experiments take steps to avoid confounding.

4. + Explaining association: causation Variable x and y show a strong association (dashed line). This association may be the result of any of several causal relationships ( solid arrow).

5. + Causal associations Causation: changes in x cause changes in y. Common response: Changes in both x and y are caused by changes in a lurking variable z. Confounding: The effect ( if any ) of x and y is confounded with the effect of a lurking variable. Even when direct causation is present. It is exactly a complete explanation of an association between two variables.

6. + Explaining association: Common Response Beware of lurking variables when thinking about an association between two variables. The observed association between the variables x and y is explained by a lurking variable z. Both x and y change to changes in z. This common response creates an association even though there may be no direct causal link between x and y.

7. + Confounding Two variables are confounded when their effects on a response variable cannot be distinguished from each other. The confounded variables may be either explanatory variables or lurking variables. Even a very strong association between two variables is not by itself good evidence that there is a cause-and-effect link between the variables.

8. +

9. +

10. + Tips :------------ Another name for an EXPLANATORY VARIABLE is a Factorr. Lurking variables create problems in two different ways: They can create extra variability in the response variable. They have the potential to become confounded with the explanatory variable if not controlled Each lurking variable is a potential confounding variable. ( However by following the principles of experimental design, you can usually prevent lurking variables from becoming confounding variables)

11. + Do CYU: P- 233. 1. This was an experiment because a treatment (brightness of screen) was imposed on the laptops. 2. This was an observational study. Students were not assigned to a particular number of meals to eat with their family per week. 3. The explanatory variable is the number of meals per week eaten with their family and the response variable is probably their GPA (or some other measure of their grades). 4. This is an observational study and there may well be lurking variables that are actually influencing the response variable. For instance, families that eat more meals together may also be families where the parents show more interest in their children's education and therefore help them to do better in school.

12. + Experiments The Language of Experiments An experiment is a statistical study in which we actually do something (a treatment ) to people, animals, or objects (the experimental units ) to observe the response. Here is the basic vocabulary of experiments. Definition: A specific condition applied to the individuals in an experiment is called a TREATMENT.  If an experiment has several explanatory variables, a treatment is a combination of specific values of these variables. The experimental units are the smallest collection of individuals to which treatments are applied. When the units are human beings, they often are called subjects. Sometimes, the explanatory variables in an experiment are called factors. Many experiments study the joint effects of several factors. In such an experiment, each treatment is formed by combining a specific value (often called a level) of each of the factors.

13. + A Louse-y Situation A study published in the New England Journal of Medicine (March 11, 2010) compared two medicines to treat HEAD LICE: an oral medication called IVERMECTIN and a topical lotion containing MALATHION. Researchers studied 812 people in 376 households in seven areas around the world. Of the 185 households randomly assigned to IVERMECTIN, 171 were free from head lice after two weeks compared to only 151 of the 191 households randomly assigned to MALATHION. Problem: Identify : the experimental units, explanatory and response variables, the treatments in this experiment.

14. + Solution: Experimental units : the households, not the individual people, since the treatments were assigned to entire households, not separately to individuals within the household. Explanatory variable : type of medication and the response variable is whether the household was lice-free. The treatments : ivermectin and malathion.

15. + Growing Tomatoes Does adding fertilizer affect the productivity of tomato plants? How about the amount of water given to the plants? To answer these questions, a gardener plants 24 similar tomato plants in identical pots in his greenhouse. He will add fertilizer to the soil in half of the pots. Also, he will water 8 of the plants with 0.5 gallons of water per day, 8 of the plants with 1 gallon of water per day and the remaining 8 plants with 1.5 gallons of water per day. At the end of three months he will record the total weight of tomatoes produced on each plant. Problem: Identify the explanatory and response variables, experimental units, and list all the treatments.

16. + Solution: The two explanatory variables are fertilizer and water. The response variable is the weight of tomatoes produced. The experimental units are the tomato plants. There are 6 treatments: (1) fertilizer, 0.5 gallon (2) fertilizer, 1 gallon (3) fertilizer, 1.5 gallons (4) no fertilizer, 0.5 gallons (5) no fertilizer, 1 gallon (6) no fertilizer, 1.5 gallons

17. + How to Experiment Well: The RandomizedComparative Experiment The remedy for confounding is to perform a comparative experiment in which some units receive one treatment and similar units receive another. Most well designed experimentscompare two or more treatments. Comparison alone isn’t enough, if the treatments are given togroups that differ greatly, bias will result. The solution to the problem of bias is random assignment. Definition: In an experiment, random assignment means that experimental units are assigned to treatments at random, that is, using some sort of chance process.

18. + Read example: SAT Prep: P- 237 Then try P- 254 Exercise 59 : Layoffs and “survivor guilt” (a)Write all names on slips of paper, put them in a container and mix thoroughly. Pull one slip out and note the name on it. That person gets assigned treatment 1. Pull another name out and assign that person to treatment 2. The third person gets assigned treatment 3. Keep rotating through the treatments until all have been assigned. (b) Assign the students numbers between 001 and 120. Pick a line on Table D and read off the first 40 numbers between 001 and 120, skipping any that aren’t between 001 and 120 or are repeats. These are assigned to treatment 1. The next 40 numbers read are assigned to treatment 2. The remaining are assigned to treatment 3. (c) Assign the students numbers between 001 and 120. Using the RandInt function on the calculator, and ignoring all repeats, assign the first 40 numbers chosen to treatment 1, the next 40 to treatment 2, and so on.

19. + Experiments The Randomized Comparative Experiment Definition: In a completely randomized design, the treatments are assigned to all the experimental units completely by chance. Some experiments may include a control group that receives an inactive treatment or an existing baseline treatment. Experimental Units Random Assignment Group 1 Group 2 Treatment 1 Treatment 2 Compare Results

20. + Do CYU : Page : 240 : Music students often don’t ……

21. + 2. Using an alphabetical list of the students, assign each student a number between 01 and 29. Pick a line of the random number table and read off two digit numbers until you have 15 numbers between 01 and 29. These students belong in the treatment group where students will meet in small groups. The other students will view the videos alone. 3. The purpose of the control group is to have a group to compare to. Presumably the students have been evaluating their own performances by themselves before. If you incorporate such a group into your experiment, you can evaluate if the group work is actually better.

22. + Experiments Three Principles of Experimental Design Randomized comparative experiments are designed to givegood evidence that differences in the treatments actuallycause the differences we see in the response. 1.Control for lurking variables that might affect the response: Use a comparative design and ensure that the only systematic difference between the groups is the treatment administered. 2.Random assignment: Use impersonal chance to assign experimental units to treatments. This helps create roughly equivalent groups of experimental units by balancing the effects of lurking variables that aren’t controlled on the treatment groups. 3.Replication: Use enough experimental units in each group so that any differences in the effects of the treatments can be distinguished from chance differences between the groups. Principles of Experimental Design

23. Example: Cell Phone and Driving Does talking on a hands-free cell phone distract drivers? Using a high-fidelity simulator, researchers can measure brake-response time, when the car in front suddenly brakes. Researchers have 40 volunteers for the experiment.

24. Cell Phone and Driving Researchers have 40 volunteers for the experiment. Identify a method of randomizing this experiment. Construct an outline of a randomized comparative experiment.

25. + Example: The Physicians’ Health Study Read the description of the Physicians’ Health Study on page241. Explain how each of the three principles of experimentaldesign was used in the study. A placebo is a “dummy pill” or inactive treatment that is indistinguishable from the real treatment.

26. + Experiments: What Can Go Wrong?. Good experiments, therefore, require careful attention todetails to ensure that all subjects really are treated identically. A response to a dummy treatment is called a placebo effect. The strength of the placebo effect is a strong argument for randomized comparative experiments. Whenever possible, experiments with human subjects should be double-blind. Definition: Single Blind experiment : the subjects are unaware of which treatment they are receiving, but the people measuring the response variable do know. Double-blind experiment: neither the subjects nor those who interact with them and measure the response variable know which treatment a subject received.

27. + Experiments Inference for Experiments In an experiment, researchers usually hope to see a differencein the responses so large that it is unlikely to happen justbecause of chance variation. We can use the laws of probability, which describe chancebehavior, to learn whether the treatment effects are larger thanwe would expect to see if only chance were operating. If they are, we call them statistically significant. Definition: An observed effect so large that it would rarely occur by chance is called statistically significant. A statistically significant association in data from a well-designed experiment does imply causation.

28. + Experiments Blocking Completely randomized designs are the simplest statistical designsfor experiments. But just as with sampling, there are times when thesimplest method doesn’t yield the most precise results. Definition A block is a group of experimental units that are known before the experiment to be similar in some way that is expected to affect the response to the treatments. In a randomized block design, the random assignment of experimental units to treatments is carried out separately within each block. Form blocks based on the most important unavoidable sources of variability (lurking variables) among the experimental units.

29. + Control what you can, block on what you can’t control, and randomize to create comparable groups.

30. + Read Example : P- 248 Men, Women, Advertising  Then try P- 258 : Exercise : 79.  The difference in soil fertility among the plots is a potential lurking variable. A completely randomized design could assign one of the varieties of corn to more fertile plots just by chance. If those plots produced extremely high yields, we wouldn’t know if it was due to the corn variety or to soil fertility. Blocking will allow researchers to control for the variability in yield due to soil fertility.

31. + 79 (b) The researchers should use the rows as the blocks. All plots in the same row have the same amount of fertility and so are as similar as possible. (c) Let the digits 1-5 correspond to the five corn varieties A-E. Begin with, say, line 110, and assign the letters to the rows from west to east (left to right). Use a new line (111, 112, 113, 114, and 115) for each row. For example, for Block 1, we obtain 3, 4, 1, 2, 5 corresponding to varieties C, D, A, B and lastly E being planted in the first row from west to east. The remaining rows are assigned using this same process. The results of this assignment are shown in the table below.

32. + -

33. + Experiments Matched-Pairs Design A common type of randomized block design for comparing twotreatments is a matched pairs design. The idea is to create blocks bymatching pairs of similar experimental units. Definition A matched-pairs design is a randomized blocked experiment in which each block consists of a matching pair of similar experimental units. Chance is used to determine which unit in each pair gets each treatment. Sometimes, a “pair” in a matched-pairs design consists of a single unit that receives both treatments. Since the order of the treatments can influence the response, chance is used to determine with treatment is applied first for each unit.

34. Matched Pairs (Type of Blocking) Only two treatments are being compared Impose both treatments on the same subjects ◦ Ex: Pre- and post-tests ◦ Note: The measurements will not be independent Appeared on the exam a couple of years ago ◦ Ex: Comparing old vs. new waterproofing treatment on leather boots…take one boot of the pair and apply new on one and old on the other and compare results

35. Do activity: Get your heart beating (P- 249)

36. + Experiments Example: Standing and Sitting Pulse Rate Consider the Fathom dotplots from a completely randomizeddesign and a matched-pairs design. What do the dotplotssuggest about standing vs. sitting pulse rates?

37. + Section 4.2 Experiments In this section, we learned that… We can produce data intended to answer specific questions by observational studies or experiments. In an experiment, we impose one or more treatments on a group of experimental units (sometimes called subjects if they are human). The design of an experiment describes the choice of treatments and the manner in which the subjects are assigned to the treatments. The basic principles of experimental design are control for lurking variables, random assignment of treatments, and replication (using enough experimental units). Many behavioral and medical experiments are double-blind. Summary

38. + Section 4.2 Experiments In this section, we learned that… Some experiments give a placebo (fake treatment) to a control group that helps confounding due to the placebo-effect. In addition to comparison, a second form of control is to form blocks of individuals that are similar in some way that is important to the response. Randomization is carried out within each block. Matched pairs are a common form of blocking for comparing just two treatments. In some matched pairs designs, each subject receives both treatments in a random order. Summary, con’t

39. + Section 4.3 Using Studies Wisely After this section, you should be able to… DESCRIBE the challenges of establishing causation DEFINE the scope of inference DESCRIBE data ethics in designing studies Learning Objectives

40. + Using Studies Wisely Scope of Inference What type of inference can be made from a particular study? The answer depends on the design of the study. Well-designed experiments randomly assign individuals to treatment groups. However, most experiments don’t selectexperimental units at random from the larger population. Thatlimits such experiments to inference about cause and effect. Observational studies don’t randomly assign individuals to groups, which rules out inference about cause and effect.Observational studies that use random sampling can makeinferences about the population.

41. + Read example from P- 262 : Vitamin C and Canker sores.

42. + Using Studies Wisely The Challenges of Establishing Causation A well-designed experiment tells us that changes in the explanatory variable cause changes in the response variable. Lack of realism can limit our ability to apply the conclusions of an experiment to the settings of greatest interest. In some cases it isn’t practical or ethical to do an experiment. Consider these questions: Does texting while driving increase the risk of having an accident?  Does going to church regularly help people live longer?  Does smoking cause lung cancer? It is sometimes possible to build a strong case for causation in the absence of experiments by considering data fromobservational studies.

43. + Using Studies Wisely The Challenges of Establishing Causation When we can’t do an experiment, we can use the following criteria for establishing causation.  The association is strong.  The association is consistent.  Larger values of the explanatory variable are associated with stronger responses.  The alleged cause precedes the effect in time.  The alleged cause is plausible. Discuss how each of these criteria apply to the observational studies of the relationship between smoking and lung cancer.

44. + Using Studies Wisely Data Ethics Complex issues of data ethics arise when we collect data from people. Here are some basic standards of data ethics thatmust be obeyed by all studies that gather data from humansubjects, both observational studies and experiments. All planned studies must be reviewed in advance by an institutional review board charged with protecting the safety and well-being of the subjects. All individuals who are subjects in a study must give their informed consent before data are collected. All individual data must be kept confidential. Only statistical summaries for groups of subjects may be made public. Basic Data Ethics

45. + Do from P -256 # 66, 72, 76, 87, 88,90.

46. + # 66 In a controlled scientific study, the effects of factors other than the nonphysical treatment (e.g., the placebo effect, differences in the prior health of the subjects) can be eliminated or accounted for, so that the differences in improvement observed between the subjects can be attributed to the differences in treatments.

47. + #72  “Double-blind” means that the treatment (testosterone or placebo) assigned to a subject was unknown to both the subject and those responsible for assessing the effectiveness of that treatment.  “Randomized” means that patients were randomly assigned to receive either the testosterone supplement or a placebo.  “Placebo-controlled” means that some of the subjects were given placebos. Even though these possess no medical properties, some subjects may show improvement or benefits just as a result of participating in the experiment; the placebos allow those doing the study to observe this effect.

48. + #76 (a) Dotplots for both groups are shown below. The differences for the patients in the active group follow a distribution with a gap between 1 and 4. We might even stretch a bit and say that the distribution is roughly symmetric with a mean of about 5. The differences ranged from 0 to 10. The distribution of the differences for the patients in the inactive group is skewed to the right with a center slightly above 1. Many patients reported no change in their pain ratings (a difference of 0) and the largest difference was 5. This means that those in the active group had a higher mean difference (5) than those in the inactive group (1) and had a distribution with much more variability (range = 10 for the active group and range = 5 for the inactive group).

49. +

50. + (b) The average difference for the active group is 5.241 and the average difference for the inactive group is 1.095. The difference in these two means (active – inactive) is 4.146. (c) Write each patient’s name and difference in pain score on an index card, shuffle them up and deal them into two piles with 29 in one pile and the rest in the other. The 29 in the first pile will be considered the active group. (d) The Fathom dotplot shows that none of the 50 simulated differences was larger than 4.146. Thus, a difference of 4.146 would be extremely unlikely if both types of magnets provided the same level of relief. We would conclude that the active magnets probably do provide relief for polio patients.

51. + #87 (a) Confounding: This experiment confounds gender with deodorant. If the students find a difference between the two groups, they will not know if it is a gender difference or due to the deodorant. (b) A better design would be a matched pairs design. In this case each student would have one armpit randomly assigned to receive deodorant A and the other deodorant B. Have each student rate the difference between their own armpits at the end of the day.

52. + #88 (a)This experiment confounds the alarm setting (either set or not) with weekend days and week days. It is likely that Justin goes to bed at different times on the weekend than he does on the weekday and this may have an effect on his average wake-up time. (b) A better design would be a randomized block design with the weekend days being one block and the week days being the other block. Justin would randomly assign one weekend day to set his alarm and the other day for having no alarm, and do likewise for the week days. This allows us to make sure the days in which the alarm is set are similar to the days in which it is not set.

53. + #90 Take the 30 students and divide randomly into two equal groups (15 students each). Now randomly select one of the two groups. In this group, have music playing while a story is read. In the other group, read a story without music in the background. Test the students for recall. Now reverse the treatments. For those students who had music with the first story, read to them with no background noise and use background music with the others. Test the students for recall. Analyze the difference in recall for the individual students.

54. +