2. Copyright © 2013, 2009, and 2007, Pearson Education, Inc. Chapter 4 Gathering Data Section 4.1 Experimental and Observational Studies

3. Copyright © 2013, 2009, and 2007, Pearson Education, Inc. 3 A researcher conducts an experiment by assigning subjects to certain experimental conditions and then observing outcomes on the response variable. The experimental conditions, which correspond to assigned values of the explanatory variable, are called treatments. Type of Study: Experiment

4. Copyright © 2013, 2009, and 2007, Pearson Education, Inc. 4 In an observational study, the researcher observes values of the response variable and explanatory variables for the sampled subjects, without anything being done to the subjects (such as imposing a treatment). In short, an observational study merely observes rather than experiments with the study subjects. An experimental study assigns to each subject a treatment and then observes the outcome on the response variable. Type of Study: Observational Study

5. Copyright © 2013, 2009, and 2007, Pearson Education, Inc. 5 A sample survey selects a sample of people from a population and interviews them to collect data. A sample survey is a type of observational study. A census is a survey that attempts to count the number of people in the population and to measure certain characteristics about them. Observational Study – Sample Survey

6. Copyright © 2013, 2009, and 2007, Pearson Education, Inc. 6 A headline read: “Student Drug Testing Not Effective in Reducing Drug Use” in a news release from the University of Michigan. Facts about the study:  76,000 students nationwide  497 high schools and 225 middle schools  Schools selected for the study included schools that tested for drugs and schools that did not test for drugs  Each student filled out a questionnaire asking about his/her drug use Example: Drug Testing and Student Drug Use

7. Copyright © 2013, 2009, and 2007, Pearson Education, Inc. 7 Questions: 1. What were the response and explanatory variables? 2. Was this an observational study or an experiment? Conditional Proportions on Drug Use Example: Drug Testing and Student Drug Use

8. Copyright © 2013, 2009, and 2007, Pearson Education, Inc. 8 This study was an observational study. In order for it to be an experiment, the researcher would had to have assigned each school to use or not use drug testing rather than leaving this decision to the school. Example: Drug Testing and Student Drug Use

9. Copyright © 2013, 2009, and 2007, Pearson Education, Inc. 9 An experiment reduces the potential for lurking variables to affect the result. Thus, an experiment gives the researcher more control over outside influences. Only an experiment can establish cause and effect. Observational studies can not. Experiments are not always possible due to ethical reasons, time considerations and other factors. Comparing Experiments and Observational Studies

10. Copyright © 2013, 2009, and 2007, Pearson Education, Inc. Chapter 4 Gathering Data Section 4.2 Good and Poor Ways to Sample

11. Copyright © 2013, 2009, and 2007, Pearson Education, Inc. 11 The sampling frame is the list of subjects in the population from which the sample is taken, ideally it lists the entire population of interest. The sampling design determines how the sample is selected. Sampling Frame and Sampling Design

12. Copyright © 2013, 2009, and 2007, Pearson Education, Inc. 12 Random Sampling is the best way of obtaining a sample that is representative of the population. A simple random sample of ‘n’ subjects from a population is one in which each possible sample of that size has the same chance of being selected. A simple random sample is often just called a random sample. The “simple” adjective distinguishes this type of sampling from more complex random sampling designs presented in Section 4.4. Simple Random Sampling, (SRS)

13. Copyright © 2013, 2009, and 2007, Pearson Education, Inc. 13 A campus club decides to raise money for a local charity by selling tickets. * The Athletic Department has donated 2 pairs of football season tickets as prizes. * The group of 60 individuals who purchased tickets to the banquet comprises the population. Partial List of Possible Samples: (1,2), (1,3), (1,4),..., (1,58), (1,59), (1,60) (2,3), (2,4),..., (2,58), (2,59), (2,60) Etc… Questions: 1.What is the chance that a particular sample of size 2 will be drawn? 2.Professor Shaffer is in attendance at the banquet and holds entry number 1. What is the chance that her entry will be chosen? SRS Example: Drawing Prize Winners

14. Copyright © 2013, 2009, and 2007, Pearson Education, Inc. 14 SRS: Table of Random Numbers Table 4.1 A Portion of a Table of Random Numbers

15. Copyright © 2013, 2009, and 2007, Pearson Education, Inc. 15 To select a simple random sample:  number the subjects in the sampling frame using numbers of the same length (number of digits).  select numbers of that length from a table of random numbers or using a random number generator.  include in the sample those subjects having numbers equal to the random numbers selected. SUMMARY: Using Random Numbers to select a SRS

16. Copyright © 2013, 2009, and 2007, Pearson Education, Inc. 16 Sample surveys are commonly used to estimate population percentages. These estimates include a Margin of Error which tells us how well the sample estimate predicts the population percentage. When a SRS of n subjects is used, the margin of error is approximately equal to Accuracy of the Results from Surveys with Random Sampling

17. Copyright © 2013, 2009, and 2007, Pearson Education, Inc. 17 A survey result states: “The margin of error is plus or minus 3 percentage points”. This means: “It is very likely that the reported sample percentage is no more than 3% lower or 3% higher than the population percentage”. Accuracy of the Results from Surveys with Random Sampling

18. Copyright © 2013, 2009, and 2007, Pearson Education, Inc. 18 Bias: When certain outcomes will occur more often in the sample than they do in the population.  Sampling bias occurs from using nonrandom samples or having undercoverage.  Nonresponse bias occurs when some sampled subjects cannot be reached or refuse to participate or fail to answer some questions.  Response bias occurs when the subject gives an incorrect response (perhaps lying) or the way the interviewer asks the questions (or wording of a question in print) is confusing or misleading. A Large Sample Does Not Guarantee An Unbiased Sample! SUMMARY: Types of Bias in Sample Surveys

19. Copyright © 2013, 2009, and 2007, Pearson Education, Inc. 19 Convenience Sample: a type of survey sample that is easy to obtain.  Unlikely to be representative of the population.  Often severe biases result from such a sample.  Results apply ONLY to the observed subjects. Poor Ways to Sample

20. Copyright © 2013, 2009, and 2007, Pearson Education, Inc. 20 Volunteer Sample: most common form of convenience sample.  Subjects volunteer for the sample.  Volunteers do not tend to be representative of the entire population. Poor Ways to Sample

21. Copyright © 2013, 2009, and 2007, Pearson Education, Inc. 21  Identify the population of all subjects of interest.  Construct a sampling frame which attempts to list all subjects in the population.  Use a random sampling design to select n subjects from the sampling frame.  Be cautious of sampling bias due to nonrandom samples (such as volunteer samples) and sample undercoverage, response bias from subjects not giving their true response or from poorly worded questions, and nonresponse bias from refusal of subjects to participate. We can make inferences about the population of interest when sample surveys that use random sampling are employed. SUMMARY: Key Parts of a Sample Survey

22. Copyright © 2013, 2009, and 2007, Pearson Education, Inc. Chapter 4 Gathering Data Section 4.3 Good and Poor Ways to Experiment

23. Copyright © 2013, 2009, and 2007, Pearson Education, Inc. 23 Experimental units: The subjects of an experiment; the entities that we measure in an experiment. Treatment: A specific experimental condition imposed on the subjects of the study; the treatments correspond to assigned values of the explanatory variable. Explanatory variable: Defines the groups to be compared with respect to values on the response variable. Response variable: The outcome measured on the subjects to reveal the effect of the treatment(s). Elements of an Experiment

24. Copyright © 2013, 2009, and 2007, Pearson Education, Inc. 24  An experiment deliberately imposes treatments on the experimental units in order to observe their responses.  The goal of an experiment is to compare the effect of the treatment on the response.  Experiments that are randomized occur when the subjects are randomly assigned to the treatments; randomization helps to eliminate the effects of lurking variables. Experiments

25. Copyright © 2013, 2009, and 2007, Pearson Education, Inc. 25 1.Control/Comparison Group: allows the researcher to analyze the effectiveness of the primary treatment. 2.Randomization: eliminates possible researcher bias, balances the comparison groups on known as well as on lurking variables. 3.Replication: allows us to attribute observed effects to the treatments rather than ordinary variability. 3 Components of a Good Experiment

26. Copyright © 2013, 2009, and 2007, Pearson Education, Inc. 26  A placebo is a dummy treatment, i.e. sugar pill. Many subjects respond favorably to any treatment, even a placebo.  A control group typically receives a placebo. A control group allows us the analyze the effectiveness of the primary treatment.  A control group need not receive a placebo. Clinical trials often compare a new treatment for a medical condition, not with a placebo, but with a treatment that is already on the market. Component 1: Control or Comparison Group

27. Copyright © 2013, 2009, and 2007, Pearson Education, Inc. 27 Experiments should compare treatments rather than attempt to assess the effect of a single treatment in isolation.  Is the treatment group better, worse, or no different than the control group? Example: 400 volunteers are asked to quit smoking and each start taking an antidepressant. In 1 year, how many have relapsed? Without a control group (individuals who are not on the antidepressant), it is not possible to gauge the effectiveness of the antidepressant. Component 1: Control or Comparison Group

28. Copyright © 2013, 2009, and 2007, Pearson Education, Inc. 28 Placebo effect (power of suggestion). The “placebo effect” is an improvement in health due not to any treatment but only to the patient’s belief that he or she will improve. In some experiments, a control group may receive an existing treatment rather than a placebo. Placebo Effect

29. Copyright © 2013, 2009, and 2007, Pearson Education, Inc. 29 To have confidence in our results we should randomly assign subjects to the treatments. In doing so, we  eliminate bias that may result from the researcher assigning the subjects.  balance the groups on variables known to affect the response.  balance the groups on lurking variables that may be unknown to the researcher. Component 2: Randomization

30. Copyright © 2013, 2009, and 2007, Pearson Education, Inc. 30 Replication is the process of assigning several experimental units to each treatment.  The difference due to ordinary variation is smaller with larger samples.  We have more confidence that the sample results reflect a true difference due to treatments when the sample size is large.  Since it is always possible that the observed effects were due to chance alone, replicating the experiment also builds confidence in our conclusions. Component 3: Replication

31. Copyright © 2013, 2009, and 2007, Pearson Education, Inc. 31  Ideally, subjects are unaware, or blind, to the treatment they are receiving.  If an experiment is conducted in such a way that neither the subjects nor the investigators working with them know which treatment each subject is receiving, then the experiment is double-blinded.  A double-blinded experiment controls response bias from the subject and experimenter. Blinding the Experiment

32. Copyright © 2013, 2009, and 2007, Pearson Education, Inc. 32 If an experiment (or other study) finds a difference in two (or more) groups, is this difference really important? If the observed difference is larger than what would be expected just by chance, then it is labeled statistically significant. Rather than relying solely on the label of statistical significance, also look at the actual results to determine if they are practically significant. Define Statistical Significance

33. Copyright © 2013, 2009, and 2007, Pearson Education, Inc. 33 Recall that the goal of experimentation is to analyze the association between the treatment and the response for the population, not just the sample. However, care should be taken to generalize the results of a study only to the population that is represented by the study. Generalizing Results

34. Copyright © 2013, 2009, and 2007, Pearson Education, Inc. 34  A good experiment has a control comparison group, randomization in assigning experimental units to treatments, blinding, and replication. The experimental units are the subjects—the people, animals, or other objects to which the treatments are applied.  The treatments are the experimental conditions imposed on the experimental units. One of these may be a control (for instance, either a placebo or an existing treatment) that provides a basis for determining if a particular treatment is effective. The treatments correspond to values of an explanatory variable. SUMMARY: Key Parts of a Good Experiment

35. Copyright © 2013, 2009, and 2007, Pearson Education, Inc. 35  Randomize in assigning the experimental units to the treatments. This tends to balance the comparison groups with respect to lurking variables.  Replicating the treatments on many experimental units helps, so that observed effects are not due to ordinary variability but instead are due to the treatment. Repeat studies to increase confidence in the conclusions. SUMMARY: Key Parts of a Good Experiment

36. Copyright © 2013, 2009, and 2007, Pearson Education, Inc. Chapter 4 Gathering Data Section 4.4 Other Ways to Conduct Experimental and Nonexperimental Studies

37. Copyright © 2013, 2009, and 2007, Pearson Education, Inc. 37 It is not always possible to conduct an experiment so it is necessary to have well designed, informative studies that are not experimental, e.g., sample surveys that use randomization.  Simple Random Sampling  Cluster Sampling  Stratified Random Sampling Sample Surveys: Random Sampling Designs

38. Copyright © 2013, 2009, and 2007, Pearson Education, Inc. 38 Cluster Random Sample Steps  Divide the population into a large number of clusters, such as city blocks.  Select a simple random sample of the clusters.  Use the subjects in those clusters as the sample. Sample Surveys: Cluster Random Sample

39. Copyright © 2013, 2009, and 2007, Pearson Education, Inc. 39 Cluster Random Sample Preferable sampling design if  a reliable sampling frame is unavailable, or  the cost of selecting a SRS is excessive Disadvantage  Usually need a larger sample size than with a SRS in order to achieve a particular margin of error. Sample Surveys: Cluster Random Sample

40. Copyright © 2013, 2009, and 2007, Pearson Education, Inc. 40 Stratified Random Sample Steps  Divide the population into separate groups, called strata.  Select a simple random sample from each strata.  Combine the samples from all strata to form complete sample. Sample Surveys: Stratified Random Sample

41. Copyright © 2013, 2009, and 2007, Pearson Education, Inc. 41 Stratified Random Sample  Advantage is that you can include in your sample enough subjects in each group (stratum) you want to evaluate.  Disadvantage is that you must have a sampling frame and know the stratum into which each subject belongs. Sample Surveys: Stratified Random Sample

42. Copyright © 2013, 2009, and 2007, Pearson Education, Inc. 42 Comparing Random Sampling Methods Figure 4.2 Ways of Randomly Sampling 40 Students. The figure is a schematic for a simple random sample, a cluster random sample of 8 clusters of students who live together, and a stratified random sample of 10 students from each class (Fresh., Soph., Jnr., Snr.). Question: What’s the difference between clustering and stratifying?

43. Copyright © 2013, 2009, and 2007, Pearson Education, Inc. 43 Comparison of different random sampling methods. A good sampling design ensures that each subject in a population has an opportunity to be selected. The design should incorporate randomness. Table 4.2 summarizes the random sampling methods we’ve presented. Comparing Random Sampling Methods

44. Copyright © 2013, 2009, and 2007, Pearson Education, Inc. 44 An observational study can yield useful information when an experiment is not practical. Types of observational studies:  Sample Survey: attempts to take a cross section of a population at the current time.  Retrospective study: looks into the past.  Prospective study: follows its subjects into the future. Causation can never be definitively established with an observational study, but well designed studies can provide supporting evidence for the researcher’s beliefs. Types of Observational Studies

45. Copyright © 2013, 2009, and 2007, Pearson Education, Inc. 45 A case-control study is a retrospective observational study in which subjects who have a response outcome of interest (the cases) and subjects who have the other response outcome (the controls) are compared on an explanatory variable. This is a popular design for medical studies in which it is not practical or ethical to perform an experiment. We can’t randomly assign subjects into a smoking group and a nonsmoking group—this would involve asking some subjects to start smoking. Retrospective Case-Control Study

46. Copyright © 2013, 2009, and 2007, Pearson Education, Inc. 46 Response outcome of interest: Lung Cancer and Smoking  The cases have lung cancer  The controls did not have lung cancer The two groups were compared on the explanatory variable smoker/nonsmoker. (The retrospective aspect refers to studying whether subjects had been smokers in the past.) Example: Case-Control Study

47. Copyright © 2013, 2009, and 2007, Pearson Education, Inc. 47 Nurses’ Health Study:  Began in 1976 with 121,700 female nurses aged 30 to 55; questionnaires are filled out every two years.  Purpose was to explore the relationships among diet, hormonal factors, smoking habits and exercise habits and the risk of coronary heart disease, pulmonary disease and stroke.  Nurses are followed into the future to determine whether they eventually develop an outcome such as lung cancer and whether certain explanatory variables are associated with it. Example: Prospective Study

48. Copyright © 2013, 2009, and 2007, Pearson Education, Inc. 48  A Multifactor experiment uses a single experiment to analyze the effects of two or more explanatory variables on the response.  Categorical explanatory variables in an experiment are often called factors. We are often able to learn more from a multifactor experiment than from separate one-factor experiments since the response may vary for different factor combinations. Multifactor Experiments

49. Copyright © 2013, 2009, and 2007, Pearson Education, Inc. 49 Example: Multifactor Experiment Antidepressants for Quitting Smoking Figure 4.3 An Experiment Can Use Two (or More) Factors at Once. The treatments are the combinations of categories of the factors, as numbered in the boxes.

50. Copyright © 2013, 2009, and 2007, Pearson Education, Inc. 50 Example: Multifactor Experiment  Why use both factors at once in an experiment?  Why not do one experiment about bupropion and a separate experiment about the nicotine patch? The reason is that we can learn more from a two-factor experiment. For instance, the combination of using both a nicotine patch and bupropion may be more effective than using either method alone. Antidepressants for Quitting Smoking

51. Copyright © 2013, 2009, and 2007, Pearson Education, Inc. 51 In a matched pairs design, the subjects receiving the two treatments are somehow matched (same person, husband/wife, two plots in the same field, etc.) In a crossover design, the same individual is used for more than one of the treatments.  Randomly Assign the two treatments to the two matched subjects or Randomize the order of applying the treatments in a crossover design  The number of replicates equals the number of pairs  Helps to reduce effects of lurking variables Matched Pairs Design

52. Copyright © 2013, 2009, and 2007, Pearson Education, Inc. 52 A block is a set of experimental units that are matched with respect to one or more characteristics. A Randomized Block Design, RBD, is when the random assignment of experimental units to treatments is carried out separately within each block. To reduce possible bias, treatments are usually randomly assigned within a block. Blocking in an Experiment

53. Copyright © 2013, 2009, and 2007, Pearson Education, Inc. 53 Compare an oral drug and a placebo for treating migraine headaches. For the entire sample, we would compare the percentage of subjects who have pain relief with the drug to the percentage who have pain relief with the placebo. The two observations for a particular subject are called a matched pair because they both come from the same person. Example: Randomized Block Design

54. Copyright © 2013, 2009, and 2007, Pearson Education, Inc. 54 RBD eliminates variability in the response due to the blocking variable; allows for better comparisons to be made among the treatments of interest. A matched pairs design is a special case of a RBD with two observations in each block. Randomized Block Design