The Nature of Statistics: Experimental Design If your experiment needs statistics, you ought to have done a better experiment. Ernest Rutherford (1871-1937) The Nature of Statistics: Experimental Design

Observation vs. Experimentation Observational Studies: observe and reveal associations. Designed Experiments: impose treatments and controls; can help establish cause-effect relationships. Many times in a statistical study, we want to investigate whether a relationship exists between 2 variables. E.g. Smoking and lung cancer. Fitness and resting heart rate.   2 types of procedures for studies like these: Observational Study: researchers simply observe characteristics and take measurements. . Observational studies can reveal only association. E.g. relationship between height and weight. Designed Experiments: researchers impose treatments and controls, and then observe characteristics and take measurements. A designed experiment is a created situation. Designed experiments can help establish causation. E.g. aspirin and the reduction in risk of fatal heart attacks.

Experimentation Deliberately imposes some treatment on the experimental units or subjects in order to observe a response. Intent of most experiments is to study the effect of changes in some variables by other variables. Sample Survey: In a sample survey, we only observe, and measure characteristics of interest. There is no attempt to manipulate or change the units being studied.    Probably the most famous designed experiments were those carried out on the polio vaccine. Experiment: In an experiment, we do something to the units (a treatment is applied), and then proceed to observe its effects on the units.

Experiment A treatment is applied to part of a population and responses are observed. Example: An experiment was performed in which diabetics took cinnamon extract daily while a control group took none. After 40 days, the diabetics who had the cinnamon reduced their risk of heart disease while the control group experienced no change. (Source: Diabetes Care) Larson/Farber 4th ed. 5

Terms used in Experimentation Experimental Units: the individuals or items on which the experiment is performed (subjects). Block: a group of subjects that are similar. Response Variable: the variable of interest in the experiment. The variable that describes the outcome. That which will be measured. (Also called the dependent variable.) Factor(s): a variable or variables whose effect on the response variable is of interest in the experiment. (Also called the explanatory or independent variable.) Experimental Units: the individuals or items on which the experiment is performed. Another term commonly used in place of experimental units is subjects. E.g. Each cactus used in the study.   Response Variable: the variable of interest to the researcher. That which will be measured. E.g. Weight gain of the cacti. Also called the dependent variable.  Dependent Variable: Is the variable whose changes we wish to observe at the end of the experiment. The dependent variable is sometimes referred to as the response variable. Note: There can be more than one dependent variable in a given experiment. Factor(s): a variable(s) whose effect on the response variable is of interest in the experiment. E.g. Hydrophilic polymer and irrigation regime. There are 2 factors. Also called the independent variable. Independent Variable: Is the variable whose effect on the dependent variable we wish to observe. The independent variable is also referred to as the factor. Note: There can be more than one independent variable (factor) in a given experiment.

Terminology (cont’d) Level(s): the different categories of the factor(s), or possible values of the factor(s). Treatments: each experimental condition. Combinations of the factor(s). Treatment Group: the group receiving the specified treatment. Control Group: the group receiving the placebo, or no treatment. Level(s): the different categories of the factor(s), or possible values of a factor(s). E.g. The hydrophilic polymer has 2 levels (with and without). The irrigation regime has 5 levels (none, light, medium, heavy, and very heavy)   Treatment(s): each experimental condition. E.g. Each treatment is a combination of hydrophilic polymer and irrigation regime. There are 10 treatments. Treatment Group: the group receiving the specified treatment. E.g. The group of cacti receiving the treatments. Control Group: the group receiving the placebo. E.g. Those cacti receiving no treatments. Note: There are times when a control group is not possible.

3 Key Elements (or Basic Principles) of Experimental Design Control: attempt to control factors not of interest to the study, but known to affect variation. Randomization: allow each unit in the experiment an equal chance of receiving any particular treatment. Replication: sufficient number of subjects to ensure that randomization creates groups that resemble each other and allow for repetition. 3 Basic Principles of Experimental Design:   Control: any factors that are not of interest to the investigator, but are known to contribute significantly to variation must be controlled. E.g. Placebo effect. Subjects respond to the idea of a specific treatment rather than to the treatment itself. The control group is usually that group which receives no actual treatment. Randomization: allows each unit in the experiment the same chances of being selected and receiving any particular treatment. This avoids unintentional selection bias . Replication: choosing a large enough number of subjects to ensure that the randomization creates groups that closely resemble each other. This increases the chances of detecting differences when such differences actually exist. Random Allocation: Is a fair (unbiased) way of assigning units to experimental groups (just like Simple Random Sampling is a fair (unbiased) way of selecting units from a population).

Key Elements of Experimental Design: Control Control for effects other than the one being measured. Confounding variables Occurs when an experimenter cannot tell the difference between the effects of different factors on a variable. Example: A coffee shop owner remodels her shop at the same time a nearby mall has its grand opening. If business at the coffee shop increases, it cannot be determined whether it is because of the remodeling or the new mall. Larson/Farber 4th ed. 9

Key Elements of Experimental Design: Control Placebo effect A subject reacts favorably to a placebo when in fact he or she has been given no medical treatment at all. Blinding is a technique where the subject does not know whether he or she is receiving a treatment or a placebo. Double-blind experiment neither the subject nor the experimenter knows if the subject is receiving a treatment or a placebo. Larson/Farber 4th ed. 10

Key Elements of Experimental Design: Randomization Randomization is a process of randomly assigning subjects to different treatment groups. Completely randomized design Subjects are assigned to different treatment groups through random selection. Randomized block design Divide subjects with similar characteristics into blocks, and then within each block, randomly assign subjects to treatment groups. Larson/Farber 4th ed. 11

Experimental Design Examples Completely Randomized Design: all experimental units are assigned randomly among all treatments. Golf ball brands and driving distance: Which brand of golf ball goes the furthest? A group of 50 randomly selected golfers are randomly assigned to test five brands of golf balls.

Completely Randomized Design for golf-ball experiment For Example: Suppose we want to compare the driving distances for five different brands of golf balls. We have 40 golfers willing to help out with the experiment. The 40 golfers are randomly assigned (8 each) to the five different brands of golf ball. This is an example of a completely randomized design, and is used in many types of experiments. However, in the case of this experiment, we could do better…

Randomized Block Design: experimental units are assigned randomly among all treatments within blocks. If the experimenter wishes to control for gender of the golfer, here perhaps assuming that the distances a golf ball is hit is affected by gender, how might this difference be controlled for? A randomized block design would be a better choice for this experiment. Since driving distance is affected by gender, it would be better to use a randomized block design, with blocking by gender. We could use 40 golfers (20 female and 20 male) Divide the 20 females into 5 groups of 4 each, and assign each group of females to drive a different brand of golf ball. Do the same with the group of males. By blocking, we can isolate and remove the variation in driving distances between men and women, and therefore make it easier to detect driving distances among the 5 brands of golf ball, if such differences exist. Why use randomized block? A randomized block design helps to enable the researcher to isolate differences due to variables that are not of interest in the experiment, but whose existence may quite possibly effect the outcome of the experiment. These variables are called blocking variables.

Randomized Block Design for golf-ball experiment

Key Elements of Experimental Design: Randomization Randomized block design An experimenter testing the effects of a new weight loss drink may first divide the subjects into age categories. Then within each age group, randomly assign subjects to either the treatment group or control group. Larson/Farber 4th ed. 16

Key Elements of Experimental Design: Replication Replication is the repetition of an experiment using a large group of subjects. To test a vaccine against a strain of influenza, 10,000 people are given the vaccine and another 10,000 people are given a placebo. Because of the sample size, the effectiveness of the vaccine would most likely be observed. Larson/Farber 4th ed. 17

Example: Experimental Design A company wants to test the effectiveness of a new gum developed to help people quit smoking. Identify a potential problem with the given experimental design and suggest a way to improve it. The company identifies one thousand adults who are heavy smokers. The subjects are divided into blocks according to gender. After two months, the female group has a significant number of subjects who have quit smoking. Larson/Farber 4th ed. 18

Solution: Experimental Design Problem: The groups are not similar. The new gum may have a greater effect on women than men, or vice versa. Correction: The subjects can be divided into blocks according to gender, but then within each block, they must be randomly assigned to be in the treatment group or the control group. Larson/Farber 4th ed. 19

Fabric Durability: A fabric researcher is studying the durability of a fabric under repeated washings. Because the durability may depend on the water temperature and the type of cleansing agent used, the researcher decides to investigate the effect of these factors on durability. The water temperature used in the experiment will be: hot (145 degrees F), warm (100 degrees F), and cold (50 degrees F). The cleansing agents used will be regular Tide, regular Cheer, and Ivory Liquid. Each piece of fabric used, will be washed 50 times in a home automatic washer with a specific combination of water and cleansing agent. The strength of the fabric will then be tested by a machine which forces a steel ball through the fabric and records the fabric’s resistance to breaking.

Fabric Durability: Identify the components of the experiment Identify what is being studied. What are the units? What is the Response Variable? What is/are the Factor(s)? How many levels per Factor? How many Treatments? Describe one Treatment Group. How have Control, Randomization and Replication been addressed? If not stated, how could they be addressed?

Fabric Durability: Detergent type Temperature Tide Cheer Ivory Liq. Trtmt. 1 Tide/145 Trtmt. 4 Cheer/145 Trtmt. 7 Ivory/145 145 deg. Trtmt. 2 Tide/100 Trtmt. 5 Cheer/100 Trtmt. 8 Ivory/100 100 deg. Now let’s apply these terms to an example (EXPERIMENT HANDOUT): A fabric researcher is studying the durability of a fabric under repeated washings. Because the durability may depend on the water temperature and the type of cleansing agent used, the researcher decides to investigate the effect of these factors on durability. The water temperatures used in the experiment will be: hot (145 degrees F.), warm (100 degrees F.), and cold (50 degrees F.). The cleansing agents used will be: regular Tide, regular Cheer, and Ivory Liquid. Each piece of fabric used, will be washed 50 times in a home automatic washer with a specific combination of water temperature and cleansing agent. The strength of the fabric will then be tested by a machine which forces a steel ball through the fabric and records the fabric’s resistance to breaking. EXPERIMENTAL UNITS: fabric pieces. RESPONSE VARIABLE: resistance to breaking. FACTORS: water temperature and type of detergent. LEVELS: water temperature: 3. 145 deg., 100 deg., and 50 deg.. Detergent: 3. Regular Tide, regular Cheer, and Ivory Liquid. TREATMENTS: 9 No control group here. Let’s take a look at a “drawing” of this experiment: Trtmt. 3 Tide/50 Trtmt. 6 Cheer/50 Trtmt 9 Ivory/50 50 deg.  

Anatomy of an Experiment: Terms and their Relationships The Experiment: The Munchy Cookie Company is famous for its Chewy Mango cookies. In order to increase production the company wants to determine if it can reduce cooking time and increase temperature and still get the same quality cookie (traditionally baked for eight minutes at 350 degrees). The conditions under consideration involve temperatures of 350 and 450 degrees and cooking times of six and eight minutes. Chewy Mango cookies can be bent to a 300 degree outside angle before breaking. If the change in cooking conditions replicates the original conditions, the company will consider this new cooking process. Anatomy of an Experiment: Terms and their Relationships EXPERIMENTAL UNITS: Experimental Units are the items (here cookies) or individuals (also called Subjects) upon which the experiment is performed. Experimental Units are assigned to Control and Experimental Conditions at random. Control Group: One of the treatment conditions may represent a Control Group to which others are compared. Factors applied to samples FACTOR (aka INDEPENDENT VARIABLE or EXPLANATORY VARIABLE): A FACTOR is a condition applied to the Experimental Units, such as cooking time or cooking temperature. Control Group: Here, the current baking process (8 min; 350o) Random Sample of 250 dough balls to each condition 350 Degrees 8 minutes Take Measurements 350 Degrees 6 minutes Compare Cooking Time/Temp. results and state Conclusions 1000 Chewy Mango cookie dough balls 450 Degrees 8 minutes TREATMENT A TREATMENT is comprised of Levels from one or more Factors (Independent Variables). If there is only one Factor, such as Temperature, then the TREATMENTS are the Levels of the single Factor, Temperature. If there is more than one Factor, then the TREATMENTS represent the combination of Levels from the Factors (here a combination of Temperature and Time). HELP! We’re getting a bit crispy Six minutes represents a Level of the Time Factor 450 Degrees 6 minutes A Treatment representing one Level from each of two Factors LEVEL: LEVELS represent the different values of a Factor that are applied to the Experimental Units, such as cooking at 350 and 450 degrees. There is always a minimum of two Levels - one which may represent the existing conditions and thereby represent a Control Level (Control Group) for a Factor and another which represents an Experimental Level. RESPONSE VARIABLE (aka DEPENDENT VARIABLE or OUTCOME VARIABLE): A RESPONSE VARIABLE is the characteristic of the experiment to be measured or observed, such as the consistency of a cookie.

End of Slides