Experimental Design Research vs Experiment

Research RESEARCH DESIGN (a plan for research) A careful search An effort to obtain new knowledge in order to answer a question or to solve a problem A protocol for measuring the values of a set of variables (response variables) under a set of condition (study condition) RESEARCH DESIGN (a plan for research)

Research Designs Naturalistic observation Case study Correlational Differential Experimental Constraint level

Research design Observation A design in which the levels of all the explanatory variables are determined as part of the observational process Experimental A study in which the investigator selects the levels of at least one factor An investigation in which the investigator applies some treatments to experimental units and then observes the effect of the treatments on the experimental units by measuring one or more response variables An inquiry in which an investigator chooses the levels (values) of input or independent variables and observes the values of the output or dependent variable(s).

Strengths of observation Can be used to generate hypotheses Can be used to negate a proposition Can be used to identify contingent relationships

Limitations of Observation Cannot be used to test hypotheses Poor representative ness Poor replicability Observer bias

Strengths of experimental Causation can be determined (if properly designed) The researcher has considerable control over the variables of interest Can be designed to evaluate multiple independent variables

Limitations of experimental Not ethical in many situations Often more difficult and costly

Design of Experiments Define the objectives of the experiment and the population of interest. Identify all sources of variation. Choose an experimental design and specify the experimental procedure.

Defining the Objectives What questions do you hope to answer as a result of your experiment? To what population do these answers apply? INTERNAL USE FIG. 02s02f04 INTERNAL USE FIG. 02s02f03

Defining the Objectives INTERNAL USE FIG. 02s02a

Identifying Sources of Variation INTERNAL USE FIG. 02s02f05 Input Variable Output Variable

Choosing an Experimental Design

Experimental Design A plan and a structure to test hypotheses in which the analyst controls or manipulates one or more variables Protocol for measuring the values of a set of variable It contains independent and dependent variables 4

What is a statistical experimental design? Determine the levels of independent variables (factors) and the number of experimental units at each combination of these levels according to the experimental goal. What is the output variable? Which (input) factors should we study? What are the levels of these factors? What combinations of these levels should be studied? How should we assign the studied combinations to experimental units?

The Six Steps of Experimental Design Plan the experiment. Design the experiment. Perform the experiment. Analyze the data from the experiment. Confirm the results of the experiment. Evaluate the conclusions of the experiment.

Plan the Experiment Identify the dependent or output variable(s). Translate output variables to measurable quantities. Determine the factors (input or independent variables) that potentially affect the output variables that are to be studied. Identify potential combined actions between factors.

Syllabus Content Week Terminology and basic concept 2 T-test, anova and CRD 3 RCBD and Latin Square 4 Mean comparison 7 Midterm 8 - 9 10 Factorial experiment 11 - 12 Special topic in factorial experiment 13

B – D → 45 – 80 (Normal distribution) Grading system Grade : 0 – 100 A > 80 B – D → 45 – 80 (Normal distribution) E < 45 Grade composition Assignment : 30 67 Mid-term Final Exam 40 Practical Work 33

Treatment/ input/independent variable Terminology Variable a characteristic that varies (e.g., weight, body temperature, bill length, etc.). Treatment/ input/independent variable A condition or set of conditions applied to experimental units in an experiment. The variable that the experimenter either controls or modifies. What you manipulate Single factor ≥ 2 factors 4

Terminology Factors Another name for the independent variables of an experimental design An explanatory variable whose effect on the response is a primary objective of the study A variable upon which the experimenter believes that one or more response variables may depend, and which the experimenter can control An explanatory variable that can take any one of two or more values. The design of the experiment will largely consist of a policy for determining how to set the factors in each experimental trial 4

Terminology Levels or Classifications The subcategories of the independent variable used in the experimental design The different values of a factor Dependent/response/output variable The response to the different levels of the independent variables A characteristic of an experimental unit that is measured after treatment and analyzed to assess the effects of treatments on experimental units

Full Factorial Treatment Design Terminology Treatment Factor A factor whose levels are chosen and controlled by the researcher to understand how one or more response variables change in response to varying levels of the factor Treatment Design The collection of treatments used in an experiment. Full Factorial Treatment Design Treatment design in which the treatments consist of all possible combinations involving one level from each of the treatment factors.

Terminology Experimental unit the unit of the study material in which treatment is applied The smallest unit of the study material sharing a common treatment The physical entity to which a treatment is randomly assigned and independently applied. Observational unit (sampling unit) The smallest unit of the study material for which responses are measured. The unit on which a response variable is measured. There is often a one-to-one correspondence between experimental units and observational units, but that is not always true.

Populations and Samples the entire collection of values for the variable being considered. Sample a subset of the population. Statistically, it is important for the sample to be a random sample.

Parameters vs. Statistics a measure that characterizes a population. Statistic an estimate of a population parameter, based on a sample.

Basic principles Formulate question/goal in advance Comparison/control Replication Randomization Stratification (blocking) Factorial experiment

Comparison/control Good experiments are comparative. Compare BP in mice fed salt water to BP in mice fed plain water. Compare BP in strain A mice fed salt water to BP in strain B mice fed salt water. Ideally, the experimental group is compared to concurrent controls (rather than to historical controls).

Replication Applying a treatment independently to two or more experimental units.

Why replicate? Reduce the effect of uncontrolled variation (i.e. increase precision). Quantify uncertainty.

Randomization Random assignment of treatments to experimental units. Experimental subjects (“units”) should be assigned to treatment groups at random. At random does not mean haphazardly. One needs to explicitly randomize using A computer, or Coins, dice or cards.

Why randomize? Avoid bias. Control the role of chance. For example: the first six mice you grab may have intrinsically higher BP. Control the role of chance. Randomization allows the later use of probability theory, and so gives a solid foundation for statistical analysis.

Stratification (Blocking) Grouping similar experimental units together and assigning different treatments within such groups of experimental units If you anticipate a difference between morning and afternoon measurements: Ensure that within each period, there are equal numbers of subjects in each treatment group. Take account of the difference between periods in your analysis.

Completely randomized design Cage positions Completely randomized design

Randomized block design Cage positions Randomized block design

Randomization and stratification If you can (and want to), fix a variable. e.g., use only 8 week old male mice from a single strain. If you don’t fix a variable, stratify it. e.g., use both 8 week and 12 week old male mice, and stratify with respect to age. If you can neither fix nor stratify a variable, randomize it.

Types of Experimental Designs Simple Designs: Vary one factor at a time Not statistically efficient. Wrong conclusions if the factors have interaction. Not recommended.

Types of Experimental Designs Factorial Experiment: 1. Full Factorial Design: All combinations. Can find the effect of all factors. Too much time and money. May try 2k design first

Types of Experimental Designs 2. Fractional Factorial Designs: Save time and expense. Less information. May not get all interactions. Not a problem if negligible interactions.

Common Mistakes in Experimentation 1. The variation due to experimental error is ignored. 2. Important parameters are not controlled. 3. Effects of different factors are not isolated. 4. Simple one-factor-at-a-time designs are used 5. Interactions are ignored. 6. Too many experiments are conducted. Better: two phases.

Interaction Effect of one factor depends upon the level of the other. Non-interacting Factors Interacting Factors

Full Factorial experiments Suppose we are interested in the effect of both salt water and a high-fat diet on blood pressure. Ideally: look at all 4 treatments in one experiment. Plain water Normal diet Salt water High-fat diet Why? We can learn more. More efficient than doing all single-factor experiments. ×

Interactions

Other points Blinding Internal controls Measurements made by people can be influenced by unconscious biases. Ideally, dissections and measurements should be made without knowledge of the treatment applied. Internal controls It can be useful to use the subjects themselves as their own controls (e.g., consider the response after vs. before treatment). Why? Increased precision.

Other points Representativeness Are the subjects/tissues you are studying really representative of the population you want to study? Ideally, your study material is a random sample from the population of interest.

Distribution of D when  = 0 Significance test Compare the BP of 6 mice fed salt water to 6 mice fed plain water.  = true difference in average BP (the treatment effect). H0:  = 0 (i.e., no effect) Test statistic, D. If |D| > C, reject H0. C chosen so that the chance you reject H0, if H0 is true, is 5% Distribution of D when  = 0

Statistical power Power: The chance that you reject H0 when H0 is false (i.e., you [correctly] conclude that there is a treatment effect when there really is a treatment effect).

Power depends on… The structure of the experiment The method for analyzing the data The size of the true underlying effect The variability in the measurements The chosen significance level () The sample size Note: We usually try to determine the sample size to give a particular power (often 80%).

Effect of sample size 6 per group: 12 per group:

Various effects Desired power   sample size  Stringency of statistical test   sample size  Measurement variability   sample size  Treatment effect   sample size 