Experimental Design Basics

What is an Experiment Process that deliberately imposes a treatment on a group of experimental units/subjects in the interest of observing the response Purpose is to prove a cause-and-effect relationship Treatment – a specific condition that researchers administer to subjects Basic experimental design subjects  treatment  observation Ex: A doctor treats a patient with a skin condition with different creams to see which is most effective.

Treatments Factors Controlled independent variable whose levels are set by the experimenter General type of category of treatment Ex: a doctor treats a patient with a skin condition with different creams to see which is most effective. Levels Amount or magnitude of the treatment Ex: 5mg, 10mg, 15mg of a medication,

An experiment: Manipulates factor levels to create treatments Randomly assigns subjects to these treatment levels Compares the responses of the subject groups across treatment levels

An experiment: Is prospective (identifies subjects in advance and collects data as events unfold) Differs from an observational study, which involves collecting and analyzing data without changing existing conditions (does not impose a treatment) - retrospective (based on historical data) Examples of observational study: documenting amount of time taken to perform a task observing food preferences of animals

Experimental Design Principles The objective of experimental design is to conclude that differences in the dependent variable of an experiment are results of differences in the treatments being considered and can not be reasonably attributed to random error.

Three Principles of Design Control: Reduce variability by controlling sources of variation among experimental units- make conditions as similar as possible for all treatment groups Reduces experimental bias and error Only one experimental variable should be used Randomization: subjects should be randomly divided into groups so each subject has the same chance of receiving treatment Replication: the more subjects that are sampled according to each treatment, the more reliable the conclusions of the experimental design

Three Principles of Design 1. Control: Good experimental design reduced variability by controlling the sources of variation Comparison – must have at least two groups so effect of treatment can be compared with either the effect of a traditional treatment or the effect of no treatment Make conditions as similar as possible for all treatment groups Controlled variables/constants

Three Principles of Design 2. Randomization Objects or individuals are randomly assigned (by chance) to an experimental group Equalizes effects of unknown or uncontrollable sources of variation Most reliable method of creating homogeneous treatment groups without involving any potential biases or judgments Protects against variability/extraneous factors/chance Reduces bias and experimental errors which can skew results

Randomized Design Types Randomized Block Design Used when an experimenter is aware of specific differences among groups of subjects or objects within an experimental group Experimental subjects are first divided into homogeneous blocks before they are randomly assigned to a treatment group Two parallel experiments Reduces unwanted variation Completely Randomized Design Objects or subjects are assigned to groups completely at random Number each subject and use table of random numbers to assign subjects to groups

Randomized Design Types Block Design Flow Chart Completely Randomized Design Flow Chart

Example of Randomized Block Design A researcher is carrying out a study of the effectiveness of four different skin creams for the treatment of a certain skin disease. He has eighty subjects and plans to divide them into 4 treatment groups of twenty subjects each. Using a randomized block design, the subjects are assessed and put in blocks of four according to the severity of skin condition four most severe cases are the first block next four most severe cases are the second block and so on to the twentieth block four members of each block are then randomly assigned, one to each of the four treatment groups

Example of Completely Randomized Design A researcher is carrying out a study of the effectiveness of four different skin creams for the treatment of a certain skin disease. He has eighty subjects. He uses a computer to randomly assign 20 subjects into four different treatment groups.

Example 2 of Completely Randomized Design Suppose we have 4 different diets which we want to compare. The diets are labeled Diet A, Diet B, Diet C, and Diet D. We are interested in how the diets affect the coagulation rates of rabbits. The coagulation rate is the time in seconds that it takes for a cut to stop bleeding. We have 16 rabbits available for the experiment, so we will use 4 on each diet. Catch all the rabbits and label them 1-16. Select four numbers 1-16 at random and put them in a cage to receive Diet A. Then select another four numbers at random and put them in a cage to receive Diet B. Continue until you have four cages with four rabbits each. Each cage receives a different diet

Randomized Design Types Matched Pairs Design Special type of block design Compares only two subjects or groups Each subject gets both treatments and acts as their own control Subjects paired in ways not under study Reduces variability in subjects to insure single experimental variable Ex: cola taste test Each subject compares two colas and picks one Order they taste colas is randomized Students are tested to see if they learn better with music or quiet conditions

Example of Matched Pair Design Reasercher tests to see if students concentrate better while listening to classical music or quiet room Measure time it takes to complete puzzle in each condition Flip coin to determine which condition is used first (music or quiet) If two subjects are used they must be matched as closely as possible on their attributes Due to differences among students Use same person to reduce variability Differences can be attributed to only tested variable

Three Principles of Design 3. Replication Independent repetition of the experiment: repeat entire experiment many times Apply the treatments to a number of subjects (repeats) Large sample size is essential Outcome of an experiment on a single subject is an anecdote, not data Reduces experimental error Increases accuracy of population

Sample Size and Margin of Error

Diagrams of Experiments It’s often helpful to diagram the procedure of an experiment The following diagram emphasizes the random allocation of subjects to treatment groups, the separate treatments applied to these groups, and the ultimate comparison of results:

Components of Experiment: What are you testing? What is a good question for an experiment? One that is testable Hypothesis: Your best thinking about how the change you make might affect another factor Tentative or trail solution to the question Must be testable If……IV………then ……DV……statement

Components of Experiment: Groups Control Group Normal condition/baseline (business as usual) Generally known outcome Does not receive variable being tested for Serves as standard of comparison to experimental group Experimental Group “Not normal” condition Receives experimental variable being tested for Outcome unknown

Components of Experiment: Variables Variables: Conditions of the experiment Independent Variable (IV): cause Manipulated to assess its effect May be administered to subjects by 'level Ex: three levels of treatment 5mg, 10mg, 15mg of medication Dependent Variable (DV): effect Observed or measured Controlled Variables/Constants: factors that remain same in both control and experimental groups Experimental Variable : factor being tested for (always in the experimental group)

Components of Experiment: Data Qualitative Observations: Your perception of what happened during the experiment Involves quality of results Identifies trends in data Do not contain numbers Quantitative Observations: Raw data in the form of numbers Displayed in data tables and graphs

Components of Experiment: Data Discrete Data Categorical Intervals between data have no meaning Use bar graph Continuous Data Associated with measurements involving a standard scale Measurements show a trend or relationship Intervals between data have meaning Use best fit or line graph

Components of Experiment: Data Data Table: Setup

Components of Experiment: Data Tables/Charts : One Variable Experiment Effect of Time on Water Temperature Effect of Fertilizer on Plant Growth

Components of Experiment: Data Tables/Charts: Two Variable Experiment: Effects of a Hydrophilic Polymer and Irrigation Regime on Cactus Weight Gain

Components of Experiment: Data Graphs: pictorial form of collected data Components of Graph

Constructing a Graph

Constructing a Graph

Constructing a Graph

Constructing a Graph

Constructing a Graph

Issues in Experimental Design: Bias Experimental Bias process where the scientists performing the research influence the results, in order to portray a certain outcome. Blinding: method to avoid bias Single blind: Subjects do not know which treatment group they are assigned Evaluators do not know which group subjects have been assigned Double blind: Everyone is blinded Neither evaluators or subjects know treatment groups Placebos: “fake” treatment that looks just like actual treatment Best method to blind subjects from knowing if they are receiving treatment or not (sugar pill)

Issues in Experimental Design Experimental Error mistake in perception, measurement or process Can be due to: Design Observation Recording Calculation Measuring tool limitation Types: Type 1 error: false positive Type 2 error: false negative

Analysis of Results Are the results statistically significant? i.e., are the results due to the treatment or chance? Null hypothesis (H0 ): no statistical observed effect of treatment Results due to chance Alternate Hypothesis (HA or H1): a statistical observed effect of treatment Results due to experimental factor/treatment Reject null hypothesis

Experimental Design Checklist Define the objectives of the experiment (what are you testing for?) What variable (IV/EV) is being used as treatment? What experimental units/subjects will you use? What are your control/experimental groups? Identify and reduce all sources of variation including: treatment factors and their levels, variation within subject groups What equipment will be used? What will you be measuring as result (DV)? Specify the measurements to be made. Determine number of observations that need to be taken. Determine timeline of experiment.