KNR 497 Exptl Design Slide 1 Experimental Design Ch. 8 – Let’s not kid ourselves, this is going to hurt

KNR 497 Exptl design Slide 2 Experimental Design  How on Earth can you ensure that 2 groups of different people are equal (in all respects, not just on the measure of choice) at the beginning of an experiment?  You can’t  But you can make it more probable (and to experimenters, good enough) Remember though, even if you achieve this, groups can still grow different after they have been formed

KNR 497 Exptl design Slide 3 Experimental Design  Searching for group equivalence  What we do:  Random assignment  Does it work?  Maybe!  Sample size, power &c.

KNR 497 Exptl design Slide 4 Experimental Design  If random assignment is the solution, and increased internal validity is the benefit, is there a cost?  Undoubtedly  Sample size big enough?  Control of social threats, & mortality  Its unreal, so improved internal validity comes at the cost of external validity

KNR 497 Exptl design Slide 5 Experimental Design  2-group experimental designs Two-group, post-test only randomized experimental design

KNR 497 Exptl design Slide 6 Experimental Design  More on probabilistic equivalence  Random assignment will distribute folk to groups such that their scores on any measure will be distributed randomly (duh)…this means they will probably be different, but that it is statistically improbable that this will be a significant difference

KNR 497 Exptl design Slide 7 Experimental Design  More on probabilistic equivalence

KNR 497 Exptl design Slide 8 Experimental Design Random selection  Random assignment External validity control Internal validity control

KNR 497 Exptl design Slide 9 Experimental Design  Classifying experimental designs  Signal enhancing vs. noise reducing  The signal vs. noise idea: Strong treatment enhances signal Good measurement reduces noise

KNR 497 Exptl design Slide 10 Experimental Design  Classifying experimental designs  Signal enhancing vs. noise reducing  Designs differ in their strengths ~  Factorial designs focus on isolating aspects or combinations of treatments that seem to affect the measurement most (signal enhancer)  Covariance/blocking designs focus on lessening the effects of known sources of noise (noise reducers)

KNR 497 Exptl design Slide 11 Experimental Design  Factorial designs  Imagine an educational program…  You are interested in (IV’s)  Time of instruction (1 hour vs. 4 hr)  Setting (in-class or pulled out of class)  You measure via study scores (DV) Note – we are now dealing with 2 independent variables for the first time

KNR 497 Exptl design Slide 12 Experimental Design: Factorial

KNR 497 Exptl design Slide 13 Experimental Design: Factorial

KNR 497 Exptl design Slide 14 Experimental Design: Factorial

KNR 497 Exptl design Slide 15 Experimental Design: Factorial

KNR 497 Exptl design Slide 16 Experimental Design: Factorial

KNR 497 Exptl design Slide 17 Experimental Design: Factorial

KNR 497 Exptl design Slide 18 Experimental Design: Factorial

KNR 497 Exptl design Slide 19 Experimental Design: Factorial  A silly example - The marshmallow peeps study  Factor 1: Alcohol (presence/absence)  Factor 2: Smoking (yes/no)

KNR 497 Exptl design Slide 20 Experimental Design: Factorial  Does alcohol have an effect?  Imbibed liberally  Moderate headache  Nausea  No permanent damage

KNR 497 Exptl design Slide 21 Experimental Design: Factorial Does tobacco have an effect? No marketing to young chicks Peep grabs a ciggie… …lights up… …begins smoking… …& continues ‘til satiated It can give up any time it wants to…no effect

KNR 497 Exptl design Slide 22 Experimental Design: Factorial  So, alcohol & nicotine are benign?  Wait..what if you combined them?  Sum of the parts?  More than the sum of the parts?

KNR 497 Exptl design Slide 23 Experimental Design: Factorial Is there an interaction? Combine the elements Faint flame…blackening …smell of caramel… Metamorphosis “ball of charred goo…” “less sweet” “crunchier” “gross”

KNR 497 Exptl design Slide 24 Experimental Design: Factorial  Variations – i. 2 x 3

KNR 497 Exptl design Slide 25 Experimental Design: Factorial  Variations – i. 2 x 3

KNR 497 Exptl design Slide 26 Experimental Design: Factorial  Variations – ii. 2 x 2 x 3 (3 factor)

KNR 497 Exptl design Slide 27 Experimental Design: Factorial  Variations – iii. 2 x 3 + control

KNR 497 Exptl design Slide 28 Experimental Design: Blocking  Reducing noise – Randomized block designs  Key point – unexplained variation in a sample reduces power  The solution is to reduce the variation within the sample by splitting the sample up  You split across some factor that you know causes the sample to differ with respect to the measure of interest (making multiple blocks)  You do not include this as a factor in the experiment, because it is not of interest  Each block will have less variability on the measure, and therefore more power

KNR 497 Exptl design Slide 29 Experimental Design: Blocking  Reducing noise – Randomized block designs Here is the design notation for what was described on the last slide

KNR 497 Exptl design Slide 30 Experimental Design: Blocking  Reducing noise – Randomized block designs “+’s” show scores for all treatment group members (average of all “+” gives treatment group score – average on x-axis is for pretest, and on y-axis is for posttest “o’s” show scores for all control group members (average of all “o” gives control group score – average on x-axis is for pretest, and on y-axis is for posttest

KNR 497 Exptl design Slide 31 Experimental Design: Blocking  Reducing noise – Randomized block designs Note that, regardless of the block, the spread of scores on the post-test is less within the block than across the entire measure

KNR 497 Exptl design Slide 32 Experimental Design: Covariates  Reducing noise – Covariance designs  Design can vary, but basic is this –  Lingo – “controlling for”, “removing the effect of”  Both terms imply use of covariates

KNR 497 Exptl design Slide 33 Experimental Design: Covariates  Reducing noise – Covariance designs

KNR 497 Exptl design Slide 34 Experimental Design: Hybrids  Solomon 4 group To examine & control testing effects in pre-post arrangements

KNR 497 Exptl design Slide 35 Experimental Design: Hybrids  Switched replication design To examine & control social interaction threats

KNR 497 Exptl design Slide 36 Experimental Design: Hybrids  Reducing social interaction threats  (other than via switched replication)  Blind & double blind set ups  Placebos  Isolation of groups