2. KNR 497 Exptl Design Slide 1 Experimental Design Ch. 8 – Let’s not kid ourselves, this is going to hurt

3. KNR 497 Exptl design Slide 2 Experimental Design  How on Earth can you ensure that 2 groups of different people are equal (in all respects, not just on the measure of choice) at the beginning of an experiment?  You can’t  But you can make it more probable (and to experimenters, good enough) Remember though, even if you achieve this, groups can still grow different after they have been formed

4. KNR 497 Exptl design Slide 3 Experimental Design  Searching for group equivalence  What we do:  Random assignment  Does it work?  Maybe!  Sample size, power &c.

5. KNR 497 Exptl design Slide 4 Experimental Design  If random assignment is the solution, and increased internal validity is the benefit, is there a cost?  Undoubtedly  Sample size big enough?  Control of social threats, & mortality  Its unreal, so improved internal validity comes at the cost of external validity

6. KNR 497 Exptl design Slide 5 Experimental Design  2-group experimental designs Two-group, post-test only randomized experimental design

7. KNR 497 Exptl design Slide 6 Experimental Design  More on probabilistic equivalence  Random assignment will distribute folk to groups such that their scores on any measure will be distributed randomly (duh)…this means they will probably be different, but that it is statistically improbable that this will be a significant difference

8. KNR 497 Exptl design Slide 7 Experimental Design  More on probabilistic equivalence

9. KNR 497 Exptl design Slide 8 Experimental Design Random selection  Random assignment External validity control Internal validity control

10. KNR 497 Exptl design Slide 9 Experimental Design  Classifying experimental designs  Signal enhancing vs. noise reducing  The signal vs. noise idea: Strong treatment enhances signal Good measurement reduces noise

11. KNR 497 Exptl design Slide 10 Experimental Design  Classifying experimental designs  Signal enhancing vs. noise reducing  Designs differ in their strengths ~  Factorial designs focus on isolating aspects or combinations of treatments that seem to affect the measurement most (signal enhancer)  Covariance/blocking designs focus on lessening the effects of known sources of noise (noise reducers)

12. KNR 497 Exptl design Slide 11 Experimental Design  Factorial designs  Imagine an educational program…  You are interested in (IV’s)  Time of instruction (1 hour vs. 4 hr)  Setting (in-class or pulled out of class)  You measure via study scores (DV) Note – we are now dealing with 2 independent variables for the first time

13. KNR 497 Exptl design Slide 12 Experimental Design: Factorial

14. KNR 497 Exptl design Slide 13 Experimental Design: Factorial

15. KNR 497 Exptl design Slide 14 Experimental Design: Factorial

16. KNR 497 Exptl design Slide 15 Experimental Design: Factorial

17. KNR 497 Exptl design Slide 16 Experimental Design: Factorial

18. KNR 497 Exptl design Slide 17 Experimental Design: Factorial

19. KNR 497 Exptl design Slide 18 Experimental Design: Factorial

20. KNR 497 Exptl design Slide 19 Experimental Design: Factorial  A silly example - The marshmallow peeps study  Factor 1: Alcohol (presence/absence)  Factor 2: Smoking (yes/no)

21. KNR 497 Exptl design Slide 20 Experimental Design: Factorial  Does alcohol have an effect?  Imbibed liberally  Moderate headache  Nausea  No permanent damage

22. KNR 497 Exptl design Slide 21 Experimental Design: Factorial Does tobacco have an effect? No marketing to young chicks Peep grabs a ciggie… …lights up… …begins smoking… …& continues ‘til satiated It can give up any time it wants to…no effect

23. KNR 497 Exptl design Slide 22 Experimental Design: Factorial  So, alcohol & nicotine are benign?  Wait..what if you combined them?  Sum of the parts?  More than the sum of the parts?

24. KNR 497 Exptl design Slide 23 Experimental Design: Factorial Is there an interaction? Combine the elements Faint flame…blackening …smell of caramel… Metamorphosis “ball of charred goo…” “less sweet” “crunchier” “gross”

25. KNR 497 Exptl design Slide 24 Experimental Design: Factorial  Variations – i. 2 x 3

26. KNR 497 Exptl design Slide 25 Experimental Design: Factorial  Variations – i. 2 x 3

27. KNR 497 Exptl design Slide 26 Experimental Design: Factorial  Variations – ii. 2 x 2 x 3 (3 factor)

28. KNR 497 Exptl design Slide 27 Experimental Design: Covariates  Reducing noise – Covariance designs  Design can vary, but basic is this –  Lingo – “controlling for”, “removing the effect of”  Both terms imply use of covariates

29. KNR 497 Exptl design Slide 28 Experimental Design: Covariates  Reducing noise – Covariance  Imagine you do a survey of the number of churches in a town, and look at crime rate in those towns also  To your surprise, you find a positive relationship  Higher crime rates are found in towns with more churches  What does it mean?  Is there another factor causing noise in the data?

30. KNR 497 Exptl design Slide 29 Experimental Design: Covariates  Reducing noise – Covariance  How about population?  So, rerun the data after “controlling for” population size  Effectively asking, for towns with near equal populations, does the +ve relationship between crime rate and #churches persist?  You would find the relationship reverses  The original finding was an artifact of the uncontrolled effect of population size

31. KNR 497 Exptl design Slide 30 Experimental Design: Hybrids  Solomon 4 group To examine & control testing effects in pre-post arrangements

32. KNR 497 Exptl design Slide 31 Experimental Design: Hybrids  Switched replication design To examine & control social interaction threats

33. KNR 497 Exptl design Slide 32 Experimental Design: Hybrids  Reducing social interaction threats  (other than via switched replication)  Blind & double blind set ups  Placebos  Isolation of groups

34. Quasi-Experimental Design Ch. 8 – Non-random assignment

35. KNR 497 Quasi-Exptl design Slide 34 Quasi-Experimental Design  Basic issue throughout is lack of random assignment…must look elsewhere for assurances of group equality (and therefore internal validity) The non-equivalent groups design (NEGD)

36. KNR 497 Quasi-Exptl design Slide 35 The NEGD  Possible outcomes…1.   ?

37. KNR 497 Quasi-Exptl design Slide 36 The NEGD  Possible outcomes…2.

38. KNR 497 Quasi-Exptl design Slide 37 The NEGD  Possible outcomes…3.

39. KNR 497 Quasi-Exptl design Slide 38 The NEGD  Possible outcomes…4.

40. KNR 497 Quasi-Exptl design Slide 39 The NEGD  Possible outcomes…5.