1. Experimental Design 1 http://thefifthlevel.blogspot.com/2011/05/prologue-genesis.html

2. Experimental Design Strongest design with respect to internal validity 2

3. If X then Y and If not X, then not Y or If the program is given, then the outcome occurs and If the program is not given, then the outcome does not occur 3

4. Dilemma 2 identical groups 2 identical contexts Same time …. 4 similarity

5. Course of Action Randomly assign people from a pool to the 2 groups –  probabilistically equivalent One group gets the treatment and the other does not 5

6. Random Selection and Assignment Random selection is how you draw the sample of people for your study from a population. Random assignment is how you assign the sample that you draw to different groups or treatments in your study.

7. Probabilistic Equivalence Means that we know perfectly the odds that we will find a difference between two groups. When we randomly assign to groups, we can calculate the chance that the two groups will differ just because of the random assignment. 7

8. External validity Experiments are difficult to carry out  artificial situation  high internal validity Limited generalization to real contexts –> limited external validity 8 ?

9. Two-Group Experimental Design Simplest form: two-group posttest-only randomized experiment No pretest required Test for differences: t-test or ANOVA 9

10. Advantages Strong against single-group threats and multi- group threats (except selection-mortality) Strong against selection testing and selection- instrumentation 10

11. Classifying Experimental Designs Two components: signal and noise signal-enhancing experimental design (factorial design) Noise-reducing experimental design (covariance designs or blocking designs) 11

12. Factorial Designs A factor is a major independent variable – Time and setting A level is a subdivision of a factor. – Time (1h/4h), setting (pull-out/in-class) 2 x 2 factorial design 12

13. Factorial Design X 11 = 1h and in-class X 12 = 1h and pull-out X 21 = 4h and in-class X 22 = 4h and pull-out 13

14. The Null Outcome The null case is a situation where both treatments have no effect. 14

15. The Main Effects A main effect is an outcome that is a consistent difference between levels of a factor. 15

16. Main Effects 16

17. Main Effects 17

18. Interaction Effects An interaction factor exists when differences on one factor depending on the level of the other factor. 18

19. How do you know if there is an interaction in a factorial design? Statistical analysis When it can be talked about one factor without mentioning the other factor In graphs of group means – the lines are not parallel 19

20. Interaction Effects 20

21. Interaction Effects 21

22. Factorial Design Variations 2 x 3 Example Factor 1: Treatment – psychotherapy – behavior modification Factor 2: Setting – inpatient – day treatment – outpatient 22

23. Main Effects 23

24. Main Effects 24

25. Interaction Effect 25

26. Interaction Effect 26

27. Factorial Design Variations A Three-Factor Example (2 x 2 x 3) Factor 1: Dosage – 100 mg – 300 mg Factor 2: Treatment – Psychotherapy – Behavior modification Factor 3: Setting – Inpatient – Day treatment – Outpatient 27

28. 2 x 2 x 3 Design 28

29. Incomplete Factorial Design Common use is to allow for a control or placebo group that receives no treatment 29

30. Randomized Block Design Stratified random sampling To reduce noise or variance in the data Division into homogeneous subgroups Treatment implemented to each subgroup Variability within each block is less than the variability of the entire sample or each block is more homogenous than the entire group 30

31. Randomized Block Design Stundents are a homogenous group with exception of semester 31 freshman sophomore junior senior

32. How blocking reduces noise? 32

33. Covariance Designs (ANCOVA) Pretest-posttest randomized design Pre-program measure = covariate Covary it with the outcome variable Covariates are the variables you adjust for – Effect is going to be removed 33

34. How does a Covariate reduce Noise? 34

35. How does a Covariate reduce Noise? 35

36. How does a Covariate reduce Noise? 36

37. How does a Covariate reduce Noise? 37

38. How does a Covariate reduce Noise? 38

39. Hybrid Experimental Designs Are new strains that are formed by combining features of more established designs. 39

40. The Solomon Four-Group Design Is designed to deal with a certain testing threat 2 groups are pre-tested, 2 are not 2 groups get a treatment, 2 do not 40

41. The Solomon Four-Group Design T = Treatment Group, C = Control Group 41

42. The Solomon Four-Group Design T = Treatment Group, C = Control Group 42

43. Switching Replication Design The implementation of the treatment is repeated or replicated. In the repetition, the two groups switch roles Finally, all participants have received the treatment Reduces social threats 43

44. Switching Replication Design Period 1 – group 1 gets the treatment Period 2 – group 2 gets the treatment 44

45. Switching Replication Design Longterm treatment effect  group 1 improves even though no further treatment was given 45