1. Experiment Design 5: Variables & Levels Martin, Ch 8, 9,10

2. Recap u Different kinds of variables –Independent, dependent, confounding, control, and random u Different kinds of validity –Internal, construct, statistical, external –Each associated with a question u Randomization –Random sampling: generalization –Random assignment: causation

3. Picking a design u Choosing how to assign participants to levels of an independent variable –Between vs. within u Choosing how many levels of an independent variable u Choosing how many independent variables

4. Between vs. Within designs u Condition 1: –Fred –Ginger –Mary u Condition 2: –Ed –Mabel –George u Condition 1: –Fred –Ginger –Mary u Condition 2: –Fred –Ginger –Mary 586586 697697 586586 697697

5. Within vs. Between Subjects u Cost –Between: More participants –Within: More time per participant u Confounding variables –Between: Group differences possible Use randomization, many subjects, matching –Within: Order effects possible Use counterbalancing

6. Transfer effects (order effects) u Definition: –When taking part in earlier trials changes performance in the later trials u Types –Learning –Fatigue –Range or context effect u Problem: –Makes within-subjects designs difficult to interpret

7. Counterbalancing u Adjust condition order to unconfound transfer effects with condition effects –A,B,C –A,C,B –B,A,C –B,C,A –C,A,B –C,B,A

8. Counter-balancing either within- or between- subjects u Between: –Joe: A,B –Mary: B,A u Within: –Joe: ABBA –Mary: ABBA

9. Things to worry about in counter-balancing u If within-subjects counter-balancing: –Linear transfer effects? Is the transfer from the 1st position to the 2nd position the same as the transfer from 2nd to 3rd position? –E.g., sometimes most learning happens in 1st trials u Always worry about asymmetrical transfer –Does A influence B more than B influences A?

10. Asymmetrical transfer Time 1 Time 2 % trigrams remembered Noisy Quiet u Effect of noise depends on order u People stick with the strategy they pick first –Or mix strategies Quiet Noisy

11. Partial counterbalancing: Latin Square u Every condition appears in every position equally: –Joe: ABC –Mary:CAB –John:BCA

12. Matching u Try to reduce between-group differences u E.g., rank hearing as Good, Fair, Poor u Unmatched, could get –Noisy: Poor1, Poor2, Fair1 –Quiet: Good1, Good2, Fair2 u Matched, get: –Noisy: Poor1, Fair2, Good1 –Quiet: Poor2, Fair1, Good2

13. Matching u Match variable(s) and DV’s should be strongly correlated u Caveat: Match test should not affect DV –e.g., use existing match variable (SAT-M) u Note: Within-subjects designs “match” automatically

14. Number of levels u How many different groups or conditions that change just one independent variable –Two: Experimental vs. control Massed vs. Distributed practice –More: Drug vs. Placebo vs. No pill # of times an item is studied: 1,2,4,8, or 16 times

15. Inter- and extra-polating ? ? insideoutside

16. Floor & Ceiling Effects

17. Single Variable vs. Multiple Variables u Single Variable: –Only one independent variable –Cannot look at interactions u Multiple Variables: –Two or more independent variables –If use factorial design, can look at interactions –Can require a lot of participants (between) or time (within)

18. Interactions Author Editor % errors detected 100 0 Proofreader u Who finds more errors, author or editor? u How to spot the interaction graphically? PrepLevel Manuscript Draft

19. Interactions u Two independent variables interact when the effect of one depends on the level of the other u Independent vs. Control vs. Random –What if PrepLevel had been a control variable? –What if PrepLevel had been a random variable? –Make it an independent variable if there is reason to believe its effect might depend

20. Factorial design u Do all combinations of factors (cells) –E.g., Language learning u A factor can be within or between

21. Converging Operations (≠converging series) u Using more than one method to test the same hypothesis –E.g., using experimental and observational methods –E.g., using cross-sectional and longitudinal designs

22. Baseline procedure u Example 1: Clinical –No drug, drug, no drug, drug,... u Example 2: Education –Regular class, new format, regular class, new format,..