1. PSYC512: Research Methods PSYC512: Research Methods Lecture 13 Brian P. Dyre University of Idaho

2. PSYC512: Research Methods Lecture 13 Outline Review of Lectures 11-12 Review of Lectures 11-12 Causation and reactivity Causation and reactivity Within vs. between subjects designs Within vs. between subjects designs More on research design More on research design Multifactor research – using two or more independent variables Multifactor research – using two or more independent variables An example of multi-factor research An example of multi-factor research Experimentation vs. Quasi-experimentation Experimentation vs. Quasi-experimentation

3. PSYC512: Research Methods Factorial Research Designs Used to assess the effects of 2 or more independent variables (factors) on your dependent variable Used to assess the effects of 2 or more independent variables (factors) on your dependent variable Using multiple IVs in one experiment is Using multiple IVs in one experiment is more economical (particularly for within-Ss) more economical (particularly for within-Ss) provides more information provides more information Main effects of each IV (separate effects of each IV) Main effects of each IV (separate effects of each IV) Interaction between the IVs (synergism): effect of one variable changes across the levels of the other variable Interaction between the IVs (synergism): effect of one variable changes across the levels of the other variable

4. PSYC512: Research Methods Factorial Research Designs: Possible Outcomes of a 2 x2 Design DV Factor A A1A2 B1, B2 DV Factor A A1A2 B1, B2 DV Factor A A1A2 B2 No effects Main Effect A Only Main Effect B Only No effects Main Effect A Only Main Effect B Only B1

5. PSYC512: Research Methods DV Factor A A1A2 DV Factor A A1A2 B1 B2 Main Effects for A and B, No Interaction Factorial Research Designs: Possible Outcomes of a 2 x2 Design Main Effects for A and B, Interaction (ordinal) B1 B2 DV Factor A A1A2 B1 B2 DV Factor A A1A2 B1 B2 Main Effect for A only, Interaction (disordinal) Main Effect for B only, Interaction (ordinal) IMPORTANT: always interpret the highest order effect or interaction

6. PSYC512: Research Methods Example Factorial Experiment: Perceiving Heading An experiment on perceiving the heading direction during steering control vs. riding and pointing An experiment on perceiving the heading direction during steering control vs. riding and pointing

7. PSYC512: Research Methods Example Factorial Experiment: Perceiving Heading IV1: observer’s task IV1: observer’s task Steering control: Steer so that you appear to be moving straight ahead Steering control: Steer so that you appear to be moving straight ahead Pointing: Point in the direction towards which you perceive yourself to be moving while “riding” Pointing: Point in the direction towards which you perceive yourself to be moving while “riding” IV2: simulated velocity of observer movement IV2: simulated velocity of observer movement Three levels: 50, 100, and 200 ms -1 Three levels: 50, 100, and 200 ms -1 Dependent Variable (measure): RMS (root-mean- squared) heading error = sqrt(  error /N) Dependent Variable (measure): RMS (root-mean- squared) heading error = sqrt(  error /N)

8. PSYC512: Research Methods Results Significant Effects Task Main Effect Interaction between task and velocity IMPORTANT! ALWAYS INTERPRET HIGHEST ORDER EFFECT

9. PSYC512: Research Methods Specialized Research Designs Combining between-subjects and within-subjects factors in research design – mixed designs Combining between-subjects and within-subjects factors in research design – mixed designs Combining experimental and correlational designs (Analysis of covariance or ANCOVA) Combining experimental and correlational designs (Analysis of covariance or ANCOVA) Quasi-Experimental Designs Quasi-Experimental Designs Pretest-posttest designs Pretest-posttest designs Developmental designs (Longitudinal or cross-sectional) Developmental designs (Longitudinal or cross-sectional)

10. PSYC512: Research Methods Combining Between-Subjects and Within Subjects Designs The Mixed Design The Mixed Design Also known as “split plot” Also known as “split plot” Groups of subjects each receive a unique level of the between Ss variable(s) and all levels of the within Ss variable(s) Groups of subjects each receive a unique level of the between Ss variable(s) and all levels of the within Ss variable(s)

11. PSYC512: Research Methods Mixed Three-factor Design: Adding Presentation Order as a Factor to Account for Differential Carryover Presentation order Variable (between) Group A (control then pointing) Group B (pointing then control) Found 3-way interaction that indicates differential carryover (asymmetrical transfer)

12. PSYC512: Research Methods Combining Between-Subjects and Within Subjects Designs Example of a Mixed Design Example of a Mixed Design Task order manipulated between Ss Task order manipulated between Ss Group A (control then pointing) Group A (control then pointing) Group B (pointing then control) Group B (pointing then control) 3-way interaction indicates differential carryover effect (asymmetrical transfer) 3-way interaction indicates differential carryover effect (asymmetrical transfer)

13. PSYC512: Research Methods Combining Experimental and Correlational Designs Covariates in experimental designs Covariates in experimental designs Measure your subjects on a covariate—a variable that you believe may be correlated with your dependent variable Measure your subjects on a covariate—a variable that you believe may be correlated with your dependent variable If left unmeasured these covariates add error variance and might obscure significant effects If left unmeasured these covariates add error variance and might obscure significant effects Measuring the covariate allows you to use correlational statistical techniques in your analysis (e.g., Analysis of Covariance or ANCOVA) to “subtract out” the error variance associated with the covariate, thereby increasing the statistical power of your experiment Measuring the covariate allows you to use correlational statistical techniques in your analysis (e.g., Analysis of Covariance or ANCOVA) to “subtract out” the error variance associated with the covariate, thereby increasing the statistical power of your experiment Example: measuring IQ in a learning experiment Example: measuring IQ in a learning experiment

14. PSYC512: Research Methods Combining Experimental and Correlational Designs Quasi-independent variable in experimental designs Quasi-independent variable in experimental designs “Quasi” means “kind of, but not really” “Quasi” means “kind of, but not really” Similar to including a covariate, except Similar to including a covariate, except measurement of covariate is used to assign Ss to groups measurement of covariate is used to assign Ss to groups Covariate is thus treated as an quasi-independent variable Covariate is thus treated as an quasi-independent variable Quasi-independent variables are referred to as “quasi” because they cannot be manipulated, they are essentially dependent variables (measures) that are treated as independent variables in the experimental design and analysis Quasi-independent variables are referred to as “quasi” because they cannot be manipulated, they are essentially dependent variables (measures) that are treated as independent variables in the experimental design and analysis

15. PSYC512: Research Methods Quasi-experimental Designs Quasi-experimental designs are those in which only quasi- independent variables are used Quasi-experimental designs are those in which only quasi- independent variables are used Time series vs. pretest-posttest designs Time series vs. pretest-posttest designs Time series: Measure behavior several times prior to and following a treatment (time series design) or change in your quasi-independent variable (interrupted time series design) Time series: Measure behavior several times prior to and following a treatment (time series design) or change in your quasi-independent variable (interrupted time series design) Pretest-posttest: Measure behavior once prior to and once following the change in your independent variable Pretest-posttest: Measure behavior once prior to and once following the change in your independent variable

16. PSYC512: Research Methods Quasi-experimental Designs Equivalent time samples design Equivalent time samples design Time-series design especially useful for treatments with transient effects Time-series design especially useful for treatments with transient effects Repeatedly measure behavior following multiple applications and withdrawals of the treatment Repeatedly measure behavior following multiple applications and withdrawals of the treatment Non-equivalent control group designs – helps control for history confounds which should affect both groups equally Non-equivalent control group designs – helps control for history confounds which should affect both groups equally

17. PSYC512: Research Methods Developmental Designs Used to assess changes in behavior related to a person’s chronological age, which serves as a quasi-independent variable Used to assess changes in behavior related to a person’s chronological age, which serves as a quasi-independent variable Cross-sectional designs Cross-sectional designs Simultaneously test subjects assigned to two or more age groups Simultaneously test subjects assigned to two or more age groups Generational effects can confound the age variable Generational effects can confound the age variable Longitudinal designs Longitudinal designs Repeatedly test a single group of subjects over time Repeatedly test a single group of subjects over time Controls for generational effects—but, may still limit external validity Controls for generational effects—but, may still limit external validity May be confounded by history, mortality, and/or multiple observation effects May be confounded by history, mortality, and/or multiple observation effects Cohort-sequential design Cohort-sequential design Combines longitudinal and cross-sectional designs by measuring multiple age groups over time which allows evaluation of generational or historical confounds Combines longitudinal and cross-sectional designs by measuring multiple age groups over time which allows evaluation of generational or historical confounds

18. PSYC512: Research Methods Next Time… More on experimentation More on experimentation Small-n designs Small-n designs