Matched Pairs, Within-Subjects, and Mixed Designs Chapter 13 Matched Pairs, Within-Subjects, and Mixed Designs PowerPoint presentation to accompany Research Design Explained 6h edition; ©2007 Mark Mitchell & Janina Jolley

Overview The Matched Pairs Design Pure Within-Subjects Designs Randomized Within-Subjects Designs Counterbalanced Within-Subjects Designs Choosing a Design PowerPoint presentation to accompany Research Design Explained 6th edition; ©2007 Mark Mitchell & Janina Jolley

The Matched Pairs Design Procedure* Considerations in Using the Matched Pairs Design* Analysis of Data* PowerPoint presentation to accompany Research Design Explained 6th edition; ©2007 Mark Mitchell & Janina Jolley

The Matched Pairs Design: Procedure Form matched pairs Randomly assign one member of each pair to the treatment condition, the other to the control condition PowerPoint presentation to accompany Research Design Explained 6th edition; ©2007 Mark Mitchell & Janina Jolley

Considerations in Using Matched Pairs Designs Finding an effective matching variable Power: A big plus External validity Advantage: Don’t restrict subject population (can have heterogeneous group) Disadvantage: Results may not generalize to participants who haven’t done the matching task Construct validity weakened because matching may tip off participants about hypothesis PowerPoint presentation to accompany Research Design Explained 6th edition; ©2007 Mark Mitchell & Janina Jolley

Analysis of Data in the Matched Pairs Design Not the between subjects t test (observations are not independent) Dependent t test: Differences between pairs/ standard error of differences PowerPoint presentation to accompany Research Design Explained 6th edition; ©2007 Mark Mitchell & Janina Jolley

Within-Subjects (Repeated Measures) Designs Considerations in using within-subjects designs Increased power Order effects harm internal validity* PowerPoint presentation to accompany Research Design Explained 6th edition; ©2007 Mark Mitchell & Janina Jolley

Overview of Four Specific Sources of Order Effects Practice effects Fatigue effects Treatment carryover effects Sensitization PowerPoint presentation to accompany Research Design Explained 6th edition; ©2007 Mark Mitchell & Janina Jolley

Dealing with Order Effects Minimizing each individual threat (practice, fatigue, carryover, sensitization) Use as few levels as possible to reduce opportunities for practice, fatigue, carryover, and sensitization Mixing up sequences to try to balance out order effects: Randomizing and counterbalancing PowerPoint presentation to accompany Research Design Explained 6th edition; ©2007 Mark Mitchell & Janina Jolley

Randomized Within-Subjects Designs Procedure* Analysis of data* Summary* PowerPoint presentation to accompany Research Design Explained 6th edition; ©2007 Mark Mitchell & Janina Jolley

Procedure As in all within-subject (repeated measures) designs, each subject is observed in at least two conditions Randomly determine the sequence of treatments for each participant PowerPoint presentation to accompany Research Design Explained 6th edition; ©2007 Mark Mitchell & Janina Jolley

Analysis of Data Dependent t test (two groups) or Within-Subjects ANOVA (more than two groups) Within-subjects t (Same test as used for matched pairs analysis) For each participant, get the difference between his/her score in Conditions 1 vs. his/her score in Condition 2. Calculate average difference Divide average difference by standard error of the differences Look up t in t table under appropriate df (number of participants -1) and significance level. PowerPoint presentation to accompany Research Design Explained 6th edition; ©2007 Mark Mitchell & Janina Jolley

Randomized Within-Subjects Designs: Summary Like all within-subjects designs, quite powerful Randomization helps balance out order effects, but is there a more effective way? PowerPoint presentation to accompany Research Design Explained 6th edition; ©2007 Mark Mitchell & Janina Jolley

Counterbalanced Within-Subjects Designs Procedure* Advantages of counterbalancing* Disadvantages of counterbalancing* Conclusions about counterbalanced within-subjects designs PowerPoint presentation to accompany Research Design Explained 6th edition; ©2007 Mark Mitchell & Janina Jolley

Procedure Randomly assign participants to your sequences Devise a set of sequences such that Every condition appears in every position the same number of times (T1 should appear first as many times as it appears last) and Every condition precedes every other condition just as many times as it follows that condition (For every sequence in which T1 comes before T2, there should be a sequence in which as T1 comes after T2) Example: Sequence 1: T1 then T2 Sequence 2: T2 then T1 Randomly assign participants to your sequences PowerPoint presentation to accompany Research Design Explained 6th edition; ©2007 Mark Mitchell & Janina Jolley

Advantages of counterbalancing Balance out routine order effects Learn about effect of the within-subjects variable of order (trials, position) Learn about the effect of the between-subjects variable of sequence PowerPoint presentation to accompany Research Design Explained 6th edition; ©2007 Mark Mitchell & Janina Jolley

Disadvantages of counterbalancing May require more subjects Analysis is more sophisticated PowerPoint presentation to accompany Research Design Explained 6th edition; ©2007 Mark Mitchell & Janina Jolley

Counterbalanced Within-Subjects Designs: Conclusions Balances out routine order effects Provides information not only about the effect of treatment, but also about the effect of order (trials, position) and sequence PowerPoint presentation to accompany Research Design Explained 6th edition; ©2007 Mark Mitchell & Janina Jolley

Choosing an Experimental Design General considerations* The two-condition case* The multiple IV case* PowerPoint presentation to accompany Research Design Explained 6th edition; ©2007 Mark Mitchell & Janina Jolley

General considerations: Pure between-subjects designs may Have more construct validity because it is harder for participants to guess the hypothesis Have more internal validity because they are not vulnerable to order effects Be easier to analyze Within-subject designs have more power Whether within-subject designs or between-subjects designs have more external validity may depend on whether the variable is “within-subjects” or “between-subjects” in real life PowerPoint presentation to accompany Research Design Explained 6th edition; ©2007 Mark Mitchell & Janina Jolley

Choosing Designs: The Two Conditions Case Pure between subjects designs Matched pairs design Randomized within-subjects designs Counterbalanced designs PowerPoint presentation to accompany Research Design Explained 6th edition; ©2007 Mark Mitchell & Janina Jolley

Choosing Designs: When You Have More Than One Independent Variable Using a within-subjects factorial design Using a between-subjects factorial design Using a mixed design PowerPoint presentation to accompany Research Design Explained 6th edition; ©2007 Mark Mitchell & Janina Jolley

Concluding Remarks You can now intelligently choose among different types of experimental designs. You can now propose almost any type of experiment You can now read almost any write-up of a study that used an experiment PowerPoint presentation to accompany Research Design Explained 6th edition; ©2007 Mark Mitchell & Janina Jolley