1. ITEC6310 Research Methods in Information Technology Instructor: Prof. Z. Yang Course Website: http://people.math.yorku.ca/~zyang /itec6310.htm Office: Tel 3049

2. 222 Within-Subjects Designs Subjects are not randomly assigned to treatment conditions –The same subjects are used in all conditions. –Closely related to the matched-groups design Advantages –Reduces error variance due to individual differences among subjects across treatment groups –Reduced error variance results in a more powerful design Effects of independent variable are more likely to be detected.

3. 333 Disadvantages –More demanding on subjects, especially in complex designs –Subject attrition is a problem –Carryover effects: exposure to a previous treatment affects performance in a subsequent treatment Within-Subjects Designs

4. 444 Sources of Carryover Learning –Learning a task in the first treatment may affect performance in the second Fatigue –Fatigue from earlier treatments may affect performance in later treatments Habituation –Repeated exposure to a stimulus may lead to unresponsiveness to that stimulus

5. 555 Sensitization –Exposure to a stimulus may make a subject respond more strongly to another Contrast –Subjects may compare treatments, which may affect behavior Adaptation –If a subject undergoes adaptation (e.g., dark adaptation), then earlier results may differ from later ones Sources of Carryover

6. 666 Dealing with Carryover Effects Counterbalancing –The various treatments are presented in a different order for different subjects –May be complete or partial The Latin Square Design –Used when you make the number of treatment orders equal to the number of treatments

7. 777 Example of a Counterbalanced Single-Factor Design With Three Treatments Subjects First Treatment Administered Second Treatment Administered Third Treatment Administered S1S1 123 S2S2 132 S3S3 213 S4S4 231 S5S5 312 S6S6 321

8. 888 Taking Steps to Minimize Carryover –Techniques such as pretraining, practice sessions, or rest periods between treatments can reduce some forms of carryover Make Treatment Order an Independent Variable –Allows you to measure the size of carryover effects, which can be taken into account in future experiments Dealing with Carryover Effects

9. 999 When to Use a Within-Subjects Design A within-subjects design may be best when –Subject variables are correlated with the dependent variable –It is important to economize on participants or subjects –You want to assess the effects of increasing exposure on behavior

10. 10 Factorial Designs Adding a second independent variable to a single-factor design results in a FACTORIAL DESIGN. Two components can be assessed –The MAIN EFFECT of each independent variable The separate effect of each independent variable Analogous to separate experiments involving those variables Represented by means for each level of each factor –The INTERACTION between independent variables When the effect of one independent variable changes over levels of a second Represented by the “cell means” in your design

11. 11 Example The researcher is interested in a study to investigate the following two hypotheses: Do object-oriented techniques increase the understandability and ease of modification of the resulting design documents over the use of structured techniques? Does the use of perceived ‘good’ and ‘bad’ design principles have any influence on the understandability and ease of modification of these design documents?

12. 12 Higher-Order Factorial Designs More than two independent variables are included in a higher-order factorial design –As factors are added, the complexity of the experimental design increases The number of possible main effects and interactions increases The number of subjects required increases The volume of materials and amount of time needed to complete the experiment increases

13. 13 The Pretest-Posttest Design Pretest administered before exposure to experimental treatment Used to assess the impact of some change on performance There is a problem with pretest sensitization –Taking the pretest may alter the way a person performs in an experiment