Copyright © Allyn & Bacon (2007) Single-Variable, Independent-Groups Designs Graziano and Raulin Research Methods: Chapter 10 This multimedia product and its contents are protected under copyright law. The following are prohibited by law: (1) Any public performance or display, including transmission of any image over a network; (2) Preparation of any derivative work, including the extraction, in whole or in part, of any images; (3) Any rental, lease, or lending of the program.

Copyright © Allyn & Bacon (2007) Experimental Design 1. Tests hypotheses about causal effects of the independent variable (IV) 2. Includes at least two levels of the IV 3. Randomly assigns participants to conditions 4. Includes specific procedures for testing hypotheses 5. Includes control for the major threats to internal validity

Copyright © Allyn & Bacon (2007) Variance Variance is necessary in any research Variance is necessary in any research –Without variance, there is nothing to test –Variance was defined in Chapter 5 Research designs control unwanted sources of variance in order to evaluate the effects of the independent variable Research designs control unwanted sources of variance in order to evaluate the effects of the independent variable

Copyright © Allyn & Bacon (2007) Forms of Variance Systematic between-groups variance Systematic between-groups variance –Experimental variance (due to the IV) –Extraneous variance (due to confounding variables) Nonsystematic within-groups variance Nonsystematic within-groups variance –Due to chance factors and individual differences We analyze the results of our study using the F-test (ANOVA) We analyze the results of our study using the F-test (ANOVA) –Ratio of between-groups variation to within-groups variation

Copyright © Allyn & Bacon (2007) Controlling Variance Maximizing experimental variance Maximizing experimental variance –Make sure that there are real differences between the groups (using a manipulation check) Controlling extraneous variance Controlling extraneous variance –Make sure the groups are as similar as possible at the start of the study –Therefore, the only difference is the IV manipulation Minimizing error variance Minimizing error variance –Control with careful measurement or with special designs (e.g., correlated-group designs)

Copyright © Allyn & Bacon (2007) Controlling Variance The relationship of the various sources of variance to the F ratio is shown here The relationship of the various sources of variance to the F ratio is shown here You want to You want to –Maximize experimental variance –Minimize error variance –Control extraneous variance

Copyright © Allyn & Bacon (2007) Manipulation Check An specific test of whether the independent variable manipulation actually worked the way it was intended An specific test of whether the independent variable manipulation actually worked the way it was intended Example: A study testing the hypothesis that females, but not males, tend to turn anger inward rather than express it externally Example: A study testing the hypothesis that females, but not males, tend to turn anger inward rather than express it externally

Copyright © Allyn & Bacon (2007) Expressed Hostility The data on the dependent measure suggests that females really do respond with less hostility than males when frustrated The data on the dependent measure suggests that females really do respond with less hostility than males when frustrated

Copyright © Allyn & Bacon (2007) Reported Anger However, the manipulation check of reported anger suggests that the females may not have been angered by the frustration manipulation However, the manipulation check of reported anger suggests that the females may not have been angered by the frustration manipulation

Copyright © Allyn & Bacon (2007) Physiological Arousal And the second manipulation check of physiological arousal seems to indicate that the report of less anger by the female participants is real And the second manipulation check of physiological arousal seems to indicate that the report of less anger by the female participants is real

Copyright © Allyn & Bacon (2007) Nonexperimental Designs Do not include the critical controls of experimental designs Do not include the critical controls of experimental designs May still be used, but caution is necessary May still be used, but caution is necessary Four designs covered in this section Four designs covered in this section –Ex post facto design –Single-group, posttest-only design –Single-group, pretest-posttest design –Pretest-posttest, natural control-group design

Copyright © Allyn & Bacon (2007) Ex Post Facto Design A very weak design A very weak design –What we do when we try to figure out, after the fact, what caused something to happen –Not good science –Does not control confounding variables

Copyright © Allyn & Bacon (2007) Single-Group, Posttest-Only Design Even with the manipulation, virtually no control over confounding variables Even with the manipulation, virtually no control over confounding variables We tend to use an implicit control group (what we think would have happened if there had been no manipulation) We tend to use an implicit control group (what we think would have happened if there had been no manipulation)

Copyright © Allyn & Bacon (2007) Single-Group, Pretest-Posttest Design The pretest documents change, but factors other than the treatment could have accounted for the change The pretest documents change, but factors other than the treatment could have accounted for the change –History, maturation, regression to the mean, etc.

Copyright © Allyn & Bacon (2007) Pretest-Posttest, Natural Control-Group Design

Copyright © Allyn & Bacon (2007) Pretest-Posttest, Natural Control-Group Design Like an experiment except that participants are not randomly assigned to the groups Like an experiment except that participants are not randomly assigned to the groups A reasonably strong design except that it does not control for selection A reasonably strong design except that it does not control for selection –Selection could be a powerful confounding factor in many studies

Copyright © Allyn & Bacon (2007) Experimental Designs Meet all criteria for an experiment Meet all criteria for an experiment Provide more powerful tests of hypotheses Provide more powerful tests of hypotheses Designs discussed in this chapter Designs discussed in this chapter –Randomized, posttest-only, control-group design –Randomized, pretest-posttest, control-group design –Multilevel, completely randomized, between-subjects designs –Solomon’s four-group designs

Copyright © Allyn & Bacon (2007) Randomized, Posttest-Only, Control-Group Design

Copyright © Allyn & Bacon (2007) Randomized, Posttest-Only, Control-Group Design Random assignment controls for selection Random assignment controls for selection Other confounding variables are controlled by comparing the treatment and no treatment groups Other confounding variables are controlled by comparing the treatment and no treatment groups –For example, history and maturation should be the same in both groups

Copyright © Allyn & Bacon (2007) Randomized, Pretest-Posttest, Control-Group Design

Copyright © Allyn & Bacon (2007) Randomized, Pretest-Posttest, Control-Group Design Adding a pretest allows us to quantify the amount of change following treatment Adding a pretest allows us to quantify the amount of change following treatment Also allows us to verify that the groups were equal initially Also allows us to verify that the groups were equal initially A strong basic research design, with excellent control over confounding A strong basic research design, with excellent control over confounding

Copyright © Allyn & Bacon (2007) Multilevel, Randomized, Between-Subjects Design

Copyright © Allyn & Bacon (2007) Multilevel, Randomized, Between-Subjects Design May or may not include a pretest May or may not include a pretest Multi-group extension of the basic experimental designs Multi-group extension of the basic experimental designs Controls virtually all sources of confounding variables Controls virtually all sources of confounding variables

Copyright © Allyn & Bacon (2007) Solomon’s Four-Group Design

Copyright © Allyn & Bacon (2007) Solomon’s Four-Group Design Combines two basic experimental designs Combines two basic experimental designs –Randomized, posttest-only, control-group design –Randomized, pretest-posttest, control- group design Allows the assessment of an interaction between the pretest and the treatment Allows the assessment of an interaction between the pretest and the treatment

Copyright © Allyn & Bacon (2007) Statistical Analysis Issues If the data are nominal, use chi-square If the data are nominal, use chi-square If the data are ordinal, use the Mann- Whitney U-test (two groups only) If the data are ordinal, use the Mann- Whitney U-test (two groups only) If the data are interval or ratio If the data are interval or ratio –If two groups, a t-test of the posttest measures will test the hypothesis –More complex designs require an ANOVA

Copyright © Allyn & Bacon (2007) Analysis of Variance Evaluates differences in group means Evaluates differences in group means –It does this evaluation by comparing different variance estimates (termed mean squares) –The F statistic is a ratio of the mean square between-groups and the mean square between-groups and the mean square within-groups the mean square within-groups The larger the differences between the group means, the greater the F value The larger the differences between the group means, the greater the F value

Copyright © Allyn & Bacon (2007) Specific Mean Comparisons A significant F-test means that at least one group is significantly different from at least one other group A significant F-test means that at least one group is significantly different from at least one other group –If you have more than two groups, you have to do follow-up tests to see which groups differ Specific mean comparisons can be Specific mean comparisons can be –Planned comparisons –Post hoc tests

Copyright © Allyn & Bacon (2007) Other Experimental Designs Other experimental designs covered in later chapters Other experimental designs covered in later chapters Correlated-groups designs (Chapter 11) Correlated-groups designs (Chapter 11) –Within-subjects designs –Matched-subjects designs –Single-subject designs Factorial designs (Chapter 12) Factorial designs (Chapter 12) –Many variations on factorial designs are possible

Copyright © Allyn & Bacon (2007) Summary Research is designed to measure and control sources of variance Research is designed to measure and control sources of variance There are several non-experimental and experimental designs available There are several non-experimental and experimental designs available Experimental designs have two elements Experimental designs have two elements –Random assignment of participants to conditions –At least one control group