1. GROUP-LEVEL DESIGNS
Chapter 9

2. CHARACTERISTICS OF “IDEAL” EXPERIMENTS
Research designs that can establish a causal relationship between variables
Six characteristics
Time order of the independent variable (IV)
IV is manipulated
The IV and DV have a relationship
Rival hypotheses are controlled for
At least one control group is used
Random assignment is used

3. Controlling the Time Order of Variables
The Independent Variable (IV) must occur before change in the dependent variable (DV) is observed

4. Manipulating the Independent Variable
Three ways to manipulate the IV
One group experiences the IV (X is present) and one group does not (X is absent)
Adjust the “dosage” or amount of exposure of the IV (small amount of X versus large amount of X)
One group experiences the IV (X is present) and one group experiences an alternative intervention (something else)

5. Establishing Relationships between Variables
The relationship between the independent and the dependent variables must be established in order to infer a cause-effect relationship

6. Controlling Rival Hypotheses
Rival hypotheses – a plausible alternative explanation to the research hypothesis
Three ways to rule out other extraneous variables that might affect the dependent variable
Holding Extraneous Variables Constant
Using Correlated Variation
Using Analysis of Covariance

7. Holding Extraneous Variables Constant
Any extraneous variables that might affect the dependent variable are held constant in an ideal experiment by
Being applied equally to all research participants
If gender is thought to affect the DV, then limit the sample to only females
Remaining unchanged for the duration
If dosage of medication is thought to affect the DV, then keep the dosage constant

8. Using Correlated Variation
If several independent variables are used in a research study, then determine to what degree they are correlated
If the correlation between two independent variables is high, then only one of those variables needs to be included

9. Using Analysis of Covariance
A statistical method that is used to compensate for differences between the groups being compared in a research study

10. Using a Control Group
The group in the research study that does not receive the intervention (or IV)
A control group is only effective when research have been randomly assigned to either the experimental or the control group
Symbols
R = Randomization (selection or assignment)
O = Observation (measurement) of the DV
X = Independent variable or IV

11. Randomly Assigning Research Participants to Groups
After research participants have been randomly selected from the population, they are randomly assigned to either an experimental or control group
Research participants are assigned to either group on the basis of chance (they have an equal chance of being in the experimental or control group)

12. Matched Pairs
Another method of dividing research participants into comparison groups
Research participants are matched on key characteristics, then one individual from each pair is place into two separate groups
Individuals that do not have a pair are eliminated from the study

13. INTERNAL AND EXTERNAL VALIDITY
Group-level designs are evaluated based on their ability to generate knowledge
Internal Validity – the degree to which a research design can ensure that the independent variable is the sole cause of change in the dependent variable
External Validity – the extent to which the findings of a research design can generalized to other groups (population) or situations

14. Threats to Internal Validity (Box 8.1)
Known threats that provide alternative explanations (rival hypotheses) for what might bring about change in the dependent variable
Differential Selection
Mortality
Reactive Effects
Interaction Effects
Inter-group Relations
History
Maturation
Testing
Instrumentation Error
Statistical Regression

15. Threats to External Validity (Box 8.2)
Known threats that limit or restrict the degree to which research study results are generalizable
Pretest-Treatment Interaction
Selection-Treatment Interaction
Specificity of Variables
Reactive Effects
Multiple Treatment Interference
Researcher Bias

16. GROUP RESEARCH DESIGNS
Group research designs are categorized along the continuum of knowledge
Exploratory Designs
Descriptive Designs
Explanatory Designs

17. Exploratory Designs
Do not contain any of the requirements of the “ideal” experiment
Threats to internal and external validity are high (i.e., virtually all apply)
These designs are used to explore a research question about which little is already known in order to uncover generalizations and to develop hypotheses for further investigation and testing

18. One-Group Posttest-Only Design
Involves a single measure or observation (O1) of the dependent variable that occurs after one group of people has experienced the intervention (X)
Design Blueprint: X O1
The design does not control for any threats to internal or external validity

19. Cross-Sectional Survey Design
Another form of a one-group posttest-only design but the intervention (X) is not specified
A cross-section of a population is observed (O1) at one particular point in time
Design Blueprint: O1

20. Multigroup Posttest-Only Design
The one-group posttest-only design is applied to multiple groups
A single measure or observation (O1) of the dependent variable occurs after each group experiences some form of the intervention (X)
Design Blueprint:
Group 1: X O1
Group 2: X O1

21. Longitudinal Case Study Design
Involves repeated measures or observations (O1) of the dependent variable that occurs after one group of people has experienced the intervention (X)
Design Blueprint: X O1 O2 O3

22. Longitudinal Survey Design
Involves repeated measures or observations (O1) of one group of people over time
Design Blueprint: O1 O2 O3
Trend Studies – repeated observations on multiple samples drawn from one population
Cohort Studies – repeated observations on one group (sample)

23. Descriptive Designs Apply some “Ideal” experiment features:
Time order of variables
Manipulation of the independent variable
Use of comparison group (not a control group)
Random selection but not random assignment
Compared to exploratory designs, threats to internal and external validity are reduced

24. Randomized One-Group Posttest-Only Design
Involves a single measure or observation (O1) of the dependent variable that occurs after one randomly selected group of people has experienced the intervention (X)
Design Blueprint: R X O1

25. Randomized Cross-Sectional Survey Design
Another form of a randomized one-group posttest-only design but the intervention (X) is not specified
A randomly selected cross-section of a population is observed (O1) at one particular point in time
Design Blueprint: R O1

26. One Group Pretest-Posttest Design
Involves repeated measures or observations of the dependent variable
The first observation (O1) occurs before the intervention (X) and the second observation (O2) occurs after it
The posttest (O2) is compared to the pretest (O1) to determine if any change occurred in the dependent variable
Design Blueprint: O1 X O2

27. Comparison Group Posttest-Only Design
Involves two groups
The “experimental” group receives the intervention (X) while the comparison group does not
A single measure (O1) of the dependent variable occurs for each group
Design Blueprint:
Group 1: X O1
Group 2: O1

28. Comparison Group Pretest-Posttest Design
Involves repeated measures or observations of the dependent variable on two groups
An “experimental” and a comparison group
The first observation (O1) occurs before the experimental group receives X and the second observation (O2) occurs after X
Design Blueprint: O1 X O2
O1 O2

29. Interrupted Time-Series Design
Involves repeated measures or observations of the dependent variable on one group
Several observations occur before the intervention (X) and several occur after
Design Blueprint: O1 O2 O3 X O4 O5 O6

30. Explanatory Designs
Most closely approximate (and include) the “ideal” experiment
Most threats to internal and external validity are eliminated or “ruled out”
The major aim of these designs is to establish a causal connection between interventions (independent variable) and outcomes (dependent variables)

31. Classical Experimental Design
Considered to be the “ideal” experiment as all six requirements of are present:
Time order of IV; manipulation of IV; relationship between IV and DV; rival hypotheses controlled; control group; random selection and random assignment
Design Blueprint: R O1 X O2
R O1 O2

32. Randomized Posttest-Only Control Group Design
Another version of the “ideal” experiment where only one observation or measure is made of the dependent variable for each group
Design Blueprint: R X O1
R O1

33. SUMMARY
Group-level designs are categorized according to the knowledge level continuum
Exploratory, descriptive, explanatory
Threats to internal and external validity are highest for exploratory designs and lowest for explanatory designs