1. Experimental and Ex Post Facto Designs
Chapter Ten
Experimental and Ex Post Facto Designs

2. Independent and Dependent Variables
Variable: any quality or characteristic in a research
investigation that has two or more values.
Cause-and-Effect Relationship: the extent to which one
variable (the cause) influences another variable (the effect).
Independent Variable: a variable that the researcher studies
as a possible cause of something else; the variable that
the researcher directly manipulates.
Dependent Variable: a variable that is potentially influenced
by the independent variable; a variable that is influenced
by and to some extent depends on the independent
variable.

3. What is a Variable? Simply, something that varies.
Specifically, variables represent persons or objects that can be manipulated, controlled, or merely measured for the sake of research.
Variation: How much a variable varies. Those with little variation are called constants.

4. Independent Variables
These variables are ones that are more or less controlled.
Scientists manipulate these variables as they see fit.
They still vary, but the variation is relatively known or taken into account.
Often there are many in a given study.

5. Dependent Variables
Dependent variables are not controlled or manipulated in any way, but instead are simply measured or registered.
These vary in relation to the independent variables, and while results can be predicted, the data is always measured.
There can be any number of dependent variables, but usually there is one to isolate reason for variation.

6. Independent V. Dependent
Intentionally manipulated
Controlled
Vary at known rate
Cause
Intentionally left alone
Measured
Vary at unknown rate
Effect

7. The Importance of Control
Internal Validity: the extent to which the design of a research
study and the data it yields allows the researcher to draw
accurate conclusions about cause-and-effect and other
relationships. Without internal validity in experimental
designs, the results are not interpretable.
Confounding Variables: account for differences in two or
more groups that are not attributable to the particular
treatment or intervention being studied.
The greatest threat to internal validity is
confounding. Confounding occurs
whenever an extraneous variable
changes systematically along with the
independent variable. Confounding
prevents us from inferring a causal
relationship between the independent and
dependent variables

8. Strategies for Controlling Confounding Variables
Keep some things constant.
Include a control group.
Randomly assign people to groups.
Assess equivalence before the treatment with one or
more pretests.
5. Expose participants to all experimental conditions.
6. Statistically control for confounding variables.

9. Categories of Experimental Designs
Pre-Experimental Designs
True Experimental Designs
Quasi-Experimental Designs
Ex Post Facto Designs
Factorial Designs

10. Pre-Experimental Designs
Not possible to show cause-and-effect relationships because
(a) the independent variable doesn’t vary or (b) experimental and
control groups are not comprised of equivalent or randomly
selected individuals.
Design 1: One-Shot Experimental Case Study (low internal validity)
Group Tx Obs
Design 2: One-Group Pretest-Posttest Design
Group Obs Tx Obs
Design 3: Static Group Comparison
Group Tx Obs
Group Obs
In a pre experimental design something is missing from the true experimental design
What is usually missing is the control group is missing.
Pre-experimental designs are used when we have time series or longitudinal data.

11. True Experimental Designs
Compared to pre-experimental designs, experimental designs offer a
great degree of control and greater internal validity.
Design 4: Pretest-Posttest Control Group Design (random assignment)
Group Obs Tx Obs
Group Obs Obs
Design 5: Solomon Four-Group Design (enhances external validity)
Group Tx Obs
Group Obs
Design 6: Within-Subjects Design (repeated measures)
Group Txa Obsa
Txb Obsb R A N D O M R A N D O M
Basic research
Pure research
Cross section of time
Experimental designs
Greatest strength: strong internal validity able to control for threats—alternate explanations for treatment effects)
Attributed to random assignment
Relies heavily on random assignment to yield equivalent groups
Can be more certain (say with confidence) than any other design about attributing the cause to the independent variable
Greatest weakness: external validity—may inappropriate to attribute generalize beyond study populations
May be difficult to implement with integrity where individuals responsible for random assignment procedures are not compliant with research protocol
May be difficult to implement due to ethical considerations and inability to deny groups treatment
Must consider the effects of attrition

12. Quasi Experimental Designs
A type of research design for conducting studies in the field or real-life situations where the researcher may be able to manipulate the some independent variable but CANNOT randomly assign subjects to the control and experimental groups

13. When do we use a Quasi Experimental Design
When randomization is not possible
Independent variable precludes the use of random assignment
Retrospective studies
Studies that focus on economic or social conditions
Randomization is too expensive, not feasible, or impossible to monitor
Ethical issues as they relate to treatment
Quick timeline

14. Quasi-Experimental Designs
if one of the following conditions cannot be met is necessary to use quasi experimental designs
1. manipulation of at least one independent variable
2. random assignment of subjects to groups
3. random assignment of independent variable (treatment) to
groups
4. Exposure of experimental group to treatment in isolation from
other factors.

15. Strengths and Limitations
supports causal inferences
explain the uncertainty surrounding the existence of the causal relationship
nonequivalent groups can help strengthen the design
Limitations
Comparison base may be biased—doesn’t give
Strengths
Provides an approximation to the experimental design and supports causal inferences
Provides a means for explaining the uncertainty surrounding the existence of the causal relationship
Using nonequivalent groups can help strengthen the design
Limitations
Comparison base may be biased

16. Types of Quasi Experimental Designs
Comparison Group Pre/post design (experimental and comparison group)
Post test only
Interrupted Time series

17. Quasi-Experimental Designs
Randomness is not possible or practical; can’t control for all confounding
variables.
Design 8: Nonrandomized Control Group Pretest-Posttest Design
Group Obs Tx Obs
Group Obs Obs
Design 9: Simple Time-Series Design
Group 1 Obs Obs Obs Obs Tx Obs Obs Obs Obs
Design 10: Control Group, Time-Series Design
Group 2 Obs Obs Obs Obs Obs Obs Obs Obs
A type of research design for conducting studies in the field or real-life situations where the researcher may be able to manipulate the some independent variable but CANNOT randomly assign subjects to the control and experimental groups
Quasi –experimental designs
partial control over independent variables
Lacking random assignment
Time series design
Interrupted time series
Interrupted Time Series Design
• Effects of “treatment” (IV) inferred from
comparison of outcome measures (DV)
• Outcome measures obtained at different
time intervals
– Before and after treatment is introduced
1. Define period of observation broadly
– Observe DV before, during, and after intervention
2. Same units used throughout analysis
– Observations and time points equally spaced
3. Time points have to be sensitive to the
particular effects of interest
4. Measurements can’t fluctuate much
– No “instrument” changes
multiple time series

18. Quasi-Experimental Designs (con’t)
Design 11: Reversal Time-Series Design
Group Tx Obs Obs Tx Obs Obs
Design 12: Alternating Treatment Design
Group 1 Txa Obs --- Obs Txb Obs --- Obs Txa Obs --- Obs Txb Obs
Design 13: Multiple Baseline Design
Group 1
Baseline
------
Obs
Treatment
Tx Obs Tx
Group 2
Obs

19. Ex-post-facto designs
Studies that investigate the relationships between independent and dependent variables in situations where it is impossible or unethical to manipulate the independent variable
Example - what is the effect of pre-kindergarten (Pre-K) attendance on first grade achievement
Cannot mandate Pre-K attendance for children
Characteristics and resources of families who do and do not send their children to Pre-K may influence first grade achievement
Similarities with correlational and experimental research designs
Ex-post-facto designs
Issues of concern
Selecting participants who are as similar as possible on all characteristics except the independent variable
Generalizing beyond the participants studied
   

20. Ex Post Facto Designs
The researcher identifies events that have already occurred or
conditions that are already present and then collects data to
investigate a possible relationship between these factors and
subsequent characteristics or behaviors.
- like correlational research, ex post facto research involves looking at
existing circumstances;
- like experimental research, ex post facto research has clearly
identifiable independent and dependent variables;
- unlike experimental research, ex post facto research involves no direct
manipulation of the independent variable – the presumed “cause” has
already occurred.
Design 14: Simple Ex Post Facto Design
Prior event Investigation period
Group exp Obs
Group Obs

21. Factorial Designs - 1
Intervention studies with 2 or more categorical explanatory variables leading to a numerical outcome variable are called Factorial Designs.
A factor is simply a categorical variable with two or more values, referred to as levels.

22. Factorial Designs
Examination of the effects of two or more independent variables in a
single study.
Design 15: Randomized Two-Factor Design
Treatment (var.1) Treatment (var. 2)
Group Tx Tx Obs
Group Tx Obs
Group Tx Obs
Group Obs
Design 16: Combined Experimental and Ex Post Facto Design
Group 1a Txa Obs
Group Expa Random assign Group 1b Txb Obs
Group Expb Random assign Group 2a Txa Obs
Group 2b Txb Obs

23. Factorial Designs Advantages of factorial Designs are:
A greater precision can be obtained in estimating the overall main factor effects.
Interaction between different factors can be explored.
Additional factors can help to extend validity of conclusions derived.