1. Chapter 11 EDPR 7521 Dr. Kakali Bhattacharya
Experimental Designs
Chapter 11
EDPR 7521
Dr. Kakali Bhattacharya

2. Why Experiments? Establish cause and effect
Two or more groups to study
Get grants 

3. Key Characteristics Random assignment
Control over extraneous variables
Manipulation of treatment conditions
Outcome measures
Group comparisons
Threats to validity

4. Control Procedures Pretests and posttests Covariates
Expectations can affect outcomes
Covariates
Relate only to dependent variable
Used to control for effects of pretests, or other variables that need to be controlled for
E.g. scores on a pretest correlated with dependent variable
Analysis of covariance – adjusts scores on dependent variable to account for covariance
Matching of participants
Gender, pretest scores, individual abilities
Homogenous samples
Blocking variables
Break into subgroups, effects of subgroups on outcome

5. Manipulating Treatment Conditions
Treatment variables
independent variables that are manipulated by the researcher
Conditions
Levels of treatment variables
Manipulate the treatment conditions
Who gets to receive what treatment in control and experimental groups

6. Outcome Measures Dependent variable
presumed effect of the treatment variable
Effect predicted in the hypothesis
Good outcome measures are sensitive to smallest amount of intervention

7. Group Comparisons
Compare means and variance both within and between groups
You would want to do this to make your study stronger
Show that you have considered multiple forms of manipulation of independent variables

8. Threats to Internal Validity
History
Maturation
Regression
Selection
Mortality
Interactions with selection
Diffusion of treatments
Experimental and control group can communicate with each other
Compensatory equalization
One group gets benefits and other doesn’t. Use comparison groups instead of control groups and offer some mitigated benefits
Switch groups (education)
Compensatory rivalry
Publicly announce benefit of being in the experimental group
Resentful demoralization
Less desirable treatment
Testing (remember)
Instrumentation (change)
Bad design, bad conclusions
Bad inferences

9. Threats to External Validity
Interaction of selection and treatment
Inability to generalize beyond the groups in the experiment
Interaction of setting and treatment
Inability to generalize to another setting
Analyze effect of setting and treatment
BAD GENERALIZABILITY

10. Types of Experimental Designs
Between Group Designs (2 or more groups)
true, quasi, factorial designs (multiple independent variables, p. 299)
Within Group or Individual Designs
Limited participants, can’t compare groups
Time series designs (Table 11.5, p. 302)
Repeated Measures Design
All participants in a single group participate in all experimental treatments with each group becoming its own control (compare outcomes, need to make treatments distinct)
Single-subject Designs
Baseline, intervention, outcome, reversal (p. 305)

11. Steps in Conducting Experimental Research
Decide if experiment addresses research purpose and questions
Form hypotheses to test cause and effect
Select experimental unit
Identify participants
Select experimental treatment
Introduce experimental treatment
Choose a type of experimental design
Conduct the experiment
Organize data
Analyze data
Write up findings