1. Chapter 11 Experimental Designs
Power Point Slides by Ronald J. Shope in collaboration with John W. Creswell

2. Educational Research 2e: Creswell
Key Ideas
Definition and uses for experimental designs
Procedures used in experimental research
Internal and external validity threats in experimental research
Types of experimental designs
Steps in conducting experimental research
Evaluating experimental research
Educational Research 2e: Creswell

3. What is experimental research?
In an experiment, you test an idea (or practice or procedure) to determine whether it influences an outcome or dependent variable.
Determine activities with which to “experiment”
Assign individuals to experience it (and have some individuals experience something different)
Determine whether those who experienced the activities (or practice or procedure) performed better on some outcome than those who did not experience it.
Educational Research 2e: Creswell

4. When do you use experimental procedures?
To establish cause and effect
Between independent and dependent variables
Control for all variables that might influence the outcome
Used when testing theories
Used when comparing two or more groups
Educational Research 2e: Creswell

5. How did experiments develop?
Psychological experiments (late 19th early 20th century)
Procedures for comparing groups (McCall 1925)
Statistics for comparing groups (e.g. Chi-Square) Identification of types of experimental designs (Campbell and Stanley 1963)
Types of basic designs and threats (Cook and Campbell 1979)
Complicated experiments with many variables (Since 1980)
Educational Research 2e: Creswell

6. Key Characteristics of Experimental Designs
Random assignment
Control over extraneous variables
Manipulation of the treatment conditions
Outcome Measures
Group Comparisons
Threats to validity
Educational Research 2e: Creswell

7. Educational Research 2e: Creswell
Random assignment
Randomly assign individuals at random conditions or to groups
Random assignment equates groups and distributes variability between or among groups and conditions
Different from Random Selection which is selection of participants at random
Educational Research 2e: Creswell

8. Control over extraneous variables
Extraneous Variables: influences in participant selection, procedures, statistics, or the design likely to affect the outcome and provide an alternative explanation results than what was expected.
Random assignment helps to control for extraneous variables
Done before the experiment begins
Educational Research 2e: Creswell

9. Control over extraneous variables
Other control procedures
pretest/posttest
covariates
matching participants
selecting homogenous samples
using blocking variables
Educational Research 2e: Creswell

10. Pre-Test and Post-Tests
Intervention
Time Time 2
Pre-Test Post-Test
Educational Research 2e: Creswell

11. Controlling for Covariates
Independent
Variable
Dependent
Variable
No Covariates
Covariate Introduced
Independent
Variable:
Type
of Instruction
Dependent
Variable:
Rates of
Smoking
Covariate:
Parents Who
Smoke
Variance
Removed
Variance
Educational Research 2e: Creswell

12. Matching Process Based on Gender
John
Jim
James
Josh
Jackson
Jane
Johanna
Julie
Jean
Jeb
Experimental
Group
Control
Group
Educational Research 2e: Creswell

13. Manipulation of the treatment conditions
Identify a treatment variable
Identify the conditions or levels of the treatment variable
Manipulate the treatment conditions
Educational Research 2e: Creswell

14. The Experimental Manipulation of a Treatment Group
Independent Variables
1. Age (can’t manipulate)
2. Gender (can’t manipulate)
3. Types of Instruction (can manipulate)
a. Lecture (control)
b. Lecture + Hazard Instruction (Comparison)
c. Lecture + Hazard Instruction + slides of damaged lungs (experiment)
Dependent Variable
Frequency of
Smoking
Educational Research 2e: Creswell

15. Educational Research 2e: Creswell
Outcomes
Criterion or effect variable
Outcome variable
Measured on a continuous scale
Educational Research 2e: Creswell

16. Group comparisons in an experiment
Phase 1: Relationship Picture
Error Correction Treatment
Spelling Accuracy
Phase 2: Timeline Picture
Test Test Test 3
6 Weeks 6 Weeks Weeks
Class A: Regular Spelling Practice (Control)
Class B: Reduced word list (Comparison)
Phase 3: Statistical Comparisons
Class A Class B Class C F-value
Test 1
Test 2
Test 3
10.3
(3.6)
10.7
(3.3)
11.1
10.8
(4.3)
10.6
(3.8)
9.9
(3.9)
13.9
(4.2)
13.1
0.27
4.90*
3.31*
Class C: Error Correction (Experimental) +
* p<.05
Educational Research 2e: Creswell

17. Threats to internal validity
History
Maturation
Regression
Selection
Mortality
Interactions with selection
Diffusion of treatments
Compensatory equalization
Compensation rivalry
Resentful demoralization
Testing
Instrumentation
Educational Research 2e: Creswell

18. Threats to external validity
Interaction of selection and treatment
Interaction of setting and treatment
Interaction of history and treatment
Educational Research 2e: Creswell

19. Types of experiments: Between groups
True Experiments
Pre- and posttest
Posttest Only
Quasi Experiments
Pre
Posttest
Factorial Designs
Educational Research 2e: Creswell

20. Means and Main Effects of Eight groups in Factorial Design
Depression
Mean rate of
smoking
Low
Health lecture
Type of Instruction
Standard lecture
Main Effects
of Type of
Instruction
Main Effects of
Medium
High
Educational Research 2e: Creswell

21. Graphs Showing Main and Interaction Effects
Standard lecture
High
Low
Medium
No Interaction Effects
(Parallel)
Health lecture
Extent of Smoking
High
Low
Medium
(b) Interaction Effects
(Crossed)
Standard lecture
Extent of Smoking
Health lecture
(c) Interaction Effects
(Not Parallel)
High
Low
Medium
Standard lecture
Extent of Smoking
Health lecture
Educational Research 2e: Creswell

22. Types of Experiments: Within-group or individual
Time series experiments
interrupted
uninterrupted
Repeated measures experiments
Single-subject experiments
A/B design
Multiple baseline design
Alternating treatments
Educational Research 2e: Creswell

23. Steps in Conducting Experimental Research
Decide if an experimental design fits research problem
Form cause/effect hypotheses
Select experimental unit and identify study participants
Select an experimental treatment and introduce it
Choose a type of experimental design
Conduct the experiment
Organize and analyze the data
Develop an experimental research report
Educational Research 2e: Creswell

24. Criteria for Evaluating Experimental Research
Does the experiment have a powerful intervention?
Does it employ few treatment groups (e.g. only two)?
Will participant gain from the intervention(s)?
Is there a systematic way the researcher derived the number of participants (cell size)?
Was there an adequate number of participants used in the study?
Were valid, reliable, and sensitive measures or observations used?
Did the study control for extraneous factors?
Did the researcher control for threats to internal validity?
Educational Research 2e: Creswell