1. Experiments and Quasi-Experiments1
Experiments and Quasi-Experiments

2. 2
Introduction
Experiment: using a controlled situation to observe a result
Involves taking and observing action
Great for hypothesis-testing
Theory-full

3. The Classical Experiment3
The Classical Experiment
Involves three major pairs of components:
Independent and dependent variables
Pre-Testing and Post-Testing
Experimental and Control groups
Randomization

4. Variables, X and Y X = Independent Variable (IV), cause, influencer4
Variables, X and Y
X = Independent Variable (IV), cause, influencer
Y = Dependent Variable (DV), effect, outcome

5. Control and Experimental Groups5
Control and Experimental Groups
Experimental group – exposed to whatever treatment, policy, initiative we are testing
Control group – very similar to experimental group, except that they are NOT exposed

6. 6
Selecting Subjects
Decide on target population 1st– the group to which the results of your experiment will apply
Cardinal rule – ensure that C and E groups are as similar as possible
Randomization helps towards this

7. Hawthorne Effect Pointed to the necessity of control groups7
Hawthorne Effect
Pointed to the necessity of control groups
IV: improved working conditions (better lighting)
DV: improvement in employee satisfaction and productivity
Workers were responding more to the attention than to the improved working conditions

8. 8
Placebo
We often don’t want people to know if they are receiving treatment or not
We expose our control group to a “dummy” IV just so we are treating everyone the same
Medical research: participants don’t know what they are taking
Ensures that changes in DV actually result from IV and are not psychologically based

9. Pre-Testing and Post-Testing9
Pre-Testing and Post-Testing
First, subjects measured on DV prior to association with the IV (pre-tested)
Next, subjects are exposed to the IV
Third, subjects are remeasured in terms of the DV (post-tested)
Difference?--must be the intervention!

10. Double-Blind Experiment10
Double-Blind Experiment
Subjects and experimenters do not know who is in the control and experimental groups

11. Experiments and Causal Inference11
Experiments and Causal Inference
Experimental design ensures:
Cause precedes effect via taking posttest
Empirical correlation exists via comparing pretest to posttest
No spurious 3rd variable influencing correlation via posttest comparison between experimental and control groups, and via randomization

12. Internal Validity Threats (12)
Conclusions drawn from experimental results may not reflect what went on in experiment
History – external events may occur during the course of the experiment
Maturation – people grow
Testing – the process of testing and retesting

13. More Internal Validity Threats13
More Internal Validity Threats
4. Instrumentation – Changes in the measurement process
5. Statistical regression – Extreme scores regress to the mean
6. Selection bias – the way in which subjects are chosen
7. Experimental mortality – subjects may drop out prior to completion of experiment
8. Causal time order – ambiguity about order of stimulus and DV – which caused which?

14. Last, Internal Validity Threats14
Last, Internal Validity Threats
9. Diffusion/imitation of treatment – when E and C groups communicate, E group may pass on elements to C
10. Compensatory treatment – C group is deprived of something considered to be of value
11. Compensatory Rivalry – C group deprived of the stimulus may try to compensate by working harder
12. Demoralization – feelings of deprivation result in C group giving up

15. Construct Validity Threats15
Construct Validity Threats
Concerned with generalizing from experiment to actual causal processes in the real world
Link construct and measures to theory
Clearly indicate what constructs are represented by what measures
Decide how much treatment is required to produce change in DV

16. External Validity Threats16
External Validity Threats
Significant for experiments conducted under carefully controlled conditions rather than more natural conditions
But, this reduces internal validity threats!
A conundrum!
Suggestion – explanatory studies -> internal validity; applied studies -> external validity

17. Statistical Conclusion Validity Threats (Low Power)17
Statistical Conclusion Validity Threats (Low Power)
Problem is likely when using small samples
With more cases, it is easier to see more differences

18. Quasi-Experimental Designs18
Quasi-Experimental Designs
When?—randomization not possible
Quasi = “to a certain degree” or, in short, “like”