1. Quasi-experimental Designs & Survey Research

2. Quasi-experiments
Hedrick, Bickman & Rog (1993) “a quasi-experimental design is not the method of choice, but rather a fallback strategy”
Cannot infer cause and effect, BUT well designed quasi-experiments enable you to demonstrate that rival interpretations are rendered unlikely

3. Non-equivalent Group Designs
Posttest only nonequivalent control group design (aka static group comparison)
X O (treatment grp)
O (nonequivalent control grp)
X=treatment O=measurement/observation
Because there is no random assignment to groups, confounding variables may explain any difference observed

4. Pre-test/Post-test Non-equivalent Control Group
O X O treatment group
O O non-equivalent control group
Pre Post

5. Example: Research Methods and Reasoning Ability
Intervention: critical thinking seminar
Research Methods students receive the intervention (i.e., participate in the critical thinking seminar)
Developmental Psychology students are used as a nonequivalent control group
(i.e., do not attend the seminar)

6. Interrupted Time Series Designs
Extension of the simple one group pre and post design
Participants are pre-tested a number of times and then post-tested a number of times after being exposed to the treatment intervention
O1 O2 O3 O4 X O5 O6 O7 O8

7. Interrupted Time Series Designs
Useful when:
cannot randomize participants
it is possible to obtain a series of assessments of the DV before and after treatment

8. Example: Interrupted Time Series Designs
Intervention: Course to change students’ study habits, implemented during the summer (after semester 4).
DV: semester GPA
Post-tx
Baseline X

9. How Many Measurements Are Needed for a Time Series Design?
Depends on:
amount of random fluctuation (noise) that may occur in the outcome being measured
how much of an impact the intervention is expected to have

10. Multiple Time Series Designs
Add a comparison group to the simple interrupted time series design:
O1 O2 O3 O4 X O5 O6 O7 O8
O1 O2 O3 O O5 O6 O7 O8
Campbell & Stanley consider this an excellent quasi-experimental design

11. Survey Research 11

12. Surveys Items are composed of 2 parts: stem & response options
The university should not raise tuition under any circumstances (stem)
Agree Disagree (response options) 12

13. Question Wording Important consideration when writing questions:
Simplicity
Avoid jargon and technical terms
Avoid abbreviations
Be specific
Avoid double-barreled questions - 13

14. Responses to Questions
Open-ended
Closed-ended 14

15. Open vs. Closed Questions
“…closed questions should be used where alternative replies are known, are limited in number, and are clear cut. Open ended questions are used where the issue is complex, where relevant dimensions are not known, and where process is being explored.”
Stacey (1969) 15

16. Rating Scales Likert Scales Semantic Differential Graphic Rating Scale
Non-verbal scale 16

17. Question Placement Very important
Confusing questionnaire = demoralizing to respondents
Can affect people’s responses to them 17

18. Layout of Questionnaire
Use closed-ended questions
Use a consistent item format
Don’t crowd questions together
Don’t split questions or response options between pages
It is most likely to be completed if easy on the eye, relevant, logical and as short as possible. 18

19. Response Biases Social desirability
Acquiescence response - tendency either to agree/disagree with statements
Yea-Saying
Nay-Saying 19

20. Checklist for Survey Use
Do the items provide a representative sample of the behaviour under study?
Do the items ask for irrelevant information?
Does the question have the same meaning for researcher and respondent?
Is the question worded clearly? Are the words too difficult?
Does the question concern a sensitive issue? 20

21. Checklist for Survey Use
Does the question suggest a socially desirable answer?
Open vs closed questions?
How many questions are necessary to obtain the required information?
How might the questions be arranged to encourage the respondent’s participation? 21