Presentation is loading. Please wait.

Presentation is loading. Please wait.

Research Problems.

Similar presentations

Presentation on theme: "Research Problems."— Presentation transcript:

1 Research Problems

2 Hawthorne Effect Western Electric’s Hawthorne Plant 1939 study of light intensity productivity went…up Potential Solutions: run experiment for a longer period use a control group

3 John Henry Effect legend of black railway worker
control group overcompensates Potential Solutions: don’t do threatening experiments don’t set up obviously competitive situations don’t tell control group that they are control group conduct in another school somewhere else unfortunately, produces new variable of different school, neighbourhood, etc.!

4 Placebo Effect introduce placebo in attempt to make conditions in treatment and control groups identical placebo effects are reactions that (after taking the placebo) cannot be explained by the chemical or medical effects of the placebo. psychological factors

5 Placebo Effect Potential Solutions
double-blind experiment secrecy but then violate principle of informed consent screen out or balance number of placebo reactors in treatment & control groups

6 the Pygmalion effect Rosenthal and Jacobson, 1968
self-fulfilling prophecy Potential solution: do not tell subjects what experiment is about but what happens to principle of informed consent?

7 Demand Characteristics
rumour setting instructions status and personality of researcher unintentional cues from experimenter experimental procedure itself

8 Demand Characteristics Potential Solutions
reduce clarity of demand characteristics generate alternative demand characteristics reduce subjects’ motivation to respond to demand characteristics

9 Validity Internal Validity External Validity
control experiment to eliminate extraneous variables External Validity outcome can be generalized to other populations in other settings

10 Threats to Internal Validity
History change producing events in addition to the experimental treatment Maturation subjects grow older, learn more, can do more

11 Threats to Internal Validity
Testing in pre-test/post-test designs — subjects may remember questions from pre-test Instrumentation changes in measurement changes in observer changes in subjects (testing effects)

12 Threats to Internal Validity
Differential Selection Differences between treatment and control group E.g., volunteers vs non-volunteers Experimental Mortality (or Attrition) subjects that drop out of experiment may differ from others in important ways

13 Threats to Internal Validity
Testing & Experimental Treatment Interaction In pre-test/post-test design, pretest may interact with experimental treatment to exaggerate result Statistical Regression

14 Threats to Internal Validity
Not Paying Attention to What’s Really Going On

15 External Validity 1. Population Validity 2. Ecological validity
can results be generalized from specific sample to the population from which sample was drawn 2. Ecological validity can results be generalized from contrived conditions created by experimenter to another set of environmental conditions (i.e., real world)

16 Experimental Design

17 Experimental Design Purpose is to design an experiment that controls for as many extraneous variables as possible Campbell and Cook, classic 1968 paper categorized experimental designs by how many variables they controlled

18 Weak Designs Single Group Designs

19 The One-Shot Case Design

20 The One-Shot Case Design
hardly experiment at all can’t be sure result was result of treatment, not history, maturation, etc. no control over group selection could have problem with subject mortality (e.g., transient student population) since only tested once, can’t even measure gain — maybe students knew it before we started consequently, results are largely meaningless

21 One Group Pre-test/Post-test Design

22 One Group Pre-test/Post-test Design
Uncontrolled Extraneous Factors (all of them!) history e.g., Hawthorne effect maturation testing Pre-test may have contributed to higher scores on post-test due to greater familiarity with types of questions or focus on certain topics instrumentation pre-test and post-test the same? Observer the same? selection: could be an atypical group mortality interaction of testing & experimental treatment statistical regression

23 One Group Pre-test/Post-test Design
useful for studying stable dependent variable justified when extraneous factors can be estimated with a high degree of certainty or can be safely assumed to be nonexistent E.g., not maturation because change is too dramatic for maturation to explain it

24 Control Group Designs

25 Posttest-Only Nonequivalent Groups Design

26 Posttest-Only Nonequivalent Groups Design
also called “static group comparisons” still a relatively weak design advantage over single one shot design that one can compare, so cancel out maturation, etc. Problems: Differential selection experimental mortality

27 Non-Equivalent Control Group Design

28 Non-Equivalent Control Group Design
stronger design (weakest of the strong designs) common design in education because usually can’t randomize assignment of students to classes pre-test measures whether initial groups are similar on tested variable could also match subjects based on pre-test but may miss other factors which could impact results (e.g., better teacher in one group)

29 Non-Equivalent Control Group Design
Factors controlled by inclusion of a control group history maturation testing instrumentation (assuming same for both groups) statistical regression Factors still NOT controlled differential selection experimental mortality (if any) testing and experimental treatment interaction

30 Pre-test/Post-test Control Group Design

31 Pre-test/Post-test Control Group Design
still two uncontrolled factors Intersession History events that are specific to one group and not the other which occur during the group sessions To control for interssion history, need to balance (control) such factors as experimenters time of day day of week by randomly assigning to groups of subjects

32 Pre-test/Post-test Control Group Design
Interaction of testing and treatment Cannot be avoided when there is a pretest eliminating the pre-test would therefore improve design

33 The Post-test Only Control Group Design

34 The Post-test Only Control Group Design
do not need pre-test with random selection disadvantages random assignment may not be fully successful in eliminating initial differences between control and experimental groups cannot form subgroups (i.e., high, medium, & low) to determine whether the experimental treatment has a different effect on subjects at different levels of the variable as measured by the pretest (because no pretest)

35 Solomon Four-Group Design

36 Solomon Four-Group Design
basically, a pretest and non-pretest experiment at the same time controls for everything, except differential mortality

37 Time Series Design

38 Time Series Design Similar to one-group pre-test /post-test design (weak #2) but additional pre- and post- measurements add power additional measurements enable researcher to rule out maturation testing effects as sources of influence shift from pretest to posttest

Download ppt "Research Problems."

Similar presentations

Ads by Google