1. EXPERIMENTS AND EXPERIMENTAL DESIGN
ETHICAL ISSUES IN RESEARCH

2. If the stimulus is given, then the outcome occurs
Logic of causality
If X, then Y
and
If not X, then not Y
Or, colloquially:
If the stimulus is given, then the outcome occurs
If the stimulus is not given, then the outcome does not occur

3. Basic requirements of a true experiment:
- At least 2 groups: one experimental & one control group - Random assignment of subjects to groups. Every case has: (1) the same chance of being selected (i.e. equal probability) (2) the chance of selection does not change (i.e. constant probability) - Manipulation of the IV. - Measurement of the DV. - Except for the experimental manipulation, the 2 groups = treated exactly alike
Manipulation checks: series of experiments in which same theoretical concept = operationalzied differently; ask subjects (orally/written) what they felt/thought during or immediately after the experimental maipulation
Can also be used to see if subjects understood/recalled essentaial directions/facts related to manipulation
A) evaluation check after the IV = manipulated, but before DV = measured (manipulation = fresh and so si R memory;
But: may not be feasable; OR reactivity (macip check may alter subjects subsequent behavior (DV)

4. Results
controlling for pre-existing subject differences; differences are to be neutralized/distributed approximately evenly among the groups  random assignment of subjects to treatment & control groups
- making sure that the events occurring within each experimental group are exactly the same, except for the manipulated independent variable  controlling for extraneous factors & experiences during the experiment

5. Randomization for groups

6. Tests
Use tests of statistical significance to determine the likelihood that the results occurred by chance rather than being caused by the treatment variable (i.e. significant differences btw. the groups).
A result is statistically significant at the 0.05 level:
it means that only 5% of the time (5 times in 100 if we were to repeat the experiment) would differences btw. the experimental & control groups occur by chance when the experimental variable actually has no effect.

7. Tests
Differences between the mean values for T and C
The differences between slopes for T and C

8. Mean values for T and C

9. Slope

10. Staging of an experiment
 1. Introduction: Instructions and Cover Story
2. Manipulation of the IV
3. Measurement of the DV (verbal reports; behavioral observation)
4. Debriefing (explain purpose of experiment; explore subjects’ reactions; alleviate subjects’ anxiety about their performance/ participation)

11. Validity
Internal Validity: the approximate truth about inferences regarding the tested causal relationships in the experimental setting
External Validity: the degree to which the conclusions in the study would hold for other persons in other places & at other times (i.e. how generalizable are the results of the study)

12. Issues to consider
Pre-testing of experiments
Laboratory experiments vs. experiments in natural settings (field experiments)
Trade-off btw. increase in external validity & control over extraneous variables & manipulation of the IV
Subjects’ reaction to the social meaning of the situation: often a problem for laboratory experiments

13. Types of experiments
Pre-test – post-test control group design
Post-test-only control group design
The Solomon four group design
Factorial design
Quasi-experimental design
Experimental Design in Survey

14. Pre-test – post-test control group design
Tt1 X Tt2 R Ct1 -- Ct2 R = random assignment of subjects to groups T = Treatment group, C = Control group t1 = first measurement, before experimental treatment (pretest), t2 = 2nd measurement, after experimental treatment (post-test)
Possible problem regarding external validity: testing-treatment interaction. The effect of the IV may be different when a pretest is present than when it is not (the subjects have been sensitized to the experimental treatment through pre-testing; thus, in real life the effect would not occur). Depends on experimental situation: in education settings, should not matter; in studies of attitude change, however, or persuasion, pretest ma alert subjects and make them more receptive/resistant).

15. Post-test-only control group design
X Tt1 R
Ct1
Many variations on this experimental design:
See - various experimental designs.

16. The Solomon four group design
Tat X Tat2
Cat Cat2 R
X Tbt1
Cbt1

17. Outcome I
T =with manipulated IV
C = control group

18. Outcome II

19. Source: www.socialresearchmethods.net Factorial design
extension of the basic experimental design, in which 2 or more IV are manipulated (each manipulated IV is called a factor). Factorial design has several important features: great flexibility for exploring or enhancing the “signal” (treatment) in studies; efficient; the only effective way to examine interaction effects.
A level = subdivision of a factor: in this example, time in instruction has two levels and setting has two levels.

20. The Main Effects and Interaction Effects
A main effect is an outcome that is a consistent difference between levels of a factor.
Because of the way we combine levels in factorial design, it also enables us to examine the interaction effects that exist between factors. An interaction effect exists when differences on one factor depend on the level of another factor.

21. Explanation of effects, I
A main effect is an outcome that is a consistent difference between levels of a factor. For instance, we would say there’s a main effect for setting if we find a statistical difference between the averages for the in-class and pull-out groups, at all levels of time in instruction. The first figure depicts a main effect of time. For all settings, the 4 hour/week condition worked better than the 1 hour/week one. It is also possible to have a main effect for setting (and none for time).
the outcome might be a test of achievement in the subject being taught.
4 groups; 2 are trained in class; 2 outside; and they are divided by number of hours of training.
In the green cells, you have the average results of testing after training is completed. Groups are equal in size.
In the orange/brownish cells (marginal values) you have the average testing grade by setting (rows), and by time (column), respectively

22. Explanation of effects, II

23. Explanation of effects, III

24. Explanation of effects, IV
An interaction effect exists when differences on one factor depend on the level you are on another factor. It's important to recognize that an interaction is between factors, not levels.

25. Explanation of effects, V
The point: margins are the same (no separate effect of time, and no separate effect of setting – no. are equal) while we have these effects if we take the data for groups into account (as opposed to margins)

26. Quasi-experimental design (QED)
QED takes an experimental approach without having full experimental control; lacks some feature of true experiments, generally randomization
- Separate sample pretest-posttest design (both groups receive the treatment, but one is tested before the treatment, and the other afterward)
- None-equivalent control group design; control group but without random assignment;
- Interrupted time series-design: series of observations before and after the treatment;
- Multiple time series design: extends the one above by adding a series of observations on a non-equivalent control group.

27. Experimental Design in Surveys
Differently worded questions: As part of a larger survey: direct a question with one wording to a randomly selected sub-sample of respondents; and a differently worded question to the remaining respondents.
Vignettes: Short, detailed and concrete descriptions of situations that contain references to the factors that are thought to be important in decision making.
To test experimentally which factors affect the subject’s decision making, a number of versions of the vignettes are developed by systematically varying the details of the situation. The different versions are then randomly assigned to respondents.
Constructing different sets of questionnaires

28. ETHICAL ISSUES IN RESEARCH
Basic question:
For what to to whoe the researchers in the social sciences are resposible?
Human subjests
Peers
Society

29. Ethics of treatment of human subjects
- Ethical considerations preclude withholding treatment from a control group: education, correction, clinical psychology, medicine frequently the case.
Institutional review boards assess the ethics of the proposed research (including on risk vs. benefits). Overall guiding principle: potential benefits of research should be weighed against potential costs (risks).

30. General rules:
- Researcher should not expose participants to substantial risk of physical or psychological harm, unless the benefits of participation exceed the risks, and the subject knowingly chooses to participate;
- Subjects should be informed that their participation is voluntary throughout the entire research; should be told about any aspects of the research that may influence their willingness to participate;
- If deception is necessary- appropriate debriefing as soon as possible after their participation;
- Confidentiality and anonymity: researchers should use all possible means to protect the confidentiality of information participants provide (identity).

31. Ethics of data collection and analysis
- Honesty and accuracy in collection & analysis of data, and reporting of results
- Scientific misconduct, if identified, kills researcher’s career

32. Ethics of responsibility to society
Awareness of potential use and misuse of scientific results by others