Presentation is loading. Please wait.

Presentation is loading. Please wait.

Experimental Design Making your experiment more valid and more efficient.

Similar presentations


Presentation on theme: "Experimental Design Making your experiment more valid and more efficient."— Presentation transcript:

1 Experimental Design Making your experiment more valid and more efficient

2 So, the basic experimental procedure in social science is: Gathering of a subject pool and then random assignment to conditions – Note: the sample is not random, but once you have a subject pool you flip a coin, etc. to determine which subjects receive which condition Depending upon the experimental design, a measure of the dependent variable may be taken from the groups prior to the manipulation

3 Subjects are exposed to the ‘treatment’ assigned to their group – May be some level of the factor or a zero-level (control) treatment – For example, the ‘’experimental group” may play a video game with lots of violence while the control group plays a non-violent game Both groups are then measured on the dependent variable of interest – In this case, some measure of aggressiveness The data are then analyzed to test the research hypothesis

4 Basic design (After-only, control group design) RXORO R=randomization, X=manipulation, O=observation

5 What’s the effect? The difference between group scores on the dependent variable is assumed to be due to the manipulation But what if the groups were different at the start? – Randomization should have taken care of that – However, pure chance can sometimes lead to differences at the outset So, measure them before the manipulation as well as after

6 Before-after, control group design RXO1O2 R O1 R=randomization, X=manipulation, O=observation

7 Before-after, control group design Now there’s a new problem: – Measuring the subjects and then measuring them again later may cause them to respond differently to the second measure than they otherwise would have (true for both experimental and control subjects) – Measuring the experimental subjects before exposing them to the treatment may change their response to the manipulation

8 Problems with the basic design The difference between control and experimental conditions includes more than violence – Experimenters may have control group play a nonviolent video game Many other variables may affect the outcome – For example, gender Measure and include in statistical analyses Use factorial design

9 So: There will always be some problems with any procedure The goal is to minimize the threats to validity as best your circumstances (including budget) allow – Especially internal validity

10 How do we do that? Experimental design Statistical analyses Careful application of procedures

11 Experimental design Two facets – Choices in identifying the independent and dependent variables to be measured and/or manipulated – Choices in developing the procedures for manipulation, measurement, environment of testing, etc.

12 Choosing the variables Theory determines variables manipulated and measured Experimental designs where the research manipulates more than one independent variable are known as factorial designs – Mediating concepts often are included

13 Choosing the measures The measures are chosen to best represent the concepts in the original theory – Measurement validity Must work in an experimental setting – Lengthy questionnaires unlikely to work – Physical measures more easily applied in a laboratory setting

14 Factorial designs Factorial designs are useful because they: – Are efficient With two independent variables we can test three hypotheses instead of just one – Don’t need three times as many subjects However, there is a practical limit to independent variables – As the number rises, the required number of subjects increases quickly (5 per cell) – Help us avoid coming to invalid conclusions They allow us to control for potentially confounding variables and interactions – We can test the impact of third variables Factorial designs can be more difficult to administer

15 An example We may want to test the following hypothesis: – “Exposure to Dancing with the Stars will lead to greater liking for the contestants” We might, however, expect that men and women will be affected differently by the show So, we might add gender as another independent variable in the design

16 Factorial design Women See Dancing with the Stars Don’t see Dancing with the Stars All groups are measured on liking for contestants Men

17 What we can test “Main effect” of exposure to Dancing with the Stars “Main effect” of gender “Interaction effect” of exposure by gender Remember: the dependent variable is liking for contestants

18 Results (% who like contestants)

19 Conclusions Main effect of gender – Yes: Gender affects liking for celebs Main effect of exposure to DWTS – Yes: Exposure increases liking Interaction effect of exposure by gender: – Yes: Exposure has a clear positive effect for women, little or even negative effect for men

20 Procedural choices How many times will you measure the subjects? – You can account for chance fluctuation by increasing the number of times the groups are measured You increase the likelihood of testing reactivity, though

21 RO1O2XO3O4RO1O2O3O4

22 Additional design concerns How many times/how long will the manipulation be presented? – Testing a single exposure to DWTS may not be valid Viewers may watch many shows during a season

23 RO1X1O2X2O3RO1O2O3

24 Removal of the treatment RO1XO2-XO3RO1O2O3

25 How many levels of the independent variable will be presented? – May want to test high v. medium v. low levels of violence, etc.

26 RO X (high) O ROX (low) O RO X (medium) O ROO

27 Factorial nomenclature The number of treatments on each factor are crossed – 2X3 factorial design has 2 treatment levels on one factor and 3 treatment levels on another factor – If we tested the effects of soft, medium and loud music and gender upon ease in a social situation, we would have a 2X3 factorial study Factor 1: music volume 3 levels: soft, medium and loud Factor 2: gender 2 levels: male, female Dependent variable: ease in a social situation

28 Additional design concerns Artificiality of the testing environment – Labs allow for the greatest control over third variables Volume, distance from screen, prevent other people from walking by, etc. – Lab experience can sometimes be made less artificial Some commercial-testing services provide the subjects with a simulated living room for exposure Loss of control over third variables weighed against increase in generalizability (external validity)

29 Statistical analyses If you measure potential confounds you can use statistical procedures to remove their impact on the dependent variable

30 Careful application of the design Unplanned variations among treatments can easily occur when conducting experiments in the real world – The person you wanted to greet everyone and provide instructions may have to leave for an emergency – Subjects may talk to each other outside the lab before they have all completed their part of the experiment – And so on

31 Remember: Experiments are the most effective quantitative research method for testing hypothesized causal relationships Experiments emphasize internal validity but are often weak with regard to external validity Many different experimental designs exist, each having a set of advantages and disadvantages Your goal is to provide the most valid test of theory your circumstances allow you


Download ppt "Experimental Design Making your experiment more valid and more efficient."

Similar presentations


Ads by Google