Experimental Research

What is experimental research?  Research investigation in which conditions are controlled so that hypotheses can be tested and alternative explanations can be ruled out  Research used to make “cause-and-effect” statements (X causes Y) X is the independent (or manipulated or causal) variable Y is the dependent variable

Experimental Research Evidence of causality (i.e., X causes Y)  Evidence of concomitant variation (X and Y co- vary) The more of X, the better chance that we will get Y  Time order sequence of variables If X causes Y, X must occur before Y  Elimination of other possible explanations of why Y occurred Key is to keep all experimental conditions equal

Experimental Research Must try to eliminate all possible extraneous influences Possible extraneous influences  History - event occurring during the course of an experiment (but not really part of the experimental manipulation) that influences the results  Maturation - changes which occur in the experimental unit which occur during the course of an experiment that are due to the passage of time

Experimental Research Possible extraneous influences (continued)  Testing Effects - changes in experimental unit due to the experiment itself (but not related to the key experimental manipulation Measurement effect -- prior measurements effect the measure of the dependent variable Interaction effects -- subject pays greater attention to stimuli than they normally would because its and experiment

Experimental Research Possible extraneous influences (continued)  Instrument variation -- changes in the measuring instrument cause changes in Y (i.e, interviewer changes the way in which they ask questions)  Selection Bias -- if two groups are compared, this notion suggests that the groups may not have been equivalent to begin with; i.e., a manipulation did not cause Y; group dissimilarities did

Experimental Research Possible extraneous influences (continued)  Experimental mortality -- respondents in experimental groups were lost during the duration of the experiment  Statistical Regression -- Extreme responses move closer to the midpoint during the course of an experiment; subjects after questionning do not want to appear to be extreme

Experimental Research Experimental Symbols  X = Experimental Treatment (e.g., ad exposure)  O = Observation  R = Random Assignment of Subjects Pre - Experimental Designs  One-Shot Case Study X O Problems -- Selection Bias, Control Group, Mortality, History, etc.

Experimental Research Pre-Experimental Designs (Cont’d)  One Group Pre-Test/Post-Test Design O 1 X O 2 Analysis -- O 2 - O 1 Problems -- History, Maturation, interactive Testing Effect, etc.  Static Comparison X O 1 O 2 Analysis -- O 2 - O 1 Problems -- Selection Bias, Mortality, etc.

Experimental Research True Experimental Designs  Before/After with Control Group (R) O 1 X O 2 (R) O 3 O 4 Analysis (O 4 – O 3 ) – (O 2 – O 1 ) Lessens History and Maturation Problems However, interactive testing effect still an issue  O 2 and O 4 may a function of O 1 and O 3

Experimental Research Experimental Designs (True Experimental Designs)  Six Group - Four Study Design (R) O 1 X O 2 (R) O 3 O 4 (R) X O 5 (R) O 6 O 2 – O 1 compared to O 4 – O 3 tells if X “worked” (manipulation check) – If yes, go on If O 5 > O 6 – we have true effects

Experimental Research True experiments are very complex Experimental designs that are used are often flawed (or not “True”) Random assignment helps

Experimental Research Analyzing Data Typically uses Analysis of Variance (ANOVA)  Tests (determines if at least one treatment group mean is different from the others) Ho:Μ 1 = M 2 = M 3 = … = M k Ha:M 1 ≠ M 2 ≠ M 3 ≠ … ≠ M k  Why not use t-tests? Probability  One comparison at 95% confidence – 95% of rejecting null when it should be rejected  Two comparisons x.95 = 90.25%  Three comparisons x.95 x.95 = 86%  Like regression Dependent Variable (interval or ratio scaled) Independent Variable (nominal or ordinal scaled)  Often group membership

Experimental Research (Responses on 1 = Bad; 7 = Outstanding) Treat/RespAd 1Ad 2Ad Mean

Experimental Research One-Way ANOVA Attempts to partition variance  Total Variance (Within Group + Between Group) Within treatment groups (Observation – Group Mean)  SS w = Σ Σ (x ij – Mean i ) 2  MS w = SS w / df w Between treatment groups (Group Mean – Grand Mean)  SS b = Σ n j (Mean j – Mean i ) 2  MS b = SS b / df b F-statistic  MS b / MS w  With df b, ; df w Follow-up test  Tukey (most common in MR)

Experimental Research Additional Issues with ANOVA Real World  Often used with survey data Compare means of several groups DV – is interval (or ratio) scaled IV -- categorical Two-Way ANOVA  One dependent variable Two or more independent variables Simultaneous effects MANOVA  Multiple dependent variables ANCOVA  Investigate effects after “controlling” for another variable (that is interval or ratio scaled)

Experimental Research Compare means of several groups  EXAMPLE (Survey Data) CoolWarm Permissive Neglecting Indulgent Restrictive Authoritarian Authoritative

Experimental Research DV = Parent Responsibility to Restrict TV Choices for Kids  F= 4.42; p =.005 Neglecting (M = 29.5) Indulgent (M = 31.26) Authoritarian (M = 30.26) Authoritative (M = 32.27)  Follow-up Tests Authoritative > Authoritarian & Neglecting Indulgent > Neglecting Authoritarian = Neglecting

Experimental Research Two-Way ANOVA – Example  Main Effects Similar to one-way ANOVA effect (for 2 variables)  Interactive Effects Main Effects look null Differences observed only when looking at both factors simultaneously

Experimental Research Two-Way ANOVA  One dependent variable – Main Effects P N NamedNot-Named

Experimental Research Two-Way ANOVA  One dependent variable – Interaction Effects P N NamedNot-Named