Experiments: Part 1

Overview How do experiments differ from observational studies? What are the three main variables we need to consider in experimental research? What are the similarities and differences between between-group and within-subject experiments?

Background on Experiments Study where a researcher systematically manipulates one variable in order to examine its effect(s) on one or more other variables Two components (2nd-most important point of this course) Includes two or more groups Participants are randomly assigned by the researcher Random = Equal odds of being in any particular group Examples People with GAD randomly assigned to three treatments so the researchers can examine which one best reduces anxiety Students assigned to a “mortality salience” or control condition so the research can examine the impact on “war support”

Variables Independent Variable Dependent Variable Manipulated by the researcher Typically categorical Also called a “factor” that has “levels” Factor = Type of anxiety treatment Level = CBT (or Psychodynamic or Control) Dependent Variable Outcome variable that is presumably influenced by (depends on the effects of) the independent variable Behavior frequencies, mood, attitudes, symptoms Typically continuous

Variables Confounds (extraneous variables, 3rd variables) Happens when unwanted differences (age, gender, researchers, environments, etc.) across experimental conditions Plan: Think of potential confounds up front Control for them methodologically Measure them to examine whether they have an effect Control for them statistically

Experimental Designs Two basic designs Between-group design Also called a “between-subjects design,” or “randomized controlled trial” (if clinically focused) Within-subject design Also called a “repeated-measures design”

Between-group Design IV: Type of group Randomization: Different people randomized to different groups DV: Usually a continuous variable

Within-subject Design IV: Type of group Randomization: Each participant goes through more than one group, with order randomly assigned DV: Usually a continuous variable, assessed repeatedly over time Example: Participants go through more than one experimental condition

Similarities Uses the same type of analyses p-values obtained from t-tests (if two conditions) or F-tests/ANOVA (if more than two conditions) Is the result statistically significant, reliable, trustworthy? Cohen’s d used to compute effect size Tells the number of standard deviations by which two groups differ (kind of like r but on a scale from -∞ to ∞) Effect r r2 d Small ≥ .1 ≥ .01 ≥ 0.2 Medium ≥ .3 ≥ .09 ≥ 0.5 Large ≥ .5 ≥ .25 ≥ 0.8

Cohen’s d Calculator http://www.psychmike.com/calculators.php Usually use the first formula, requires M, SD, and n Can calculate by hand with a simple formula, but it doesn’t account for differences in sample size across conditions, so less accurate d = = (Mean difference) / standard deviation s = average standard deviation across groups

+/- sign is arbitrary, so usually just dropped Calculation Example: Does athletic involvement improve physical health? M1 = 6.47 M2 = 6.75 s = (1.87+1.94) / 2 = 1.91   d = (6.47 – 6.75) / 1.91 = -0.28 / 1.91 = -0.15 = 0.15 weak effect! +/- sign is arbitrary, so usually just dropped

2014 article in Lancet (impact factor: 45 2014 article in Lancet (impact factor: 45.2) Take-home from the abstract:

Differences Between-group design required when it is impossible or impractical to put participants through more than one condition Within-subject design is more powerful More likely to get significant p-value and bigger effect sizes. Why? It allows each participant to serve as their own control, canceling out a lot of cross-participant variability Between-group design requires more people Within-subject design is prone to ordering effects (order of conditions can effect results), such as progressive effects, or carryover effects Solution: Counterbalancing