Meta-analysis

Overview Definition  A meta-analysis statistically combines the results of several studies that address a shared research hypotheses.  A study collects data from individual subjects (such as 100 subjects = 100 “data points”)  A meta-analysis collects data from individual studies (such as 100 studies = 100 “data points”)

Step1 Defines your hypothesis  e.g., Does “authoritarianism” predict verdict choice? relationship between X (authoritarianism) and Y (verdict)

Step 2 Locate studies  e.g., Sannito & Arnolds (1982) McGowen & King (1982) Boehm (1968) etc.

Step 3 Find “effect size” for each study  e.g., convert data (means, p-value, etc) to “r”

Step 4 Average the “effect sizes” together  e.g., weight by sample size, then take mean

Step 5 If you want, you can analyze “moderators”  e.g., does the “average effect size” get smaller or bigger based upon factors like if the studies have actual jurors versus college students, or if the studies employ written summaries versus actual trials, etc.

Why do a meta-analysis? Easy  Steps are simple, there is software to calculate everything Cost-effective  Since you have already read a bunch of articles to write a paper, not much more work to synthesize them together Best type of article  Most highly cited type of article.  Advantages of both qualitative and quantitative research  Truly answers research questions within the literature (compared to single studies which can’t truly generalize)

Step1 (again) Defines your hypothesis  Typically the hypothesis is relationship between X and Y  Best to find studies with diversity (e.g., findings that vary in size, involve multiple IVs or DVs or stimuli, etc)  Best to find studies where controversies or inconsistencies exist (which the meta-analysis can resolve)  From practical point of view, ideally choose a topic with enough primary studies (20-40) but not too many studies (100+) because hardest part is finding the studies (step 2)

Step 2 (again) Locate studies  A meta-analysis is only informative if it adequately summarizes the existing literature  Techniques - database searches, ancestry approach, descendancy approach, hand searching, invisible college  Doesn’t have to be comprehensive (fail-safe n) but needs to be close to comprehensive

Step 3 (again) Find “effect size” for each study  For every test that we have covered in this class, you can convert all of them to “r” or “d”  Download “es_calculator.zip” from  In an excel file, have a separate row for each study in which you input information such as sample size, effect size, codings from moderators, etc. Example -

“r” and “d” The r family  Correlation Coefficient - The "r" family includes all types of correlation coefficients (e.g., r, phi, rho, etc) and Johnson & Eagly, 2000 suggest using r when the studies composing the meta analysis primarily report the correlation between variables.Johnson & Eagly, 2000 The d family  Standardized Difference - The "d" family includes Cohen's d (unweighted) and Hedges g (weighted), and Johnson & Eagly, 2000 suggest using d when the studies composing the meta-analysis primarily report ANOVAs and t-tests comparisons between groups.Johnson & Eagly, 2000

Step 4 (again) Average the “effect sizes” together  First, weight them by sample size  Second, sum them together  Third, divide by sum of total sample size.  Macros for SPSS can be downloaded from:

Step 5 (again) If you want, you can analyze “moderators”  First, need to ascertain “homogeneity” which tells you if variance exists in average effect size  Can test categorical moderators (categories like college student versus actual juror) similar to ANOVA  Can test continuous moderators (such as length of stimulus) similar to regression  Use macros downloaded from

For more information… PsychWiki -

Some cool things about meta-analyses…