Download presentation

Presentation is loading. Please wait.

Published byZackary Brann Modified over 3 years ago

1
What is meta-analysis? ESRC Research Methods Festival Oxford 8 th July, 2010 Professor Steven Higgins Durham University s.e.higgins@durham.ac.uk

2
Key points Understanding ‘effect-size’ Why do we need it? What are its limitations? What is its potential?

3
What is an “effect size”? Standardised way of looking at difference Different methods for calculation Odds Ratio Correlational (Pearson’s r) Standardised mean difference Difference between control and intervention group as proportion of the dispersion of scores

4
Examples of Effect Sizes: ES = 0.2 “Equivalent to the difference in heights between 15 and 16 year old girls” 58% of control group below mean of experimental group Probability you could guess which group a person was in = 0.54 Change in the proportion above a given threshold: from 50% to 58% or from 75% to 81%

5
“Equivalent to the difference in heights between 14 and 18 year old girls” 69% of control group below mean of experimental group Probability you could guess which group a person was in = 0.60 ES = 0.5 Change in the proportion above a given threshold: from 50% to 69% or from 75% to 88%

6
“Equivalent to the difference in heights between 13 and 18 year old girls” 79% of control group below mean of experimental group Probability you could guess which group a person was in = 0.66 ES = 0.8 Change in the proportion above a given threshold: from 50% to 79% or from 75% to 93%

7
Significance versus effect size Traditional test is of statistical ‘significance’ The difference is unlikely to have occurred by chance However it may not be: Large Important, or even Educationally ‘significant’

8
The rationale for using effect sizes Traditional reviews focus on statistical significance testing Highly dependent on sample size Null finding does not carry the same “weight” as a significant finding Meta-analysis focuses on the direction and magnitude of the effects across studies From “Is there a difference?” to “How big is the difference?” and “How consistent is the difference?” Direction and magnitude represented by “effect size”

9
Meta-analysis Synthesis of quantitative data Cumulative Comparative Correlational “Surveys” educational research (Lipsey and Wilson, 2001)

10
Forest plots Effective way of presenting results Studies, effect sizes, confidence intervals Provides an overview of consistency of effects Summarises an overall effect (with confidence interval) Useful visual model of a meta-analysis

11
Anatomy of a forest plot… Studies N of study Line of no effect C.I Study effect size Pooled effect size Study effect size (with C.I.) Weighting of study in meta- analysis

12
Issues and challenges in meta-analysis Conceptual Reductionist - the answer is 42 Comparability - apples and oranges Atheoretical - ‘flat-earth’ Technical Heterogeneity Publication bias Methodological quality

13
Schulze, R. (2007) The state and the art of meta-analysis Zeitschrift fur Psychologie/ Journal of Psychology, 215 pp 87 - 89.

14
RDI in Quantitative Synthesis Collaboration between the Universities of Durham, Birmingham and the Institute of Education, University of London Rob Coe, Mark Newman, James Thomas and Carole Torgerson Levels 1 and 2: Intro and practical training Durham, Edinburgh London, Belfast, York, Cardiff Level 3: expert workshops Prof Mark Lipsey, Prof Larry Hedges, Doctoral support through BERA

15
References, further readings and information Books and articles Borenstein, M., Hedges, L.V., Higgins, J.P.T. & Rothstein, H.R. (2009) Introduction to Meta Analysis (Statistics in Practice) Oxford: Wiley Blackwell. Chambers, E.A. (2004). An introduction to meta-analysis with articles from the Journal of Educational Research (1992-2002). Journal of Educational Research, 98, pp 35-44. Cooper, H.M. (1982) Scientific Guidelines for Conducting Integrative Research Reviews Review Of Educational Research 52; 291. *Cooper, H.M. (2009) Research Synthesis and meta-analysis: a step-by-step approach London: SAGE Publications (4th Edition). Cronbach, L. J., Ambron, S. R., Dornbusch, S. M., Hess, R.O., Hornik, R. C., Phillips, D. C., Walker, D. F., & Weiner, S. S. (1980). Toward reform of program evaluation: Aims, methods, and institutional arrangements. San Francisco, Ca.: Jossey-Bass. Glass, G.V. (2000). Meta-analysis at 25. Available at: http://glass.ed.asu.edu/gene/papers/meta25.html (accessed 9/9/08)http://glass.ed.asu.edu/gene/papers/meta25.html Lipsey, Mark W., and Wilson, David B. (2001). Practical Meta-Analysis. Applied Social Research Methods Series (Vol. 49). Thousand Oaks, CA: SAGE Publications. Torgerson, C. (2003) Systematic Reviews and Meta-Analysis (Continuum Research Methods) London: Continuum Press. Websites What is an effect size?, by Rob Coe: http://www.cemcentre.org/evidence-based-education/effect-size-resourceshttp://www.cemcentre.org/evidence-based-education/effect-size-resources The meta-analysis of research studies: http://echo.edres.org:8080/meta/http://echo.edres.org:8080/meta/ The Meta-Analysis Unit, University of Murcia: http://www.um.es/metaanalysis/http://www.um.es/metaanalysis/ The PsychWiki: Meta-analysis: http://www.psychwiki.com/wiki/Meta-analysishttp://www.psychwiki.com/wiki/Meta-analysis Meta-Analysis in Educational Research: http://www.dur.ac.uk/education/meta-ed/http://www.dur.ac.uk/education/meta-ed/

Similar presentations

OK

Statistical Applications for Meta-Analysis Robert M. Bernard Centre for the Study of Learning and Performance and CanKnow Concordia University December.

Statistical Applications for Meta-Analysis Robert M. Bernard Centre for the Study of Learning and Performance and CanKnow Concordia University December.

© 2018 SlidePlayer.com Inc.

All rights reserved.

Ads by Google

Ppt on piezoelectric power generation Ppt on area of parallelogram and triangles in real life Ppt on shell scripting pdf Jit ppt on manufacturing companies Ppt on different occupations in the medical field Ppt on porter's five forces analysis template Ppt online download for youtube Ppt on 3d handwriting technology Ppt on amway business plan Ppt on computer malwares bytes