Download presentation

Presentation is loading. Please wait.

Published byJaylin Croke Modified over 3 years ago

1
Effect Size Robert Coe @ProfCoe ResearchED 2013, Dulwich College, 7 Sept 2013

2
2 The case for using effect size I.What is Effect Size? II.The case for using effect size (5 reasons) III.Problems in using effect size (7 problems) IV.Recommendations (13 recommendations) V.Further reading

3
© 2003 Robert Coe, University of Durham 3 I What is Effect Size?

4
Effect Size is the difference between the two groups, relative to the standard deviation Effect Size = Mean of experimental group – Mean of control group Standard deviation

6
© 2003 Robert Coe, University of Durham 6 II The case for using effect size measures

7
© 2003 Robert Coe, University of Durham 7 1. Effect size allows differences in uncalibrated measures to be interpreted How would you compare/interpret o 5 marks difference on a home-made test o A gain of 8 points on a standardised test o Improvement of 1 sub-level at KS2 o An effect of half a grade at GCSE Comparing each to the spread of values in the population gives it some meaning

8
© 2003 Robert Coe, University of Durham 8 2. Effect size allows the accumulation of knowledge Meta-analysis can combine results from different studies: o What is the average effect? o What factors mediate the effect? o Which interventions have the biggest effects? Small studies are worth doing because they can be added to the pool

9
Impact vs cost Cost per pupil Effect Size (months gain) £0 0 8 £1000 Meta-cognitive Peer tutoring Early Years 1-1 tuition Homework (Secondary) Mentoring Summer schools After school Aspirations Performance pay Teaching assistants Smaller classes Ability grouping Promising May be worth it Not worth it Feedback Phonics Homework (Primary) Collaborative Small gp tuition Parental involvement Individualised learning ICT Behaviour Social www.educationendowmentfoundation.org.uk/toolkit

10
© 2003 Robert Coe, University of Durham 10 3. Effect size emphasises amounts, not just statistical significance The dichotomous “significant/not” decision is almost never appropriate The size of a difference is almost always important “Significance” has many meanings, but is inevitably related to the size of the difference

11
© 2003 Robert Coe, University of Durham 11 Don’t ignore amounts

12
© 2003 Robert Coe, University of Durham 12 4. Effect size draws attention to the margin of error Statistical power is important, but often overlooked Much apparent disagreement is actually just sampling error

13
© 2003 Robert Coe, University of Durham 13 5. Effect size may help reduce reporting bias The “file-drawer” problem is alive and well Within-study reporting bias can also be a problem

14
© 2003 Robert Coe, University of Durham 14 III Problems in using effect size measures

15
Problems with effect size 1.Alternative effect size measures 2.Which standard deviation? 3.Measurement reliability 4.Non-normal distributions 5.Interpreting effects: small, medium, large 6.Incommensurability: outcomes, treatments, populations 7.Is it really an ‘effect’? © 2013 Robert Coe, University of Durham 15

16
© 2003 Robert Coe, University of Durham 16 V Further reading …

17
© 2013 Robert Coe, University of Durham 17 Coe, R. (2002) It's the effect size, stupid: what effect size is and why it is important. Paper presented at the Annual Conference of the British Educational Research Association, University of Exeter, England, 12-14 September 2002. www.cem.org/attachments/ebe/ESguide.pdf Coe, R.J. (2012) ‘Effect Size’ in J. Arthur, M. Waring, R. Coe, and L.V. Hedges (Ed.s) (2012) Research Methods and Methodologies in Education. London: Sage. An introduction, guide and tool for calculating effect sizes: www.cem.org/evidence-based-education/effect-size-resources 1 2 3

18
© 2013 Robert Coe, University of Durham 18 http://www.nfer.ac.uk/nfer/publications/SEF01/SEF01.pdf Coe, R. (2004) ‘Issues arising from the use of effect sizes in analysing and reporting research’ in I. Schagen and K. Elliot (Eds) But what does it mean? The use of effect sizes in educational research. Slough, UK: National Foundation for Educational Research. 4

Similar presentations

OK

(This presentation may be used for instructional purposes)

(This presentation may be used for instructional purposes)

© 2018 SlidePlayer.com Inc.

All rights reserved.

Ads by Google

Download ppt on height and distance for class 10 Ppt on culture of goa Ppt on data transmission by laser Ppt on business plan for new business Ppt on levels of organization in biology Ppt on forward rate agreements Ppt on different types of transport Ppt on depth first search java Ppt on 4 p's of marketing mix Ppt on exponent and power