Presentation is loading. Please wait.

Presentation is loading. Please wait.

1 The Appropriate Use of NAPLAN Data National Symposium, 23 July, 2010 Margaret Wu University of Melbourne 1.

Similar presentations


Presentation on theme: "1 The Appropriate Use of NAPLAN Data National Symposium, 23 July, 2010 Margaret Wu University of Melbourne 1."— Presentation transcript:

1 1 The Appropriate Use of NAPLAN Data National Symposium, 23 July, 2010 Margaret Wu University of Melbourne 1

2 2 NAPLAN Tests Conducted once a year About 40 test questions per subject area Test scores are used to infer ◦ the achievement levels of students How reliable can NAPLAN test scores reflect ◦ Student achievement level? ◦ School performance? 2

3 3 Margin of error in measuring student performance David - a Grade 5 student in Reading score was 25 out of 40. David’s reading test scores could vary between 20 and 30, out of 40. ◦ if similar tests are administered (e.g., 2009, 2010 tests ) One test collects only a small sample of performance. Variation in scores is called Measurement Error. 3

4 4 How big an error size is acceptable? The answer is ◦ It depends. An example ◦ Effectiveness of a weight loss program ◦ Expect a loss of 0.5 kg after one week. ◦ Measurement scale is accurate to 1kg. ◦ Not good enough for measuring individual change ◦ OK for a group change, if group size is ‘large’. 4

5 5 On the NAPLAN scale… 5

6 6 6

7 7 Measuring Growth Growth measure? Margin of error of growth measure ± 76 points Expected growth is 50 points 7

8 8 Class mean scores Average score for a class ◦ Effect of measurement error reduces New source of error ◦ Sampling error Cohort of students changes from year to year Variation in class mean score because of the sample of students in a class Class mean ± 20 points ◦ (1 year’s growth) 8

9 9 Teacher effect A high performing teacher can raise student standards by one more year of growth as compared to a low performing teacher. excellent teacher average teacher poor teacher 50 points Margin of error of teacher effect based on two testing occasions: ± 20 points 9

10 10 MySchool Website It is a league table ◦ It compares and ranks schools It is the worst kind of league table ◦ Because it is claimed that the red bars reflect “underperforming schools” ◦ Simple league tables do not have this claim.

11 11 Summary - 1 NAPLAN results are NOT suitable for measuring Student achievement level  ◦ beyond a rough “lower”, “average”, “higher” groups Student progress  Teacher effect  School performance  11

12 12 Summary - 2 NAPLAN results are for the systems, e.g. ◦ Compare girls and boys ◦ Compare rural and urban ◦ Trends, if equating design is improved NAPLAN results should NEVER be published. Parents/caregivers should not be encouraged to use the results to judge schools. 12

13 13 Finally… Conflicting advice from different experts? An easy way to check out: Ask proponents of MySchool website to publicly name one underperforming school. 13

14 14 References Wu, M.L. (2010). Measurement, sampling and equating errors in large-scale assessments. Educational Measurement: Issues and Practice, (In press: Volume 29 Number 4). Nye, B., Konstantopoulos, S., & Hedges, L. (2004). How Large Are Teacher Effects? Educational Evaluation and Policy Analysis, Vol. 26, No. 3 (Autumn, 2004), pp

15 15 Leigh, A. (2009). Estimating teacher effectiveness from two-year changes in students’ test scores. Economics of Education Review. Byrne, Coventry, Olson, Wadsworth, Samuelsson, Petrill, Willcutt and Corley. (2009). Teacher Effects in Early Literacy Development: Evidence From a Study of Twins. Journal of Educational Psychology, 2009.


Download ppt "1 The Appropriate Use of NAPLAN Data National Symposium, 23 July, 2010 Margaret Wu University of Melbourne 1."

Similar presentations


Ads by Google