Presentation is loading. Please wait.

Presentation is loading. Please wait.

Educational Measurement and School Accountability Directorate The small schools dilemma – reliable analysis and target setting from NAPLAN data Educational.

Similar presentations


Presentation on theme: "Educational Measurement and School Accountability Directorate The small schools dilemma – reliable analysis and target setting from NAPLAN data Educational."— Presentation transcript:

1 Educational Measurement and School Accountability Directorate The small schools dilemma – reliable analysis and target setting from NAPLAN data Educational Measurement and School Accountability Directorate (EMSAD)

2 Educational Measurement and School Accountability Directorate Small school data analysis Medium and large school analysis relies on comparison of outcomes with those of previous years and with outcomes of similar schools. Small schools analysis needs to focus on the progress of cohorts rather than comparison of outcomes for different cohorts.

3 Educational Measurement and School Accountability Directorate Data instability Aggregated data for small schools are inherently unstable. Confidence intervals around means and band percentages are much greater than for larger schools. Cohort differences are likely to be much greater.

4 Educational Measurement and School Accountability Directorate Percentage in bands – large school

5 Educational Measurement and School Accountability Directorate Means trend – large school

6 Educational Measurement and School Accountability Directorate Percentage in bands – small school

7 Educational Measurement and School Accountability Directorate Means trend – small school

8 Educational Measurement and School Accountability Directorate Small school confidence intervals Confidence interval of +/- 52

9 Educational Measurement and School Accountability Directorate Confidence interval of +/- 13 Large school confidence intervals

10 Educational Measurement and School Accountability Directorate What is a ‘small school’ for analysis and reporting purposes Reporting limits are usually set out of privacy considerations. ACARA uses a minimum of 5 students. NSW DEC uses a minimum of 10 students. Numbers required for comparison of cohort performance are greater than for reporting purposes. Overall school performance measures used in school performance graphs require results for a minimum of 15 students to construct a ‘reliable’ estimate. (Performance measures based on 10 to 14 students are provided but should be interpreted with caution.) The decisions you make regarding analysis and target setting need to be based on the size of the group and the nature of the data (e.g. means or band percentages.)

11 Educational Measurement and School Accountability Directorate School size data The average enrolment of NSW DEC primary schools is 260 students (approximately 37 per year). Ten per cent of NSW DEC primary schools have less than 23 students (3 per year). Twenty per cent of NSW DEC primary schools have less than 50 students (7 per year) Thirty per cent of NSW DEC primary schools have less than 90 students (13 per year) About a third of NSW DEC primary schools could described as ‘Small schools’ for analysis and reporting purposes.

12 Educational Measurement and School Accountability Directorate Percentage of government primary schools with less than 50 students State Percentage ACT 7 NSW 31 NT Unavailable QLD 37 SA 27 TAS 21 VIC 28 WA 22

13 Educational Measurement and School Accountability Directorate Useful data sources for small schools Expected growth data Item analyses by syllabus outcomes for tracked cohorts and individual students Relative performance graphs Data aggregated over several years

14 Educational Measurement and School Accountability Directorate Y3 to Y5 average growth in reading by Y3 reading score Y3 reading score Mean Y3 reading score Mean Y3 reading score Mean

15 Educational Measurement and School Accountability Directorate Expected growth Expected growth differs according to the starting score. Approximately 57% of Year 5 students achieved expected growth in reading

16 Educational Measurement and School Accountability Directorate Year 5 student growth chart

17 Educational Measurement and School Accountability Directorate Year 7 student growth chart

18 Educational Measurement and School Accountability Directorate Cohort item analysis – Year 3 reading 2008 RS 2.5: Makes some inferences about ideas implicit in texts

19 Educational Measurement and School Accountability Directorate Cohort item analysis – Year 5 reading 2010 RS 2.5: Makes some inferences about ideas implicit in texts

20 Educational Measurement and School Accountability Directorate Individual item analysis – Year 3 reading 2008 RS 2.5: Makes some inferences about ideas implicit in texts

21 Educational Measurement and School Accountability Directorate Relative performance – numeracy v reading

22 Educational Measurement and School Accountability Directorate Relative performance – writing v reading

23 Educational Measurement and School Accountability Directorate Using rolling 3-Year average data year total3 year %age Band Band Band Band Band Band Total Average

24 Educational Measurement and School Accountability Directorate Target setting for small schools Targets should be achievable (realistic) and (preferably) measurable. It is preferable if small school targets don’t rely on comparison with raw performance of previous cohorts. Comparison with 3-year averages may be appropriate. Small school targets are best set in relation to the prior performance of the students to whom the targets relate.

25 Educational Measurement and School Accountability Directorate Target setting options Using the school target setting tool (Replace Year 5 data with Year 3 data and express targets in terms of percentage proficient, for example. ) Expected growth targets Three year rolling average targets Improvement targets in terms of syllabus outcomes (data extracted from individual student learning plans).

26 Educational Measurement and School Accountability Directorate Using the school target setting tool Replace Year 5 data with 2009 Year 3 band percentages

27 Educational Measurement and School Accountability Directorate Possible types of targets Increase percentage of all students with individual learning plans (K-6) achieving 80% of their learning goals for reading (from 2011 achievement average of 66%) Increase the percentage of all students achieving greater than or equal to expected growth to 70% in 2011 (from 59% in 2010) Increase the percentage of the 2009 Year 3 cohort in the top 3 NAPLAN bands from X% to Y% in Year Increase the three year aggregated percentage of Year 5 students achieving in the top three bands to 81% (from the average of 87.5%).

28 Educational Measurement and School Accountability Directorate


Download ppt "Educational Measurement and School Accountability Directorate The small schools dilemma – reliable analysis and target setting from NAPLAN data Educational."

Similar presentations


Ads by Google