1. 1 Assessment and Monitoring Christine.Merrell@cem.dur.ac.uk www.cemcentre.org

2. 2 Centre for Evaluation and Monitoring (CEM) Part of Durham University Monitoring Systems 1.1 million assessments delivered each year Pupils aged 3 – 18 years CEM systems used in 44 countries

3. 3 Why assess? Newton (2007) 18 purposes for which educational judgements may be used –Not exhaustive –Expanding all the time –Interesting perspective on the historical debate about formative and summative assessment

4. 4 Why assess? From birth ~ weight, hearing etc. Qualifications ~ driving test, GCSEs, degree Profile of strengths and weaknesses for planning appropriate learning experiences Indicator of special educational needs Monitor progress and attitudes of pupils and cohorts over time

5. 5 Comparisons –C–Children within a class –G–Groups such as boys/girls –C–Classes within a year-group –C–Current cohorts with previous ones –O–Other schools within a consortium and nationally Progress over time Research –W–Within school –N–Nationally and internationally

6. 6 Layers of information (different levels of detail): –Diagnostic at pupil-level –Group and class trends –School-level information (including trends over time) –Consortium/Authority-level

7. 7 Judgements Observations – E.g. Behaviour Objective tests –E.g. Attainment Practical Pencil and paper Computer-delivered Ways to Assess and Cautions Do you know the psychometric properties of the assessments that you use? Are these methods always reliable and valid? Early Years Foundation Stage Profile? Age 2 – 3 progress check Year 1 phonics check

8. 8 Standardised Assessment –Administration procedure –Comparison against representative norms

9. 9 Standards over time Consistent content Consistent sample of schools Merrell, C. and Tymms, P. (2011) Changes in Children’s Cognitive Development at the Start of School in England 2001 – 2008, Oxford Review of Education, Vol. 37 (3), June 2011, p333 – 345. Example of a high-level use of standardised assessment

10. 10 Initiatives in the Early Years Foundation Stage –Age 3 – 5 years –Increased nursery provision –Curriculum –Assessment Sure Start Education Action Zones Etc. Have these initiatives changed children’s cognitive development at the start of school?

11. 11 PIPS On-entry Baseline Assessment Computer delivered assessment of: –Vocabulary –Phonological awareness & early reading –Early mathematics Additional data: –Date of birth –Sex –First Language

12. 12 Test/Re-test Reliability = 0.98 Internal Reliability (Cronbach’s alpha) = 0.94 Correlation with attainment at age 11 = 0.68

13. 13 Sample 472 State Schools in England

14. 14 Background Variables

15. 15 Mean PIPS T Scores Academic Year Mean total PIPS BLA T score (SD in brackets) 200148.67 (9.66) 200248.50 (9.77) 200348.31 (9.43) 200448.34 (9.49) 200548.23 (9.47) 200648.25 (9.42) 200747.84 (9.41) 200847.59 (9.22)

16. 16 Mean PIPS Raw Scores Early ReadingEarly MathsPicture Vocabulary Total Academic Year Mean (max = 170 ) SDMean (Max = 69) SDMean (Max = 23) SDMean Max = 262 ) SD 200114.3213.9022.719.2113.664.4850.6923.64 200213.9613.4422.669.1713.534.5650.1423.29 200313.3213.0222.809.1613.535.0549.6423.15 200413.2813.0322.849.2513.705.0449.8223.27 200513.2013.0722.388.7913.525.0449.1022.88 200613.5913.9323.169.1513.665.1150.4023.82 200713.2413.6723.629.2213.515.1150.3723.57 200813.1413.3723.749.1813.345.0850.2523.28 Effect Sizes 0.11-0.07

17. 17 Main Effects & Interactions GLMs to analyse changes in BLA scores in relation to: –Year –EAL –Sex –Age at test was entered as covariate

18. 18 Conclusions Statistically significant decrease from 2001 to 2008 for early reading and picture vocabulary. However, the effect sizes of the differences were small. Significant increase in early maths scores although again the effect size was small.

19. 19 Why Should the BLA Scores Remain So Stable Over Time? Success from early interventions aimed at influencing the development of young children is difficult to achieve Reduction in efficacy of small-scale programmes when rolled out Limitations with data analysed but nevertheless this is a large dataset that adds to current studies of trends over time

20. 20 Thank you Christine.Merrell@cem.dur.ac.uk