1. numberssignificancedataWrong? RAISE-ing concerns about school-level data

2. If you torture data sufficiently, it will confess to almost anything. Fred Menger

3. RAISE-ing concerns about school-level data  The Age of Data  A brief history of Value Added analysis and RAISEonline  Some challenges to RAISEonline  Moving forward

4. The Age of Data -Data is collected throughout the school system -Recent focus on ‘performance’ data (<30 years or so)

5. A brief history of Value Added analysis and RAISEonline -Concerns raised when introduced in the 1990s

6. A brief history of Value Added analysis and RAISEonline ‘Final report on Feasibility studies for a national system of Value Added indicators’ some cohorts would be too small to provide reliable indicators for a single year’s data and would need to be excluded prior to any use of the data for purposes of accountability; some schools with atypical intakes would be more fairly represented if additional factors (e.g. sex, English as a second language) were taken into account, possibly providing adjustments to the value-added scores; the newly available national datasets should be fully explored and analysed by several contractors; the impact of additional variables should be explored, particularly with knowledge of local conditions, to promote insights into school effectiveness. (p8)

7. The Value Added National Project Final report, ‘Feasibility studies for a national system of Value Added indicators’ Carol Taylor Fitz-Gibbon, CEM, 1997.

8. A brief history of Value Added analysis and RAISEonline Peter Smith, ‘On the unintended consequences of publishing performance data in the public sector’ (1995), named eight problems associated with non-effective or counter-productive systems: tunnel vision sub-optimisation myopia measure fixation misinterpretation misrepresentation gaming ossification -Government ploughed on regardless -Schools seem to have been unable to challenge malign effects

9. Challenging use and interpretation of school-level data

10. Seven things you might not know about our schools, FFT EduDatalab 2015

11. Challenging use and interpretation of school-level data -Value Added measures and their problems -Use of inappropriate statistics -Most RAISE data is Not Even Wrong -Unit of Education? -Teacher Input = Pupil Output?

12. Significance tests for school performance indicators: Why they’re OK really (but we should probably call them something else)

13. Moving forward

14. The Value Added National Project Final report, ‘Feasibility studies for a national system of Value Added indicators’ Carol Taylor Fitz-Gibbon, CEM, 1997.

15. Moving forward RAISE can’t realistically do what governments want it to “Education is a complex, open system which needs greater nuance and recognition of uncertainty”

16. RAISE-ing concerns about school-level data  The Age of Data  A brief history of Value Added analysis and RAISEonline  Some challenges to RAISEonline  Moving forward

17. numberssignificancedataWrong? @jack_marwood RAISE-ing concerns about school-level data