Measuring quality of non-market output in education: Approaches developed in England Deborah Garniss Presentation for the joint OECD/ONS/Government of Norway workshop, October 2006

What are we trying to measure? ► Education seen as an activity  output is volume of this activity provided  seen as amount of teaching delivered ► So output is number pupils actually taught ► Measuring quality of teaching delivered ► Difficult to measure directly

How can we measure it? Quality of teaching delivered Quality of pupil learning Exam attainment BUT: ignores many of the wider outputs and outcomes of education School inspection ratings Assume a relationship between teaching, learning and attainment…..

What do we need to do? ► Match attainment to pupils taught in year  but don’t have exams every year  no measure of yearly attainment per pupil ► So attainment not perfect measure ► But what attainment data do we have?  final attainment at end of schooling  Key Stage tests at ages 7, 11 and 14

A number of alternatives ► Final output measures  % passing threshold attainment level (5 good GCSEs)  Average attainment scores over all exams ► ‘Intermediate’ measures  Average attainment scores at end each Key Stage ► Added value (cohort progress) measures  Average progress made between each Key Stage ► Apportionment measures  Apportioning attainment to each year of schooling

Conceptual issues with attainment ► Changes in attainment need accurately to reflect changes in learning / teaching ► So strong assumptions about link between attainment and quality  x% Δ attainment = x% Δ quality  zero attainment = zero quality

Practical issues with attainment ► Incomplete measures of output quality  measured at 4 points during 11 yrs schooling; no data for other 7 years  don’t all capture full range of attainment (at top and bottom ends of distribution) ► Inconsistencies over time  changing scales for measuring attainment  changing scope of final output measures

What do we conclude to date? ► Final output measures  Transparent and simple, related to what employers look for from education system, can value in £ terms  but ignore rest of education system, and long time- lag before impact of policy shows up (up to 11 years) ► Added-value measures  Greater coverage across the system, and closer to added-value in private sector  but reliant on a number of assumptions, which need further analysis

What difference do they make?

Is attainment the right approach? ► Argue education is productive process  output more than just about activity levels  multiple outputs  tangible and observable (e.g. qualifications)  some less so (e.g. soft skills) ► More going on in school than just teaching ► Output & quality in Nat Accts very narrow ► So any productivity estimates are partial