Presentation on theme: "Linking administrative data to TALIS and PISA Dr. John Jerrim UCL."— Presentation transcript:
Linking administrative data to TALIS and PISA Dr. John Jerrim UCL
Structure Data linkage in TALIS (previous work) What data is available Why bother to link? How was the linking done? What additional results were produced? Data linkage in PISA (future work) What data is available? What projects might linked PISA data make possible?
What do we mean by ‘admin’ data? Typically collected by national authorities..... Known for every unit (e.g. Pupil / teacher / school) in the population.... Includes very basic information (e.g. School type / location) but could be a lot more detailed (e.g. Test scores) Every country will have some kind of administrative data!! (Though some likely better than others)
Why are admin data so good?
Strengths of admin data.... Very low amounts of missing data Available for both survey and non-survey units Typically very well measured Provides extra information not collected within the survey FREE!!!! More salient national definitions than ‘international’ variables (see next slide)
Private schools in England..... Table 2.17 OECD report 49% in England!? We don’t fit international definition very well! Data on school type (national definition) linked in from admin data about 7% of population (much more salient to readers of our TALIS report)
What administrative data is available in England?
School level School type (Academy, maintained independent) Average achievement of intake (age 11) Historical school average performance (age 16 exams) Pupils eligible for free school meals (i.e. level of deprivation) Ofsted rating (‘quality’ based upon school inspections)
Teacher level Teacher workforce census Teacher pay (and pay progression) Qualifications Sickness / absences from work Demographic information (e.g. Ethnicity) Type of contract Roles and responsibilities
Pupil level National pupil database Test scores at age 5, 7, 11, 14 and Very detailed ethnicity Detailed special educational needs (SEN) Deprivation (FSM; local area based measures) Absences from schools School exclusions
What can be done with administrative-linked TALIS data?
What can be done with linked TALIS data? 1.Address new and interesting questions of national interest - E.g. Teacher working hours by OFSTED rating - E.g. Link between teacher pay and job satisfaction? 2. Exam patterns (and potentially correct for) non-response - E.g. Compare age distribution of teachers in TALIS to population 3.Look at measurement properties of certain variables - E.g. How well do teacher reported qualifications match up with administrative records
A longitudinal TALIS study? Possible to link TALIS to teacher workforce census Hence can track the TALIS teachers as they move forward in their career…. E.g. What are the predictors of teachers leaving the profession? What are the drivers of teachers moving to another school? What predicts who becomes a headmaster?
How do you link?
Theory = easy! Each unit (e.g. teacher) has a unique record number if this number observed in both TALIS and admin data...then straight forward to match (in theory!) Alternative Unique record number might not be in both files (particularly in linking done ex-post = won’t be in TALIS) Match on other characteristics (e.g. Forename, surname, DoB) Will result in some error!
Beware of the pitfalls!! (England example) Every school in England has its own URN We observe this in both TALIS and admin data Should therefore be easy to match.... But it was more complicated than that!! 2012/13 some schools changed their ‘status’ from a ‘maintained’ school to an ‘academy’ Basically still the same school...but given a new URN! Hence URN in TALIS and Admin data didn’t always match! Had to go back and manually find out the new URN!!
What analysis did we do with the administrative linked dataset?
Analysis by Ofsted rating
Headteachers’ job satisfaction by Ofsted rating
% of teachers reporting lack of support for TPD
Teachers’ views on working conditions by Ofsted rating
Analysis by age 16 test scores (school average)
England did not do the PISA-TALIS link Why? We already know a lot about the TALIS schools: E.g. Pupils entry scores (age 11) Pupils exit scores (age 16) Proxy for family background etc Less value added for England than other countries..... So we just linked to the administrative data instead!
Headteacher job satisfaction by historical school performance
Teachers’ views on working conditions by school average age 16 scores
Appendix on teacher non- response
A linear probability model of teacher non-response Some minor differences observed between sampled and participating teachers.....
Please download our national report!!!! https://www.gov.uk/government/publications/teachers-in- secondary-schools-evidence-from-talis-2013 https://johnjerrim.files.wordpress.com/2013/07/national_report_dfe.pdf
Linking PISA to administrative data
PISA International study of 15 year olds skills in reading, maths and science Conducted by OECD every three years Lots of media, academic, public policy attention Detailed demographic information Also a lot of attitudinal data also Recently noted by OECD (long-term strategy of PISA) To continue to seek methodological and analytical means to strengthen the policy relevance and analytical power of PISA, including establishing best practice for linking PISA with national assessments
PISA-NPD linked data In England, PISA can be linked to NPD This means we can observed PISA scores along with children’s test scores at ages 5, 7, 11, 14 and 16 This makes it possible to consider children’s progress towards PISA scores during secondary school (e.g. ‘value-added’ measures) Can also link to children’s post-16 qualifications and destinations Can therefore look at the link between PISA scores and the probability of going to university, subject choice etc. In other words – allows longitudinal follow-up of PISA cohorts!
Use 1: Link between PISA scores and England’s national exams There is a strong link between PISA scores and age 16 exam performance…… …helps demonstrate the validity of PISA as a test score measure
Use 2: Investigate and correct non- response bias England missed response target in PISA 2003 Kicked out of the international report Linked data allowed us to examine the likely bias / impact upon results…..
Use 3: Regional estimates in England (forthcoming) Several countries now producing regional PISA estimates…. … e.g. results for certain US states Interest in England too – but there are a lot of local education authorities (152) meaning a high cost Also a lot of extra burden upon schools Forthcoming work…… …..Use administrative data to produce proxy PISA estimates by LEA ……Linked administrative data helps us answer this interesting policy question
Many advantages to administrative linked data - Available for population - Well measured. Low missing data - New / extra information - FREE! Can use to address many interesting questions - Link to teacher and pupils - Relationship to national examinations - Investigate non-response issues Other countries can probably do this too!!!