Presentation on theme: "Linking administrative data to TALIS and PISA"— Presentation transcript:
1Linking administrative data to TALIS and PISA Dr. John JerrimUCL
2Structure Data linkage in TALIS (previous work) What data is availableWhy bother to link?How was the linking done?What additional results were produced?Data linkage in PISA (future work)What data is available?What projects might linked PISA data make possible?
3What do we mean by ‘admin’ data? Typically collected by national authorities.....Known for every unit (e.g. Pupil / teacher / school) in the population....Includes very basic information (e.g. School type / location).......but could be a lot more detailed (e.g. Test scores)Every country will have some kind of administrative data!!(Though some likely better than others)
5Strengths of admin data.... Very low amounts of missing dataAvailable for both survey and non-survey unitsTypically very well measuredProvides extra information not collected within the surveyFREE!!!!More salient national definitions than ‘international’ variables (see next slide)
6Private schools in England..... Table 2.17OECD report49% in England!?We don’t fit international definition very well!Data on school type (national definition) linked in from admin data........about 7% of population (much more salient to readers of our TALIS report)
7What administrative data is available in England?
8School level School type (Academy, maintained independent) Average achievement of intake (age 11)Historical school average performance (age 16 exams)Pupils eligible for free school meals (i.e. level of deprivation)Ofsted rating (‘quality’ based upon school inspections)
9Teacher level Teacher workforce census Teacher pay (and pay progression)QualificationsSickness / absences from workDemographic information (e.g. Ethnicity)Type of contractRoles and responsibilities
10Pupil level National pupil database Test scores at age 5, 7, 11, 14 andVery detailed ethnicityDetailed special educational needs (SEN)Deprivation (FSM; local area based measures)Absences from schoolsSchool exclusions
11What can be done with administrative-linked TALIS data?
12What can be done with linked TALIS data? Address new and interesting questions of national interest- E.g. Teacher working hours by OFSTED rating- E.g. Link between teacher pay and job satisfaction?2. Exam patterns (and potentially correct for) non-response- E.g. Compare age distribution of teachers in TALIS to populationLook at measurement properties of certain variables- E.g. How well do teacher reported qualifications match up with administrative records
13A longitudinal TALIS study? Possible to link TALIS to teacher workforce censusHence can track the TALIS teachers as they move forward in their career….E.g. What are the predictors of teachers leaving the profession?What are the drivers of teachers moving to another school?What predicts who becomes a headmaster?
15How do you link? Theory = easy! Each unit (e.g. teacher) has a unique record number........if this number observed in both TALIS and admin data...then straight forward to match (in theory!)AlternativeUnique record number might not be in both files.........(particularly in linking done ex-post = won’t be in TALIS)Match on other characteristics (e.g. Forename, surname, DoB)Will result in some error!
16Beware of the pitfalls!! (England example) Every school in England has its own URNWe observe this in both TALIS and admin dataShould therefore be easy to match....But it was more complicated than that!!2012/13 some schools changed their ‘status’ from a ‘maintained’ school to an ‘academy’Basically still the same school...but given a new URN!Hence URN in TALIS and Admin data didn’t always match!Had to go back and manually find out the new URN!!
17What analysis did we do with the administrative linked dataset?
23England did not do the PISA-TALIS link Why England did not do the PISA-TALIS link Why? We already know a lot about the TALIS schools: E.g. Pupils entry scores (age 11) Pupils exit scores (age 16) Proxy for family background etc Less value added for England than other countries So we just linked to the administrative data instead!
24Headteacher job satisfaction by historical school performance
25Teachers’ views on working conditions by school average age 16 scores
30PISAInternational study of 15 year olds skills in reading, maths and scienceConducted by OECD every three yearsLots of media, academic, public policy attentionDetailed demographic informationAlso a lot of attitudinal data alsoRecently noted by OECD (long-term strategy of PISA)To continue to seek methodological and analytical means to strengthen the policy relevance and analytical power of PISA, including establishing best practice for linking PISA with national assessments
31PISA-NPD linked data In England, PISA can be linked to NPD This means we can observed PISA scores along with children’s test scores at ages 5, 7, 11, 14 and 16This makes it possible to consider children’s progress towards PISA scores during secondary school (e.g. ‘value-added’ measures)Can also link to children’s post-16 qualifications and destinationsCan therefore look at the link between PISA scores and the probability of going to university, subject choice etc.In other words – allows longitudinal follow-up of PISA cohorts!
32Use 1: Link between PISA scores and England’s national exams There is a strong link between PISA scores and age 16 exam performance………helps demonstrate the validity of PISA as a test score measure
33Use 2: Investigate and correct non-response bias England missed response target in PISA 2003Kicked out of the international reportLinked data allowed us to examine the likely bias / impact upon results…..
34Use 3: Regional estimates in England (forthcoming) Several countries now producing regional PISA estimates….… e.g. results for certain US statesInterest in England too – but there are a lot of local education authorities (152) meaning a high costAlso a lot of extra burden upon schoolsForthcoming work………..Use administrative data to produce proxy PISA estimates by LEA……Linked administrative data helps us answer this interesting policy question
36Conclusions Many advantages to administrative linked data - Available for population- Well measured. Low missing data- New / extra information- FREE!Can use to address many interesting questions- Link to teacher and pupils- Relationship to national examinations- Investigate non-response issuesOther countries can probably do this too!!!