1. Linking administrative data to TALIS and PISA
Dr. John Jerrim
UCL

2. Structure Data linkage in TALIS (previous work)
What data is available
Why bother to link?
How was the linking done?
What additional results were produced?
Data linkage in PISA (future work)
What data is available?
What projects might linked PISA data make possible?

3. What do we mean by ‘admin’ data?
Typically collected by national authorities.....
Known for every unit (e.g. Pupil / teacher / school) in the population....
Includes very basic information (e.g. School type / location)....
...but could be a lot more detailed (e.g. Test scores)
Every country will have some kind of administrative data!!
(Though some likely better than others)

4. Why are admin data so good?

5. Strengths of admin data....
Very low amounts of missing data
Available for both survey and non-survey units
Typically very well measured
Provides extra information not collected within the survey
FREE!!!!
More salient national definitions than ‘international’ variables (see next slide)

6. Private schools in England.....
Table 2.17
OECD report
49% in England!?
We don’t fit international definition very well!
Data on school type (national definition) linked in from admin data.....
...about 7% of population (much more salient to readers of our TALIS report)

7. What administrative data is available in England?

8. School level School type (Academy, maintained independent)
Average achievement of intake (age 11)
Historical school average performance (age 16 exams)
Pupils eligible for free school meals (i.e. level of deprivation)
Ofsted rating (‘quality’ based upon school inspections)

9. Teacher level Teacher workforce census
Teacher pay (and pay progression)
Qualifications
Sickness / absences from work
Demographic information (e.g. Ethnicity)
Type of contract
Roles and responsibilities

10. Pupil level National pupil database
Test scores at age 5, 7, 11, 14 and
Very detailed ethnicity
Detailed special educational needs (SEN)
Deprivation (FSM; local area based measures)
Absences from schools
School exclusions

11. What can be done with administrative-linked TALIS data?

12. What can be done with linked TALIS data?
Address new and interesting questions of national interest
- E.g. Teacher working hours by OFSTED rating
- E.g. Link between teacher pay and job satisfaction?
2. Exam patterns (and potentially correct for) non-response
- E.g. Compare age distribution of teachers in TALIS to population
Look at measurement properties of certain variables
- E.g. How well do teacher reported qualifications match up with administrative records

13. A longitudinal TALIS study?
Possible to link TALIS to teacher workforce census
Hence can track the TALIS teachers as they move forward in their career….
E.g. What are the predictors of teachers leaving the profession?
What are the drivers of teachers moving to another school?
What predicts who becomes a headmaster?

14. How do you link?

15. How do you link? Theory = easy!
Each unit (e.g. teacher) has a unique record number.....
...if this number observed in both TALIS and admin data
...then straight forward to match (in theory!)
Alternative
Unique record number might not be in both files......
...(particularly in linking done ex-post = won’t be in TALIS)
Match on other characteristics (e.g. Forename, surname, DoB)
Will result in some error!

16. Beware of the pitfalls!! (England example)
Every school in England has its own URN
We observe this in both TALIS and admin data
Should therefore be easy to match....
But it was more complicated than that!!
2012/13 some schools changed their ‘status’ from a ‘maintained’ school to an ‘academy’
Basically still the same school...but given a new URN!
Hence URN in TALIS and Admin data didn’t always match!
Had to go back and manually find out the new URN!!

17. What analysis did we do with the administrative linked dataset?

18. Analysis by Ofsted rating

19. Headteachers’ job satisfaction by Ofsted rating

20. % of teachers reporting lack of support for TPD

21. Teachers’ views on working conditions by Ofsted rating

22. Analysis by age 16 test scores (school average)

23. England did not do the PISA-TALIS link Why
England did not do the PISA-TALIS link Why? We already know a lot about the TALIS schools: E.g. Pupils entry scores (age 11) Pupils exit scores (age 16) Proxy for family background etc Less value added for England than other countries So we just linked to the administrative data instead!

24. Headteacher job satisfaction by historical school performance

25. Teachers’ views on working conditions by school average age 16 scores

26. Appendix on teacher non-response

27. A linear probability model of teacher non-response
Some minor differences observed between sampled and participating teachers.....

28. Please download our national report!!!!

29. Linking PISA to administrative data

30. PISA
International study of 15 year olds skills in reading, maths and science
Conducted by OECD every three years
Lots of media, academic, public policy attention
Detailed demographic information
Also a lot of attitudinal data also
Recently noted by OECD (long-term strategy of PISA)
To continue to seek methodological and analytical means to strengthen the policy relevance and analytical power of PISA, including establishing best practice for linking PISA with national assessments

31. PISA-NPD linked data In England, PISA can be linked to NPD
This means we can observed PISA scores along with children’s test scores at ages 5, 7, 11, 14 and 16
This makes it possible to consider children’s progress towards PISA scores during secondary school (e.g. ‘value-added’ measures)
Can also link to children’s post-16 qualifications and destinations
Can therefore look at the link between PISA scores and the probability of going to university, subject choice etc.
In other words – allows longitudinal follow-up of PISA cohorts!

32. Use 1: Link between PISA scores and England’s national exams
There is a strong link between PISA scores and age 16 exam performance……
…helps demonstrate the validity of PISA as a test score measure

33. Use 2: Investigate and correct non-response bias
England missed response target in PISA 2003
Kicked out of the international report
Linked data allowed us to examine the likely bias / impact upon results…..

34. Use 3: Regional estimates in England (forthcoming)
Several countries now producing regional PISA estimates….
… e.g. results for certain US states
Interest in England too – but there are a lot of local education authorities (152) meaning a high cost
Also a lot of extra burden upon schools
Forthcoming work……
…..Use administrative data to produce proxy PISA estimates by LEA
……Linked administrative data helps us answer this interesting policy question

35. Conclusions

36. Conclusions Many advantages to administrative linked data
- Available for population
- Well measured. Low missing data
- New / extra information
- FREE!
Can use to address many interesting questions
- Link to teacher and pupils
- Relationship to national examinations
- Investigate non-response issues
Other countries can probably do this too!!!