Department of Economics, University of Stellenbosch What a difference a good school makes! The impact of school quality on academic performance Marisa von Fintel Servaas van der Berg ReSEP, Department of Economics, University of Stellenbosch 28 September 2017

Introduction and Background Project for Western Cape Education Department (WCED) Considered progress through the system, persistence of academic performance and impact of school quality

Three data sources CEMIS (Western Cape Education Management Information System) data from WCED containing child characteristics Following same children across years 2007-2014 Systemic Evaluation test data from 2007-2014 Standardised tests in language and mathematics Written in grade 3, 6 and 9 Matric data from 2013

Two cohorts First cohort: grade 6 in 2007, grade 9 in 2010, grade 12 in 2013 Second cohort: grade 3 in 2008, grade 6 in 2011, grade 9 in 2014 Two sources of bias: learners dropping out of system (could be dropping out, leaving Western Cape or moving to private school) issues with matching learners between datasets (matching in earlier years based on name, surname, date of birth and gender) Unmatched learners significantly older, more likely to receive Child Support Grant, less likely to attend quintile 4 and 5 schools Bias worse for earlier cohort, so only follow from grade 9 in 2010 and grade 12 in 2013

Questions Using the CEMIS and Systemic Evaluation data, how predictive is performance in earlier grades of the performance in later grades? Using the fact that we have a learner-level panel, what is the impact of school quality on academic performance?

Persistence in performance: grade 3, 6 and 9   Language Mathematics 1 2 3 4 Grade 6 language 0.661*** 0.449*** 0.394*** 0.366*** (0.004) (0.005) Grade 3 language 0.431*** 0.390*** 0.353*** Grade 6 maths 0.879*** 0.653*** 0.569*** 0.534*** (0.006) Grade 3 maths 0.278*** 0.230*** 0.217*** N 26 682 26 689 R-squared 0.489 0.587 0.612 0.637 0.605 0.646 0.686 0.700 Learner-level controls Y School quintile controls School district controls Notes: Other controls included (but not reported) are overage, gender, race, CSG receipt

Summary of results For language, 1 pp increase in Grade 6 language test score associated with 0.37 pp increase in the Grade 9 language test score For language, 1 pp increase in Grade 3 language test score associated with 0.35 pp increase in Grade 9 language test score For mathematics, 1 pp increase in Grade 6 mathematics test score associated with 0.53 pp increase in Grade 9 mathematics test score For mathematics, 1 pp increase in Grade 3 test score only associated with a corresponding 0.22 pp increase in Grade 9 test score

Persistence in performance: grade 9 and matric

Persistence in performance: grade 9 and matric (average unweighted mark) 1 2 3 4 5 Grade 9 lang score -0.298*** -0.012 0.028 0.033 (0.028) (0.025) Grade 9 lang score squared 0.007*** 0.002*** (0.000) Grade 9 maths score 0.577*** 0.352*** 0.366*** (0.012) (0.013) Grade 9 maths score squared -0.002*** -0.000** -0.001*** Constant 45.67*** 37.81*** 35.68*** 35.49*** 36.28*** (0.832) (0.225) (0.729) (0.735) (0.782) N 24 992 R-squared 0.391 0.500 0.544 0.562 0.566 School quintile controls Y School district controls Notes: Other controls included (but not reported): overage, gender, race, CSG receipt

School quality, school choice and academic performance Legislative framework to constrain school choice to closest neighbourhood school However, de facto schools often accept learners who live outside these boundaries (De Kadt, 2011) Given divisions within system, parents often extremely motivated to send children to either: private schools ex-model C schools Msila (2008), Lemon and Lennard (2010) Qualitative evidence of parents exercising exit option Provides qualitative evidence of signals of good schools according to parents (e.g. English skills, presence of a uniform etc.)

School quality, school choice and academic performance School quality and test scores (Van der Berg et al, 2011; Taylor,2011; Spaull, 2012; Shepherd, 2013) Positive relationship between school quality and earnings (Branson and Leibbrandt, 2013) International literature: Charter schools in USA (Angrist et al, 2012 & 2006; Hoxby & Murarka, 2009) Private schools in India and Pakistan (Andrabi et al, 2011; Muralidharan & Kremer, 2009) Elite public schools in Kenya (Lucas & Mbiti, 2014)

Identifying good schools Rank schools according to performance in Systemic Evaluation If scored in top 20% in any of the language or mathematics tests, then top 20 school Separate list for high schools and primary schools 20% of high schools 25% of primary schools

Percentage top 20% schools per quintile   Primary Schools Secondary/Combined Schools Combined Schools Total number of schools* Quintile 1 10.4% 0% 9.2% Quintile 2 4.8% 3.6% Quintile 3 5.4% 1.6% Quintile 4 11.8% 9.1% 8.7% Quintile 5 66.4% 50% 35% 59.9% Total 25.3% 21.9% 23.5% Notes: * Excludes schools with missing quintile data

Top 20% school performance

Number of learners switching to top 20% schools over time Year 2009 2010 2011 2012 2013 2014 Total per grade Gr 4 1 174 (1.67%) 228 (1.96%) 29 (3.03%) 2 (2.5%)   1 433 (1.72%) Gr 5 4 (3.08%) 703 (1.15%) 248 (1.77%) 32 (1.99%) (2.88%) 991 (1.29%) Gr 6 549 (0.98%) 168 (1.08%) 28 (1.46%) 3 (2.11%) 748 (1.01%) Gr 7 481 (0.92%) 207 (1.41%) 33 (1.38%) 721 (1.04%) Gr 8 14 (12.96%) 5 959 (12.38%) 1 446 (10.17%) 7 419 (1.10%) Gr 9 1 (33.33%) 679 (1.54%) 680 Total per year 1 178 (1.57%) 931 (1.27%) 826 (1.16%) 697 (1.00%) 6 199 (9.54%) 2 161 (3.56%) 11 992 (2.43%)

Differences in learner characteristics   Grade 3 in 2008 Grade 6 in 2011 Grade 9 in 2014 Mean (sd) Top 20% Rest Stats diff? Maths Score 68.01 (17.86) 37.24 (20.47) Y 59.28 (19.79) 35.50 (15.25) 59.55 (20.05) 25.35 (13.60) Language Score 89.54 (11.25) 68.22 (18.30) 63.91 (18.96) 44.29 (16.33) 72.52 (12.12) 50.80 (14.76) Overage 0.169 (0.375) 0.294 (0.455) 0.133 (0.340) 0.235 (0.424) 0.097 (0.296) 0.177 (0.382) Female 0.500 (0.500) 0.493 N 0.519 0.539 (0.498) 0.530 (0.499) 0.572 (0.495) isiXhosa home language 0.029 (0.169) 0.259 (0.438) 0.018 (0.133) 0.219 (0.414) 0.009 (0.096) 0.191 (0.393) Receive CSG 0.088 (0.284) 0.404 (0.491) 0.068 (0.251) 0.379 (0.485) (0.168) 0.304 (0.460) Repeating current year 0.031 (0.031) 0.069 (0.254) 0.024 (0.152) 0.058 (0.234) N/A Number of learners 11 754 35 983 14 974 40 367 10 221 31 017

Performance in top 20% versus rest of schools

Impact of school quality (Fixed Effects) Full sample Black learners Language Math Top performing school 0.064*** 0.281*** 0.123*** 0.293*** (0.008) (0.020) (0.019) Grade 6 -1.440*** -0.425*** -1.597*** -0.583*** (0.071) (0.073) (0.084) (0.081) Grade 9 -1.247*** -0.930*** -1.390*** -1.149*** (0.141) (0.144) (0.166) (0.160) Age 0.248*** 0.169*** 0.291*** 0.339*** (0.027) (0.028) (0.036) (0.034) Age squared -0.010*** -0.006*** -0.011*** -0.012*** (0.001) Child support grant 0.016* 0.033*** 0.027 0.028* (0.009) (0.010) (0.016) Constant -0.672*** -0.754*** -1.285*** -2.151*** (0.228) (0.232) (0.288) (0.278) Number of learners 67 289 17 256 R-squared (overall) 0.242 0.140 0.351 0.116

Impact of switching in primary and secondary school (Fixed Effects) Switch between Grade 3 and Grade 6 Switch between Grade 6 and Grade 9 Language Mathematics Top performing school 0.182*** 0.203*** 0.049** 0.260*** (0.063) (0.061) (0.02) (0.019) Grade 3 1.465*** 0.716*** (0.117) (0.112) Grade 9 0.131 -0.741*** (0.164) (0.150) Age 0.178*** 0.471*** 0.425*** 0.262*** (0.058) (0.056) (0.064) (0.059) Age squared -0.008*** -0.016*** -0.015*** -0.007*** (0.001) Child support grant 0.019 0.055 0.036** 0.038** (0.051) (0.049) (0.017) (0.016) Constant -2.037*** -3.789*** -3.833*** -2.471*** (0.556) (0.536) (0.703) (0.642) Number of learners 16 946 13 383 R-squared (overall) 0.413 0.013 0.033 0.090

Timing of switch Switch between Grade 3 and Grade 6 Language test score grade 6 Mathematics test score grade 6 Language test score grade 9 Mathematics test score grade 9 Switched to top performing school: 1 year ago -0.042 -0.008 0.157*** 0.152***   (0.058) (0.055) (0.024) (0.028) 2 years ago 0.096* 0.117** 0.281*** 0.541*** (0.054) (0.051) (0.018) (0.021) 3 years ago 0.118*** 0.135*** (0.045) (0.043) Switched school twice 0.016 -0.058** -0.072** -0.124*** (0.029) (0.027) (0.034) Constant -2.6245*** -0.604* -12.735*** -12.187*** (0.380) (0.361) (0.907) (1.047) Number of learners 48 875 36 459 Adjusted R-squared 0.286 0.341 0.385 0.449 Notes: OLS regression output. Other controls included (but not reported) are age, age squared, CSG, female, race, school quintile current school

Policy Relevance

Conclusion Moving from a weaker school to a top performing school associated with 28% of a standard deviation in mathematics (translates to almost 1 additional year of education) For language, the impact is smaller at 6% of a standard deviation. However, language impact grows to 12% of a standard deviation for the sample of black learners (benefit the most from moving to school where the language used for instruction in all other subjects is taught well)