Larysa Minzyuk - Felice Russo Department of Management and Economics - University of Salento (Lecce) “Improving Education through Accountability and Evaluation Lessons from Around the World” Rome, 3-5 October 2012 The Causal Effect of Class Size on Pupils’ Performance: Evidence from Italian Primary Schools

OUTLINES  Motivations and purposes  Identification strategy  Regression Discontinuity design  Data and procedure  Results and conclusive remarks

Motivations and purposes The relationship between class size and education attainment has been widely explored, but the existing evidence on the class size effect is still contrasting and somewhat inconclusive In this paper, we estimate the class size effect in Italian public primary schools by using Regression Discontinuity (RD; Thistlewaite-Campbell, 1960, JEP) design, which has recently become a standard evaluation framework for solving causal issues with non-experimental data (not only in education) –in Italy a limited research has been done on this issue so far  e.g. data limitations; among the known studies, see Bratti et al. (2007, GE), Brunello and Checchi (2005, EER), Quintano et al. (2009, REST), Russo (2010) Our main results 1.In Italian data, we do not find a significant evidence which supporting class-size reduction policy 2.There is an evidence of sorting of pupils’ characteristics around cut-offs points (25 pupils): pupils with “unfavorite” socio- economic background are in smaller classes

Regression Discontinuity Design 1.An RD-based evaluation is appropriate when increases in grade- enrolment (forcing variable) are linked with jumps in class-size (treatment variable) as predicted by: (a) the threshold rule generating a (b) discontinuous relation between the two variables 2.Individuals (schools and families) cannot precisely manipulate the grade-enrolment in order to receive or avoid treatment (i.e. to affect whether or not the fall on one side of the threshold or the other) 3.Smoothness condition: Other variables are smooth functions of the forcing variable conditional on treatment (i.e. the only reason pupils’ outcomes should jump at the cut-off is due to the discontinuity in the level of treatment) If 1.,2.,3. jointly hold, effects of class size on pupils’ test scores can be interpreted as the local average treatment effect of class size

Data and procedure (1) We conduct our study using information from two sources: 1.The first is the INVALSI test results of V grade pupils in primary schools in 2008/09. These results are available for 150,000 pupils coming from 5,303 public and private primary schools (‘circoli didattici’) from all Italian regions –we restrict our analysis to the public schools whose testing procedure was assisted by INVALSI supervisors 2.The second source of information is school-level administrative data from the Italian Ministry of Education (MIUR; those data do not contain information about schools in regions with a special statute) –we matched the INVALSI data sample with the dataset on school characteristics and class size coming from MIUR –because of missing data on pupils characteristics in INVALSI data, we have 25,407 pupils coming from 1,561 school units

Predicted class size (Angrist and Lavy, 1999) Φ is is the V grade enrolment at school s where the pupil i studied in 2008/09, int ( ) is the function that takes the greatest integer less than the given argument Average and Predicted Class Size, 2008/09 Data and procedure (2)

Compliance of Schools to the Rule, 2008/09

Data and procedure (3) A standard model of fuzzy RD can be described as follows (van der Klaauw, 2002, p. 1262): P is is the test score of pupil’s i in school s, CS is is average class size in V grade at school level, Φ is is the V grade enrolment at school level, indicates the cut-off values of enrolment (multiples of 25), and α () and β () are functions of enrolment When enrolment is a discrete variable, class-size effect can be estimated only parametrically (Lee and Card, 2008)  we decide a linear specification for both control functions α ( ) and β ( ), choosing the piecewise linear splines whose kinks correspond to the values of cut-offs (Urquiola and Verhoogen, 2009, AER; Zada et al., 2009) (2SLS). For instance (1 st stage; 2 knots): CS is = β + β 1 1[Φ is > 25] + β 2 1[Φ is > 50] + β 3 Φ is + β 4 (Φ is - 25)1[Φ is > 25] + β 5 (Φ is - 50)1[Φ is > 50] + μ is

First stage and base IV specifications (full sample), First stage IV stageIV stage with selected obs. MathItalian languageMathItalian language Class size [Φ is > 25] *** 1[Φ is > 50] *** 1[Φ is > 75] *** 1[Φ is > 100] *** Φ is *** yes (Φ i - 25)1[Φ i > 25] *** yes (Φ i - 50)1[Φ i > 50] ** yes (Φ i - 75)1[Φ i > 75] * yes (Φ i - 100)1[Φ i > 100] yes Isei index of father *** Isei index of mother *** Mother’s education ***0.0203*** Pupils with special needs * Constant **0.5316***0.611***0.4836***0.5786*** Observations 25,407 Adjusted R-squared Note: In all regressions, standard errors are clustered by enrolment levels, see Lee and Card (2008). *** p<0.01, ** p<0.05, * p<0.1.

Reduced form of selected observables (full sample), 2008/09 ISEI index father ISEI index mother Mother's education Pupils with special needs 1[Φ is > 25] * *** ** *** 1[Φ is > 50] * 1[Φ is > 75] * 1[Φ is > 100] Φ is *** *** *** *** (Φ i - 25)1[Φ i > 25] *** ** (Φ i - 50)1[Φ i > 50] ** *** ** (Φ i - 75)1[Φ i > 75] * (Φ i - 100)1[Φ i > 100] Constant *** *** *** Observations25,407 Adjusted R-squared Note: In all regressions, standard errors are clustered by enrolment levels, see Lee and Card (2008). *** p<0.01, ** p<0.05, * p<0.1.

First stage and base IV specifications (+/- 3 pupils intervals), First stage IV stageIV stage with selected obs. MathItalian languageMathItalian language Class size [Φ is > 25] yes 1[Φ is > 50] *** 1[Φ is > 75] ** 1[Φ is > 100] ** Φ is *** (Φ i - 25)1[Φ i > 25] ***yes (Φ i - 50)1[Φ i > 50] ***yes (Φ i - 75)1[Φ i > 75] ***yes (Φ i - 100)1[Φ i > 100] ***yes Isei index of father no *** *** Isei index of mother no *** *** Mother’s education no *** *** Pupils with special needs no yes Constant *** ** *yes Observations5,396 Adjusted R-squared Note: In all regressions, standard errors are clustered by enrolment levels, see Lee and Card (2008). *** p<0.01, ** p<0.05, * p<0.1.

Reduced form of selected observables (+/- 3 pupils intervals), 2008/09 ISEI index father ISEI index mother Mother's education Pupils with special needs 1[Φ is > 25] yes *** 1[Φ is > 50] *** *yes0.4053*** 1[Φ is > 75] yes 1[Φ is > 100] yes *** Φ is yes ** (Φ i - 25)1[Φ i > 25] yes *** (Φ i - 50)1[Φ i > 50] **yes *** (Φ i - 75)1[Φ i > 75] 2.031**yes (Φ i - 100)1[Φ i > 100] *** *** *** *** Constant *** *** *** *** Observations5,396 Adjusted R-squared Note: In all regressions, standard errors are clustered by enrolment levels, see Lee and Card (2008). *** p<0.01, ** p<0.05, * p<0.1.

First stage IV stageIV stage with selected obs. MathItalian languageMathItalian language Class size [Φ is > 25] yes 1[Φ is > 50] yes 1[Φ is > 75] yes 1[Φ is > 100] ** Φ is *** * yes (Φ i - 25)1[Φ i > 25] *** yes (Φ i - 50)1[Φ i > 50] *** yes (Φ i - 75)1[Φ i > 75] yes (Φ i - 100)1[Φ i > 100] *** yes * yes Isei index of father no *** *** Isei index of mother no yes Mother’s education no *** *** Pupils with special needs no *** *** Constant *** *** *** *** *** Observations 10,685 Adjusted R-squared First stage and base IV specifications (+/- 5 pupils intervals), Note: In all regressions, standard errors are clustered by enrolment levels, see Lee and Card (2008). *** p<0.01, ** p<0.05, * p<0.1.

ISEI index father ISEI index mother Mother's education Pupils with special needs 1[Φ is > 25] yes *** 1[Φ is > 50] yes ** yes 1[Φ is > 75] yes 1[Φ is > 100] *** *** yes *** Φ is *** yes ** yes (Φ i - 25)1[Φ i > 25] ** yes (Φ i - 50)1[Φ i > 50] yes * yes (Φ i - 75)1[Φ i > 75] yes ** *** (Φ i - 100)1[Φ i > 100] *** *** *** yes Constant *** *** *** *** Observations 10,685 Adjusted R-squared Reduced form of selected observables (+/- 5 pupils intervals), 2008/09 Note: In all regressions, standard errors are clustered by enrolment levels, see Lee and Card (2008). *** p<0.01, ** p<0.05, * p<0.1.

Treatment effect (+/- 3 and 5 pupils intervals), cut-off (25)2 cut-off (50)3 cut-off (75)4 cut-off (100) Three-pupil interval Dep.Var: Math yes * yes ** Average class-size (left side of cut-off) yes yes Average class-size (right side of cut-off) yes yes Obs. 2,7031, Three-pupil interval Dep.Var: Italian yes ** Average class-size (left side of cut-off) yes Average class-size (right side of cut-off) yes Obs. 2,7031, Five-pupil interval Dep.Var: Math *0.0728** yes *** Average class-size (left side of cut-off) yes Average class-size (right side of cut-off) yes Obs. 5,4143,1841, Five-pupil interval Dep.Var: Italian yes 0.063** yes *** Average class-size (left side of cut-off) yes yes Average class-size (right side of cut-off) yes yes Obs. 5,4143,1841, Note: In all regressions, standard errors are clustered by enrolment levels, see Lee and Card (2008). *** p<0.01, ** p<0.05, * p<0.1.

Number of schools in enrolment intervals (+/- 3 pupils), 2008/09 Number of schools in enrolment intervals (+/- 3 pupils, sample compliant), 2008/09

Reduced form estimates of selected observables, sample of compliant schools (+/- 3 pupils intervals), 2008/09 ISEI index father ISEI index mother Mother's education 1[Φ is > 25] *** *** *** 1[Φ is > 50] * ***yes 1[Φ is > 75] ***yes * 1[Φ is > 100] **yes Φ is yes (Φ i - 25)1[Φ i > 25] *** *** *** (Φ i - 50)1[Φ i > 50] * ** yes (Φ i - 75)1[Φ i > 75] ***yes *** (Φ i - 100)1[Φ i > 100] *** *** *** Constant *** *** *** Observations4,111 Adjusted R-squared Note: In all regressions, standard errors are clustered by enrolment levels, see Lee and Card (2008). *** p<0.01, ** p<0.05, * p<0.1.

In this paper we make an attempt to estimate the class-size effect on the pupils' performance using the data from Italian primary schools. We base our estimation strategy on RD design. To apply RD estimation strategy, practitioners have to test if the assumptions of RD analyses are not infringed, otherwise it would invalid to infer a “treatment” effect of class size on pupils' test results 1.In Italian data, we do not find a significant evidence which would strongly support class-size reduction policy 2.When focusing on small intervals (+/-3 and 5 pupils) around selected enrolment cut-offs, selection problem is not evident, as, on the one hand, we do not observe clear stacking behaviour of schools at the thresholds, and, on the other hand, pupils' characteristics result to be distributed smoothly in the large majority of cut-offs for this subsample Conclusive remarks (1)

Conclusive remarks (2) 3.In contrast, as we observe in our data, we find that the stacking behavior is more evident in the reduced sample of compliant schools (+/-3 pupils) and there is a clearer evidence of sorting of pupils' characteristics around cut-offs points: in this sample, right sides of cut-off intervals include more pupils with “unfavorite” socio-economic background  class size are largely used in primary public schools as a kind of compensatory policy (West-Woessman, 2006) Urquiola and Verhoogen (2009) have found an evidence that Chilean schools might exercise selection policy on enrolment. The authors suggest caution when using RD, especially in application to private schools that may have a better control over enrolment compared to public schools Zada et al. (2009) have found an evidence of “selection” policy in public secondary schools in Israel as well. It is worth noting that the authors have not found it in Israelian public primary schools.