1. Improving learning outcomes through information provision: Experimental evidence from Indian villages
Farzana Afridi (ISI, Delhi)
Bidisha Barooah (3ie) Rohini Somanathan (Delhi School of Economics)
December 29, IGC-Jadavpur-ISI Conference

2. Motivation
Lack of accountability is often cited as a reason for the poor quality of public service provision.
Insufficient accountability and low stakeholder awareness can co-exist in the provision of education, leading to poor learning outcomes in schools.

3. Motivation High Enrolment
Over 92% of children aged 6-12 are enrolled in school
Almost all villages have a public primary school
First generation learners – 30% illiteracy among adults
Poor learning levels
50% of grade 5 students in public schools cannot read a grade 2 text (ASER)
Steady increase in private schooling
30% of students aged 6-14 enrolled in rural private schools (ASER)
Grew by over 11 ppt in 8 years between
Better learning outcomes but high variation in quality

4. Literature
Evidence on the effectiveness of school accountability programs on students’ learning outcomes is mixed (Figlio and Loeb, 2011)
positive effects of these initiatives on students’ learning outcomes (Ladd 1999; Jacob 2005; Wong et al. 2009)
- accountability programs could lead school managements to behave strategically in order to cross the learning thresholds
(Koretz and Barron 1998; Jacob and Levitt 2003; Figlio and Getzler 2006)

5. Literature
Information on schools’ performance on standardized tests to local stakeholders can improve learning outcomes by improving accountability through choice, participation and voice (World Bank, 2011).
- Experimental evidence on such initiatives from developing countries has, however, been inconclusive (Andrabi et al. 2014; Banerjee et al. 2008)
- The channels through which information provision can improve student outcomes remain unclear.

6. This paper
Random assignment of households and schools to either a control group in which no attempt was made to bridge existing information gaps in the education market or one of four treatments.
In each treatment either households or schools or both were provided a report card on the performance of students in curriculum based tests designed and administered by us.
Treatments provided either absolute or absolute and relative learning performance to stake holders.
Identify the channels through which outcomes are affected.

7. Preview of findings
We find significant improvement in test scores of private schools students in response to provision of information on relative learning levels to both households and schools.
There is no improvement in the learning outcomes of public school students in any treatment.
Evidence of change in household inputs – lower school absence and school choice - as a result of information provision.

8. Our context Ajmer district of northern state of Rajasthan
Poor adult literacy (59%, lower than state average)
High urbanisation
High primary school enrolment
High prevalence of private schools

9. Study Design Randomized experiment
159 schools from 72 villages assigned to one control and 4 treatment groups.
Both public and private primary schools included.
Standardized tests administered to all students in grades 4 and 5.
Household survey of 1499 randomly selected students.

10. Timeline Date Activity Data Jul, Aug, Sept, 2011 Baseline
Village survey
School survey
Household survey
Student test scores
Oct, Nov 2011
Report card intervention
Feb, Mar 2012
Midline
Aug, Sep, Oct 2012
Endline
Household survey Student test scores

11. Report card intervention
Control T1 T2 T3 T4
Report card recipient
Type of report card
Household
None P1
P1 and P2
School S1
S1 and S2
Number of schools 35 29 37 28 30
Public 18 16 26 20
Private 17 13 11 12 10
Number of students
1064
860
1319
918
996
523
499
858
486
599
541
361
461
432
397

12. Report card intervention
P1: Parental Report Card (Absolute Info)
Student's score in three subjects out of 100
Student's rank in class
P2: Parental Report Card (Relative Info)
Student's rank in panchayat
S1: School Report Card (Absolute Info)
Average scores of each grade
Average learning levels of students in school
S2: School Report Card (Relative Info)
Relative rank of school in panchayat

13. Parental (P) report cards

14. School (S) report cards

15. School (S) report cards

16. Data Child characteristics balance at baseline Treatment - Control
Treatment - Control
Control T1 T2 T3 T4
(1)
(2)
(3)
(4)
(5)
Individual characteristics
(N=1064)
(N=859)
(N=1319)
(N=918)
(N=995)
Overall raw test score
64.72
-8.66*
-5.23
-5.58
-2.64
(2.422)
(4.527)
(4.430)
(5.910)
(2.802)
Hindi test score
27.77
-3.73*
-2.38
-3.06
-1.78
(1.070)
(1.871)
(2.089)
(2.645)
(1.266)
Math test score
18.71
-1.96
-0.68
-0.99
-0.25
(0.720)
(1.269)
(1.096)
(1.552)
(0.812)
English test score+
17.84
-2.87*
-2.10
-1.59
0.73
(0.901)
(1.557)
(1.484)
(1.798)
(0.100)
Enrolled in private school
0.51
-0.09
-0.16
-0.04
-0.11
(0.059)
(0.085)
(0.102)
(0.132)
(0.079)
Male child
0.54
0.10***
0.06*
0.05
0.00
(0.024)
(0.036)
(0.034)
(0.041)
(0.035)
Child enrolled in grade 4
0.53
-0.03
-0.01
-0.02
(0.018)
(0.026)
(0.032)
(0.023)
 Household characteristics
(N=327)
(N=273)
(N=346)
(N=263)
(N=291)
Age of child
10.71
-0.42**
-0.24
(0.108)
(0.170)
(0.124)
(0.150)
(0.162)
0.50
0.01
0.02
(0.010)
(0.015)
(0.014)
(0.013)
Household head’s education
1.94
-0.15
-0.07
-0.05
0.08
(0.168)
(0.209)
(0.225)
(0.240)
(0.246)
Household head daily wage worker
0.47
(0.044)
(0.065)
(0.054)
(0.069)
(0.064)
Household’s wealth index
4.89
0.14
-0.06
(0.145)
(0.218)
(0.213)
(0.267)
Household’s education expenditure
( )
( )
( )
( )
( )
Standard errors clustered at panchayat-level in parenthesis. Significance *** 1% **5% *10%.

17. Data School characteristics balance at baseline Treatment - Control
Treatment - Control
Control T1 T2 T3 T4
(1)
(2)
(3)
(4)
(5 )
(N=35)
(N=29)
(N=37)
(N=28)
(N=30)
Private school
0.49
-0.04
-0.19**
-0.06
-0.15
(0.068)
(0.095)
(0.090)
(0.137)
(0.103)
Monthly teacher salary (Rs.)
( )
( )
( )
( )
( )
Annual tuition (Rs.)
755.09
-19.88
( )
( )
( )
( )
( )
Proportion of graduate teachers
0.83
-0.08
-0.12**
-0.02
-0.07
(0.033)
(0.072)
(0.053)
(0.051)
(0.062)
Proportion of local teachers
0.25
-0.05
-0.01
0.01
(0.066)
(0.092)
(0.082)
(0.085)
(0.086)
Total enrolment in school
204.63
-19.25
-2.14
-9.13
1.44
(14.495)
(22.532)
(20.679)
(34.441)
(30.219)
Pupil-teacher ratio
28.24
2.02
6.47***
3.18
3.07
(1.055)
(3.045)
(1.658)
(2.200)
(2.380)
Presence of SMC
0.89
-0.14
-0.10
(0.037)
(0.096)
(0.097)
Proportion of facilities in school
3.69
-0.41**
-0.50***
-0.47**
-0.45***
(0.094)
(0.160)
(0.135)
(0.220)
(0.142)
Standard errors clustered at panchayat-level in parenthesis. Significance *** 1% **5% *10%.

18. SCHOOL ACCOUNTABILITY
Data
Private and public schools comparable in resources but not accountability at baseline
SCHOOL RESOURCES
SCHOOL ACCOUNTABILITY
School Enrolment
Infrastructure index
Prop. Graduate teachers
Pupil – teacher ratio
SMC exists
Prop. Local teachers
Teacher salary (Rs.)
Public
188.04
3.25
0.770
32.63
0.95
0.151
[N=96]
(13.370)
(0.074)
(0.029)
(1.280)
(0.024)
(0.033)
( )
Private
216.41
3.44
0.768
29.14
0.58
0.350
[N=63]
(13.008)
(0.108)
(0.030)
(1.423)
(0.063)
(0.040)
( )
Difference
-0.194
0.001
3.487*
0.366***
-0.199***
***
(19.587)
(0.126)
(0.044)
(1.957)
(0.059)
(0.052)
( )
Public school students’ households more constrained and less informed at baseline
Education of household head
Daily wage worker
Wealth index
Education expenditure (Rs.)
Desired years of schooling
Know of existence or non-existence of SMC
Public
1.64
0.55
4.56
520.76
8.57
0.34
 [N=897]
(0.047)
(0.017)
(0.046)
(16.514)
(0.091)
(0.016)
Private
2.30
0.38
5.26
9.25
0.44
 [N=602]
(0.054)
(0.020)
(0.056)
(77.069)
(0.082)
Difference
-0.661***
0.169***
-0.698***
***
-0.677***
-0.096***
(0.072)
(0.026)
(66.267)
(0.130)
(0.025)
Standard errors clustered at panchayat-level in parenthesis. Significance *** 1% **5% *10%.

19. Public school students perform worse at baseline

20. Data
Divergence between parental expectation and student performance at baseline
Scholastic skill
Public Schools
Private Schools
Parental Expectation
Student
performance
Difference
Student performance
Differe-nce
(1)
(2)
(1)-(2)
(3)
(4)
(3)-(4)
Hindi
Alphabet recognition
0.98
0.71
0.267***
0.99
0.91
0.087***
(0.019)
(0.014)
Word construction
0.93
0.81
0.117***
0.96
0.027***
(0.017)
(0.010)
Sentence construction
0.64
0.49
0.149***
0.89
0.87
0.018
(0.022)
(0.018)
Comprehension
0.50
0.68
-0.176***
0.80
-0.091***
(0.026)
(0.023)
Math
Count
0.115***
0.97
0.029***
(0.009)
2-digit operation without carry over
0.85
-0.016
0.004
(0.011)
3-digit operation without carry over
0.58
0.26
0.323***
0.60
0.269***
(0.025)
English
0.67
0.264***
0.082***
(0.020)
0.51
0.74
-0.236***
0.95
-0.153***
(0.024)
(0.021)
Standard errors clustered at panchayat-level in parenthesis. Significance *** 1% **5% *10%.

21. Empirical methodology
Impact on test scores
Yit = α0 + ∑ βk Tk + φYi0 + εit
Yit : normalized score of student i in post-intervention round t, t=1,2
T k : dummy for treatment k=1,2,3,4
Yi0 : normalised baseline score of student i
εi : error term

22. Results
Treatment improved learning outcomes of private school students 
Treatment type
Endline - Baseline
Difference
Treatment
Control
(1)
(2)
(1) – (2)
Public Schools
T 1
0.039
0.074
-0.036
(0.087)
T 2
-0.010
-0.085
(0.073)
T 3
-0.059
-0.133
T 4
0.041
-0.033
(0.085)
Private Schools
-0.207
-0.329
0.122*
(0.065)
-0.190
0.139**
(0.062)
-0.348
-0.019
(0.063)
-0.229
0.101*
(0.056)

23. Results
Overall test scores at end line improved for private school students in T4
Private
Public
(1)
(2)
T 1
0.129
-0.100
(0.133)
(0.162)
T 2
0.111
0.009
(0.147)
(0.167)
T 3
-0.020
-0.038
(0.100)
(0.139)
T 4
0.308***
-0.047
(0.108)
(0.104)
Baseline score
0.584***
0.598***
(0.046)
(0.048)
Constant
-0.809**
-0.157
(0.335)
(0.566)
Joint Significance
6.148
0.197
[0.001]
[0.938]
T1=T2
0.038
0.522
[0.848]
[0.476]
T2=T3
1.585
0.069
[0.218]
[0.795]
T3=T4
23.69
0.004
[0.000]
[0.953]
All controls
Yes N
1338
1658 R2
0.368
0.280
Standard errors clustered at panchayat-level in parenthesis. Significance *** 1% **5% *10%.

24. Results
Strongest improvements in private school students’ language scores at end line
Private
Public
Hindi
English
Math
T 1
-0.001
0.0840
0.303
-0.234
0.017
-0.091
(0.112)
(0.111)
(0.188)
(0.156)
(0.171)
(0.166)
T 2
-0.009
0.169
0.191
-0.031
-0.029
0.062
(0.128)
(0.107)
(0.189)
(0.131)
(0.172)
(0.149)
T 3
-0.047
-0.037
0.008
-0.120
-0.003
(0.087)
(0.083)
(0.141)
(0.106)
(0.132)
(0.190)
T 4
0.315***
0.260**
0.281*
-0.057
-0.090
0.074
(0.089)
(0.096)
(0.150)
(0.080)
(0.135)
(0.133)
Baseline score
0.556***
0.445***
0.415***
0.499***
0.486***
0.309***
(0.037)
(0.045)
(0.032)
(0.047)
(0.051)
Constant
-0.377
-0.896***
-1.033**
0.243
-0.541
-0.093
(0.297)
(0.312)
(0.440)
(0.469)
(0.523)
(0.608)
All controls
Yes N
1338
1658 R2
0.309
0.246
0.237
0.258
0.202
0.113
Standard errors clustered at panchayat-level in parenthesis. Significance *** 1% **5% *10%.

25. Results - Mechanisms
But public school students responded by improving school presence at endline
Private
Public
(1)
(2)
Treatment
0.0854**
(0.0676)
(0.0401)
Baseline score
0.116***
0.0113
(0.0330)
(0.0250)
Constant
0.598***
0.179
(0.200)
(0.138)
All Controls
Yes N
367
784 R2
0.141
0.045
The dependent variable is a dummy which takes value 1 if child was absent at midline but present at endline and 0 if
absent at mid and endline. The comparison group is students who were absent at mid and endline.
Standard errors clustered at panchayat-level in parenthesis. Significance *** 1% **5% *10%.

26. Results - Mechanisms
Both private and public schools students exercised school choice at endline
Private
Public
Student level
School-grade level
(1)
(2)
(3)
(4)
(5)
(6)
Treatment
0.068*
0.024
0.100**
0.040*
0.036
0.069***
(0.033)
(0.032)
(0.045)
(0.021)
(0.025)
(0.024)
Constant
0.116***
0.183
0.072**
0.014
0.009
-0.000
(0.038)
(0.110)
(0.030)
(0.020)
(0.068)
(0.019)
Student level controls
Yes No
School and village controls N
525
111
747
161 R2
0.018 
0.078
0.033
0.011 
0.028
0.040
The sample consists of students whose households were surveyed.
Standard errors clustered at panchayat-level in parenthesis. Significance *** 1% **5% *10%.

27. Results - Mechanisms
But private schools students are less resource constrained and chose higher ranked schools
Private
Public
T 1-3
0.023
-0.017
(0.042)
(0.027) T4
0.070*
0.036
(0.040)
(0.038)
Constant
0.069
0.004
(0.121)
(0.064)
All controls
Yes R2
0.056
0.033 N
525
747
The dependent variable equals 1 if the rank of child’s school at endline is higher than school rank at baseline. School rank is based on student’s overall score at the school and panchayat level at baseline. The sample consists of students whose households were surveyed. Standard errors clustered at panchayat-level in parenthesis. Significance *** 1% **5% *10%.

28. Proportion of baseline
Robustness
High and selective attrition
All
Private
Public N
Proportion of baseline
Round 1
5157
2192
2965
Rounds 1 and 2
4000 78
1825 83
2175 73
Rounds 1 and 3
2996 58
1338 61
1658 56

29. Robust impact using inverse probability weights
Robustness
Robust impact using inverse probability weights
Private
Public T1
0.420***
(0.115)
(0.301) T2
0.329***
(0.107)
(0.077) T3
0.241**
-0.135
(0.111)
(0.109) T4
0.487***
(0.118)
(0.093) N
1338
1658

30. Robust impact using Lee bounds
Robustness
Robust impact using Lee bounds
Private
Public T1 T2 T3 T4
Lower bound
-0.349***
0.163**
-0.529***
-0.118
-0.395***
-0.143
(0.080)
(0.123)
(0.097)
(0.068)
(0.144)
(0.109)
(0.134)
(0.135)
Upper
0.266***
0.0935
0.132
0.486***
-0.232*
0.0635
0.0116
0.149
bound
(0.087)
(0.086)
(0.060)
(0.127)
(0.154)
(0.117) N
902
1002
973
938
1022
1381
1009
1122

31. Summary
Test scores in private schools improve when both parents and schools know relative school quality.
Public schools may not be able to adjust resources or have the incentive to respond.
Public school students’ households respond on attendance and school choice but their response is not sufficient to improve learning outcomes.
Private school students respond by exercising improved school choice.

32. Conclusions
Providing relative information on student performance may lead to greater household responsiveness.
Household pressure may be inadequate to incentivize schools to respond, at least in the short run.