Case Studies Harry Anthony Patrinos World Bank November 2009

1.Private Education Provision 2.Comprehensive Education 3.Using Assessments

Private Education Provision and Public Finance: The Netherlands

Origins 1917: ‘schools to the parents’ Segregation ended conflict Freedom of education, religion, constitution Today: Less religious, unified But schools retain independence Ease of entry

Secondary School Shares

Primary Schools by Orientation (%)

A Top Performer

But Need to Estimate Causal Impact Randomized trials are gold standard Strategy: find variable that matters for treatment (relevance) but not outcome (exclusion) Control for endogeneity in choice (enter private school) arising from selection on unobservables A good instrument predicts choice, but not achievement

Religion Important in Selection School choice and diversity of providers Parents choosers PISA 2006 asks if choice based on religion  50% choose based on religion (Teelken 1998)  91% of private schools religious  59% of all schools religious  Private schools make up 65% of all schools

Real Impact of Private Schools

School choice with public finance, with strong regulations Learning achievements with equity Large private sector with public funding does not mean weak central role Rigorous analysis shows private schools have causal impact on achievement Summary

The Impact of the 1999 Education Reform in Poland: Extending Comprehensive Schooling

Goals of 1999 Reform in Poland Raise level of education Equalize education opportunities Support quality improvements

Change in Structure of System Old Structure Matura Exam New Structure

Impressive Gains PISA OECD average

Empirical Method Estimate distribution of scores change Assess effects of factors of interest Extension obligatory education by 1 year – Postponing vocational education by 1 year Specifically whether students who were in vocational schools in 2000 would have similar scores in 2003 or 2006 in new lower secondary comprehensive schools Propensity score matching

Results of Matching Factual and counterfactual scores of students in different upper secondary tracks Reading achievement 9 th grade 1 st plausible value individual scores PISA 2000 factual weighted mean score (obs) PISA 2003 factual weighted mean score (obs) PISA 2003 matched counterfactual score ( matched obs) PISA 2006 factual weighted mean score (obs) PISA 2006 matched counterfactual score (matched obs) Kernel matching 1-1 matching Kernel matching 1-1 matching (1)(2)(3)(4)(5)(6)(7) All schools (3654) (4196) (4151) (2528) (5233) (5229) (3056) ISCED 3C schools (vocational) (983) (4010) (926) (5141) (1090) ISCED 3B schools (technical) (1491) (4150) (1527) (5163) (1823) ISCED 3A schools (general) (1180) (4064) (1233) (5221) (1376) ISCED 3A and 3B schools (2671) (4157) (2206) (5233) (2609)

Results of Matching Relative score change (difference-in-differences) for students in vocational schools Relative score change from PISA 2000 to PISA 2003from PISA 2000 to PISA 2006 Kernel matching1-1 matchingKernel matching1-1 matching ISCED 3C versus ISCED 3A+3B ISCED 3C versus ISCED 3A ISCED 3C versus ISCED 3B

Summary Not all transition countries improved Students might be accustomed to test taking, but not reason for improvement Delay of vocational led to change Increase hours of relevant instruction

Assessments can be used to Inform Policy Decisions: The Case of Jordan

Results can be used to Trigger Reform PISA/TIMSS provide opportunity for countries to: – Evaluate education systems – Establish benchmarks for future tracking – Inform policy responses

Case of Jordan Compares education system against others Analyzes progress towards educational quality Participates in World Education Indicators – which benchmarks systems to OECD countries

IAEP II 1990 Jordan first Arab country to participate in International Assessment of Educational Progress (IAEP II) IAEP II study launched simultaneously with Jordan’s review of education system, which led to comprehensive reform

Jordan Participated in TIMSS 1999 The results of the study came as a shock About 75% of students in mathematics and 67% of students in science scored lower than the international average Jordan ranked 3rd from the bottom in both subjects among the 20 participating countries

Education Reform Expert committee established to investigate causes of poor performance Item-by-item examination of the test and school curricula Jordan re-administered the entire TIMSS examination Results identical to those obtained during the first round of testing However, the results served to inform efforts to reform educational quality

Actions Taken 1.Establish benchmarks of 13-year-olds’ achievement 2.Identify strength and weakness in each subject 3.Compare performance of students 4.Inform teacher training 5.Analyze characteristics related to achievement 6.Target negative and positive influences

Jordan Improves in TIMSS

Summary Use of results provides significant returns Cost of assessment is worthwhile, given the significant benefits Assessments are part of efforts to evaluate education systems Used to establish benchmarks Most importantly, inform policy responses

Harry Anthony Patrinos