1. Case Studies Harry Anthony Patrinos World Bank November 2009

2. 1.Private Education Provision 2.Comprehensive Education 3.Using Assessments

3. Private Education Provision and Public Finance: The Netherlands

4. Origins 1917: ‘schools to the parents’ Segregation ended conflict Freedom of education, religion, constitution Today: Less religious, unified But schools retain independence Ease of entry

5. Secondary School Shares

6. Primary Schools by Orientation (%)

7. A Top Performer

9. But Need to Estimate Causal Impact Randomized trials are gold standard Strategy: find variable that matters for treatment (relevance) but not outcome (exclusion) Control for endogeneity in choice (enter private school) arising from selection on unobservables A good instrument predicts choice, but not achievement

10. Religion Important in Selection School choice and diversity of providers Parents choosers PISA 2006 asks if choice based on religion  50% choose based on religion (Teelken 1998)  91% of private schools religious  59% of all schools religious  Private schools make up 65% of all schools

11. Real Impact of Private Schools

13. School choice with public finance, with strong regulations Learning achievements with equity Large private sector with public funding does not mean weak central role Rigorous analysis shows private schools have causal impact on achievement Summary

14. The Impact of the 1999 Education Reform in Poland: Extending Comprehensive Schooling

15. Goals of 1999 Reform in Poland Raise level of education Equalize education opportunities Support quality improvements

16. Change in Structure of System Old Structure Matura Exam New Structure

17. Impressive Gains PISA OECD average

19. Empirical Method Estimate distribution of scores change Assess effects of factors of interest Extension obligatory education by 1 year – Postponing vocational education by 1 year Specifically whether students who were in vocational schools in 2000 would have similar scores in 2003 or 2006 in new lower secondary comprehensive schools Propensity score matching

20. Results of Matching Factual and counterfactual scores of students in different upper secondary tracks Reading achievement 9 th grade 1 st plausible value individual scores PISA 2000 factual weighted mean score (obs) PISA 2003 factual weighted mean score (obs) PISA 2003 matched counterfactual score ( matched obs) PISA 2006 factual weighted mean score (obs) PISA 2006 matched counterfactual score (matched obs) Kernel matching 1-1 matching Kernel matching 1-1 matching (1)(2)(3)(4)(5)(6)(7) All schools 480.0 (3654) 501.7 (4196) 495.6 (4151) 499.3 (2528) 513.8 (5233) 517.3 (5229) 514.6 (3056) ISCED 3C schools (vocational) 357.8 (983) - 444.5 (4010) 453.3 (926) - 472.4 (5141) 476.0 (1090) ISCED 3B schools (technical) 480.4 (1491) - 487.0 (4150) 487.9 (1527) - 503.8 (5163) 504.5 (1823) ISCED 3A schools (general) 543.7 (1180) - 524.6 (4064) 528.2 (1233) - 545.8 (5221) 553.2 (1376) ISCED 3A and 3B schools 514.6 (2671) - 507.9 (4157) 506.1 (2206) - 527.5 (5233) 525.9 (2609)

21. Results of Matching Relative score change (difference-in-differences) for students in vocational schools Relative score change from PISA 2000 to PISA 2003from PISA 2000 to PISA 2006 Kernel matching1-1 matchingKernel matching1-1 matching ISCED 3C versus ISCED 3A+3B93.4103.0101.7107.0 ISCED 3C versus ISCED 3A105.9109.8112.5108.8 ISCED 3C versus ISCED 3B80.286.791.294.2

22. Summary Not all transition countries improved Students might be accustomed to test taking, but not reason for improvement Delay of vocational led to change Increase hours of relevant instruction

23. Assessments can be used to Inform Policy Decisions: The Case of Jordan

24. Results can be used to Trigger Reform PISA/TIMSS provide opportunity for countries to: – Evaluate education systems – Establish benchmarks for future tracking – Inform policy responses

25. Case of Jordan Compares education system against others Analyzes progress towards educational quality Participates in World Education Indicators – which benchmarks systems to OECD countries

26. IAEP II 1990 Jordan first Arab country to participate in International Assessment of Educational Progress (IAEP II) IAEP II study launched simultaneously with Jordan’s review of education system, which led to comprehensive reform

27. Jordan Participated in TIMSS 1999 The results of the study came as a shock About 75% of students in mathematics and 67% of students in science scored lower than the international average Jordan ranked 3rd from the bottom in both subjects among the 20 participating countries

28. Education Reform Expert committee established to investigate causes of poor performance Item-by-item examination of the test and school curricula Jordan re-administered the entire TIMSS examination Results identical to those obtained during the first round of testing However, the results served to inform efforts to reform educational quality

29. Actions Taken 1.Establish benchmarks of 13-year-olds’ achievement 2.Identify strength and weakness in each subject 3.Compare performance of students 4.Inform teacher training 5.Analyze characteristics related to achievement 6.Target negative and positive influences

30. Jordan Improves in TIMSS

33. Summary Use of results provides significant returns Cost of assessment is worthwhile, given the significant benefits Assessments are part of efforts to evaluate education systems Used to establish benchmarks Most importantly, inform policy responses

34. Harry Anthony Patrinos hpatrinos@worldbank.org