1. Evaluation of NSRF measures on the development of public education Gábor Balás Hétfa Research Institute 30 April 2013

2. Goals and focuses of the evaluation Goal: Evaluation of measures of the 2007-13 period for public education Education programme development, teachers’ training, equipment stock development, real estate development from 5 aspects: relevance (targeting), linkage with the institutional integration, factors affecting demand and fund absorption, results, effects, synergies. Additionally Some other (e.g. EEOP) calls, Kindergartens only in case of real-estate developments

3. Background and methodology of the evaluation Three important background process from 2007: „PISA shock” reduction of funding transformation of the institutional system of the education Recent changes relevant to the recommendations: -regulations of the new public education act -establishment of the Klebelsberg Institution Maintanace Centre Mixed qualitative and quantitative methodology: econometric analysis (EMIR-competencies assessment-TeIR), 16 case studies of schools, 18 expert interviews, document analysis

4. Evaluation of relevance Strategic relevance before 2010: The controversial basis: the medium term development strategy (Ministry of Education 2003); 2007-2008 prolonged negotiation process (Ministry of Education and Culture, HRP-MA) However: the evaluated calls fit to the strategy Strategic relevance after 2010: remarkable paradigm shift (public education act); The most recent calls follow this, yet with the mitigation of the risks; The change in management cause some, yet manageable risks

5. Assessment of targeting – 1. Where project selection exists (e.g. real-estate): The development aspects besides public education objectives make an effect (sustainability territorial cohesion vs. competencies results, equipment supply) VariablesApplied Rejected from applicants With existing contract from applicants the sponsoring organisation0,2256-0,0224-1,8590 economic power per capita-0,05770,08265,4826 change in the number of population aged 3-50,00780,182715,5673 The institution is in a larger villages0,3776-0,25532,6154 The instititution is in a county seat city-0,40450,211721,0628 LAMR developed with a complex programme0,3276-0,26191,4801 micro-region with small villages0,1580-0,26539,0678 N14150790 Probit coefficients: Real-estate development, in case of convergence regions Source: EMIR - competencies assessment – TeIR Coefficients marked grey are significant

6. Assessment of targeting – 2. In case of „Sprinkle" funding (teachers, equipment, content): The applicant activity and the capacity are the decisive factors, The institutions with weaker capacities are left out VariablesApplied Rejected from applicants With existing contract from applicants economic power per capita-0,11880,1542-0,1159 The institution is in a village-0,46040,1968-0,0985 The institution is in a larger village-0,35770,4764-0,8154 The instititution is in a county seat city0,14250,1594-0,0549 Settlements with low tax capacity-0,1722-0,21100,1190 Settlements with high unemployment-0,17310,3514-0,1783 N14150406 Probit coefficients: Teachers’ training measures, in case of convergence regions Source: EMIR - competencies assessment – TeIR Coefficients marked grey are significant

7. Demand analysis Relationship with the institutional integration: There is no linkage between them, but We have not even found such intention, however It has helped the integration (e.g. common trainings etc.). Factors affecting popularity: Applying pressure for funds, but adjustable for own needs – Broad scope of activities allowed – Large project size (interoperability, flexibility) – Fitting to the existing (funding searching) programmes Factors and importance of the demand The demand affects more, than the NDA’s intention The educational performance affects only via territorial aspects Application consultants are important, the larger schools built their own capacities

8. Demand assessment – Illustration 1. Geographical distribution of grants for equipment stock development per school-age inhabitants, and distribution of successful applications

9. Geographical distribution of grants for teachers’ training awarded per school- age inhabitants and geographical distribution of granted applications Demand assessment – Illustration 2.

10. Distribution of real-estate development grants awarded per school-age inhabitants and geographical distribution of granted applications Demand assessment – Illustration 3.

11. Geographical distribution of grants targeting quality improvements of education programmes per school-age inhabitants and geographical distribution of granted applications Demand assessment – Illustration 4.

12. Impacts The time scale of the databases are too short to conduct data analysis, but According to our case studies there are positive effects, e.g. on: – the organisational structure of the institutions, – the popularity of subjects, – the decrease of number of the absents without leave, Which can have an impact on the competencies results in the future Questionable: How quickly they will be transformed into competencies results; Will they be sustained on the long term (e.g. the retirement of the teacher is a negative risk).

13. Evaluation of Synergies 1. Linkage between the calls: The interoperability between the OPs grew; The typical co-applications are not ones which the authorities planned Synergies on the institutional level The institutions are trying to build the developments on each other The granting scheme does not help, yet There are examples of success; The main motivation however is the continuation of existing educational program, among the scarcity of funds

14. Evaluation of Synergies 2. example: correlation of the applicant groups in the convergence regions Equipment stock Educational programme Teachers’ training Real-estate stock Other SROPEEOPOther Equipment stock 1,0000 Educational programme 0,24601,0000 0,0000 Teachers’ training 0,01830,24241,0000 0,25630,0000 Real-estate stock 0,37810,24660,10491,0000 0,0000 Other SROP 0,03540,28490,30260,15251,0000 0,02820,0000 EEOP 0,13180,20430,14000,12450,20621,0000 0,0000 Other 0,19930,26700,19350,38810,24520,14631,0000 0,0000

15. Recommendations 1. The reasons behind the lack of efficiency of the developments: -The high costs of coordination as obstacles for synergies -The lack of information about the development needs The appearance of the central government as direct sponsoring institution in the public education means new possibilities, but it is also a challenge. Recommendation: The design and implementation of the developments should be more strongly integrated into the system of the education policy; The synergies should be strengthened by the support of interoperability and not by the system of parallel calls; The methodological key projects supporting developments should be the ones started first;

16. Detailed demand and capacity assessment should be started along the goals of public education system, but in a comprehensive manner; Instead of granting – For the funding of compulsory tasks, – On the basis of these problem maps, – Need indicator based funding should be introduced, – with the supervision of school-district authority; For the non-compulsory tasks, there should remain grants which are rewarding creativity, innovation, and in the same time maintain the motivation of schools and strengthen their governance capacities. Recommendations 2.

17. Thank you for your attention! Hétfa Research Institute H-1051 Budapest Október 6. utca 19. www.hetfa.hu