Evaluation practice in the Nordic countries: Different national traditions or a common approach? Hanne Foss Hansen Department of Political Science University of Copenhagen

Structure Educational evaluation: Concepts and approaches Case 1: Higher education -Brief reviews country by country -Similarities and differences Case 2: Primary and secondary education (P/S) The effects of all this evaluation? The future?

The concept of evaluation Everyday language: Measurement, assessment, judgement Evaluation language: ”A careful assessment of the merit and worth of processes, structures, output and outcome of interventions and organizations, intended to play a role in future, practical actions situations”

The concept of educational evaluation Testing, student assessment, programme evaluation, personel evaluation, auditing, accreditation, benchmarking, curriculum evaluation and probably even more.

Educational evaluation: Focus on many levels -Individuals (pupils, students, teachers) -Classrooms/courses -Curriculum/programmes -Organizations (schools, universities) -Fields (all schools in a municipality, all programmes in a discipline) -The national level (national quality development and quality assurance systems) -The international level (PISA, EQUIS in the business school area)

Educational evaluation: Many purposes, many uses -Documenting -Controlling -Learning/improving -Reforming -Legitimating -Symbolizing

Focus today primarely on The new forms of evaluation (programme evaluation, auditing, accreditation etc.) not on the classical questions of testing and student assessment Meso-evaluation defined as evaluation coupled not only to professional practice but also to educational policy

Higher education I Adoption of evaluation in the late 1980´s : The Danish Center for Evaluation of Higher Education 1999: The center is reorganized into the Danish Evaluation Institute (EVA)

Higher education II 1990’s: Programme evaluation 2002: Accreditation is introduced 2003: A new university law stresses the responsibility of the universities themselves to conduct evaluations (EVA unclear role) 2004: Auditing is introduced : EVA is made responsible for accreditation of professional education

Higher education I Adoption of evaluation in the late 1960’s and early 1970’s, gaining renewed priority in the mid 1990’s 1995 The National Agency for Higher Education (Högskoleverket) is established

Higher education II : Auditing is the main task : Programme evaluation becomes the main task Accreditation is also part of the picture

Higher education I Adoption of evaluation in the mid 1990´s 1996: The Finnish Higher Education Evaluation Council (Finheec) is etablished

Higher education II As law places responsibility for evaluation with the higher educational institutions an important purpose of the council is to help institutions to develop quality assurance and development systems The council also initiates evaluations of different types Accreditation is important in relation to polytechnics and professional courses 2004: Auditing

Higher education I Adoption of evaluation in the late 1990´s 1998: ”Norgesnettrådet” is established 2003: The Norwegian Agency for Quality Assurance in Education is established

Higher education II Auditing of all higher educational institutions Accreditation of programmes and institutions applying for new programmes and institutional status The Ministry of Education initiates evaluations of higher educational reforms (Høgskolereformen, Kvalitetsreformen)

Higher education Adoption of evaluation in the mid 1990’s 1999: It becomes mandatory for higher educational institutions to develop quality assurance systems The Ministry of Education initiates programme evaluation ad hoc No formalised accreditation system

Similarities across countries in talk, organisation and focus Adoption of meso-evaluation in all countries Anchoring evaluation in semi-autonomous organizations specialized in evaluation (not Iceland) Educational evaluation is decoupled from evaluation of research A turn towards auditing (N, DK, FIN)?

From national imitation to international regulative pressures? National pressures International pressures weak strong weak strong Bologna

Factors explaining convergence Public-sector reforms: New Public Management, focus on results and effectiveness Internationalization: The Bologna proces and the aim of establishing a European Higher Education Area in 2010 Networking across agencies at Nordic as well as European level

Differences in institutional processes Time span in adoption (from Sweden in the late 1960’s, to Denmark in the late 1980’s and Norway in the late 1990’s) Time span in institutionalization (e.g. routinization in Denmark from 1992, in Norway from 2003) Norway as the late adopter has constructed the most radical system

Differences in balances between quality development (QD) and control (C) purposes DK: QD more than C (except professional education) S: From C more than QD to QD more than C FIN: QD more than C (except professional education) N: C but also QD IS: QD more than C

Differences in decision contexts From Denmark where there is no direct coupling to sanctioning and rewarding (except in professional education) to Norway where there is a direct coupling to sanctioning and rewarding with Sweden somewhere in between

Differences in evaluation models - Self-evaluation is an important element in DK, S and FIN but not in N

Differences in composition of evaluation panels DKNS Peers Educational research -+ + Educational leadership -+ - Students-++ Other users+ --

Differences in coordination across individual evaluations Coordination by: DenmarkNorway ProceduresStrong Specified criteria Only used in some evaluations Strong Panel membersWeakStrong Board decisions-Strong

Factors explaining divergence Differences in: political-administrative cultures strategies in public-sector reforms structures and traditions in educational systems timing and content of higher educational reforms

P/S education Late 1990’s the Ministry of Education introduces a program ”Quality development in public Schooling” (attention and tools ) 1999: EVA gets responsibility for evalution in P/S 2002: A law about transparency and openness makes it compulsory to educational institutions to publish evaluations of the quality of teaching 2005: Government proposes to establish a council and an agency for quality development

P/S education 1997: Municipalities have each year to work out written quality reports 2003: The agency for education is split up in the Swedish Agency for Education and an agency for school development : Inspection programme. Inspection reports serve as starting points for improving the quality of municpal schooling.

P/S education 2003: A council for educational evaluation is established. The council has to plan and implement external evaluations as well as develop methods and coordinate local evaluation

P/S education 2004: The Directorate for Primary and Secondary Education is established. The directorate is responsible for an internet- based quality assessment system ensuring transperency in quality information.

P/S education Schools have to do and publish self- evaluations Every 5th year The Ministry of Education assesses the evaluation methods used by schools (site-visits)

P/S education: Similarities Evaluation adopted in all countries International studies have put educational quality and evaluation on the agenda (PISA & TIMMS) All countries build national institutional capacity to deal with quality and evaluation (increasing state control) Transparency in monitoring is important (strenghtening market forces)

P/S education: Differences Balances between quality development and control purposes (S: C control but also QD; DK, N, FIN and IS: more soft approaches) ? – Too early to really conclude on the practice of the new agencies

Comparing the two cases Higher education -Time span in adoption (from late 1960’s to late 1990’s) -Policy-driven development P/S -Later adoption but no time span -Problem-driven development (DK, N)

Evaluation practice in the Nordic countries: Different national traditions or a common approach? Conclusions Similarities in talk Similarities and important differences in actions Evaluation is an elastic concept giving room for national and local constructions

Effects of growth in meso- evaluation I Two very different ways of thinking: 1)Optimism related to the development of learning organizations and a knowledge society 2)Pessimism related to the development of an audit society based om distrust

Effects of growth in meso- evaluation II Are educational institutions transformed into learning organizations or into ”auditable commodities”? Is professional practice part of or de-coupled from evolving evaluation cultures? Limited empirical knowledge in the Nordic countries

The future A turn towards: -Auditing and accreditation? -Evidence-based professional practice? -Evidence-based educational policy?