1. Enhancing quality of Finnish higher education – Impact of institutional audits 2005-2012 Senior advisor Kirsi Hiltunen Finnish Higher Education Evaluation Council, FINHEEC 13 June 2013, Glasgow

2. Outline of the presentation The Finnish QA system Impact of the audits ‒ General remarks on the first audit round ‒ State of quality management in the Finnish HEIs ‒ HEIs’ own views on the impact of the audit Conclusions Questions to be answered ‒ How effective has the national audit model been? ‒ Are there differences between the two higher education sectors? ‒ Is an audit, as the main national evaluation tool, the most fit for purpose and effective enough to assure the quality of higher education in Finland?

3. The main source material Four studies funded and/or commissioned by FINHEEC on the impact of audits ‒ a study conducted at FINHEEC: Moitus, S (2010) ‒ a study funded by FINHEEC but conducted by a freelance researcher: Talvinen, K (2012) ‒ two studies funded by FINHEEC but conducted at universities: Ala-Vähälä, T (2011); Haapakorpi, A (2011) Institutions’ follow-up reports on the impact of the audit and their post-audit development work (2009-2013) ‒ Reports submitted to FINHEEC three years after the audit Feedback gathered from the audited institutions

4. FINHEEC (The Finnish Higher Education Evaluation Council) An independent expert body The Council ‒ 12 members (universities, universities of applied sciences, students, working life) ‒ appointed for a 4-year period by the Ministry of Education and Culture from candidates proposed by HEIs and student organisations The Secretariat (11 officers) Full operational independence Financed by the Ministry of Education and Culture Full member of ENQA (European Association of Quality Assurance Agencies) Listed in EQAR (European Quality Assurance Register for Higher Education)

5. FINHEEC audit Finland’s response to the requirements of the Bologna process to develop a comprehensive national higher education QA system Institutional approach – the same model for both higher education sectors (universities and universities of applied sciences) Comprehensive approach – covers research, education and social impact – and overall quality management External assessment of internal QA - reflects institutions’ autonomy and responsibility, and a large measure of trust HEIs are responsible for the quality of their operations Each institution develops its quality system based on its own needs and goals

6. Enhancement-led approach Aim to support HEIs in the enhancement of quality and establishment of quality culture by (1) producing information to assist HEIs to develop their activities, and by (2) exchanging and disseminating good practices among other HEIs As a result of the audit, an institution either passes the audit and receives a quality label or is subjected to a subsequent re-audit Institutions are neither rewarded for a positive result nor punished for a negative one – there are no financial incentives or loss of degree-awarding powers No ranking among institutions is established on the basis of audits

7. The first audit round 2005-2012 Universities of applied sciencesUniversities All Finnish higher education institutions Publication year of the audit report Total number of audits Re-audit decisions Re-audit decisions, total % of re-audit decisions/all audits 20052 00% 200662 233% 200751 120% 20086 2233% 20091011220% 20108 1113% 20119 1111% 20123 00% Total4945918%

8. General remarks on the first audit round Nearly one in five audits (18%) resulted in a re-audit decision ‒ 14% of the UASs vs. 24% of the universities ‒ All but one (university) passed its re-audit The audits have directly affected thousands of persons in higher education (preparations; site visits - well over 100 interviewees in each audit; national experts) Feedback from the audited HEIs ‒ On the whole, HEIs quite satisfied with the audits; satisfaction remained relatively constant throughout the first audit round ‒ UASs slightly more satisfied than universities ‒ The management and central administration had the most positive view on the quality work in both higher education sectors

9. General features of quality systems Most HEIs use the Deming cycle as the conceptual framework of quality management Almost all institutions are run as process-based organisations Some HEIs apply widely recognised quality standards and models (e.g. the European Foundation for Quality Management model, or ISO standard), while others have developed their own quality assessment methods All HEIs that passed the audit have a quality manual or respective document Most of the HEIs have hired specific quality personnel – active national networks As a rule, the management in HEIs is highly committed to quality work Students are widely involved in the institutions’ quality work Specific procedures such as internal audits and joint events to foster quality culture

10. Specific quality system brands For example, a Quality Bakery of the Oulu University of Applied Sciences:

11. Quality management of HEIs’ basic duties Quality management of degree education in the best shape ‒ Various procedures for collecting feedback, links between teaching and research emphasised, external and internal evaluations of degree programmes, procedures to ensure the pedagogical quality of education, tutoring and mentoring systems for new students and staff, alumni systems, personal study plans etc. ‒ International and postgraduate students not as involved as other students ‒ Quality management of doctoral education a common development area Quality management of research generally quite well taken care of ‒ Definition of strategic research profiles, process descriptions, project management guidelines, feedback systems, various internal and external evaluation and rewarding systems ‒ UASs lack mechanisms to monitor the quality of research Quality management of societal impact ‒ A need for clarifying discussion on what is meant by social impact and how it should be measured

12. HEIs’ views on the impact of the audit(1/2) Improvement of management and feedback systems Improvement of management systems – strengthening and gelling of the strategic work Quality management better linked to strategic planning and management as well as operations management Several UASs report on the link between the quality system and the improved results of their activities (regarding, e.g., dropout rate, progression and completion of studies) Improvement of feedback systems (student, working life and alumni) Participation of students and external stakeholders in the development of operations enhanced and supported Post-audit development of quality systems linked with the on-going organisational reforms and renewals of operations management

13. HEIs’ views on the impact of the audit (2/2) New operational and quality cultures More consistent and clarified procedures Operations planned and developed on a more long-term basis and more extensively from the premises of students and external stakeholders The establishment and development of quality culture enhanced by improving and systematising communication within institutions and to external stakeholders Dissemination of good practices within and between HEIs More cooperation within institutions between different units and between HEIs Benchmarking activities have increased New evaluation cultures – external evaluations now seen as more significant tools in the development (international evaluations utilised at different organisational levels)

14. Conclusions(1/2) The Finnish audit system can be considered a success – audits as enhancement-led evaluations have received wide acceptance ‒ HEIs and FINHEEC mutually trust one another, communication open and supportive ‒ All Finnish HEIs have established comprehensive quality systems Audits have contributed to the development of Finnish higher education by ‒ Changing quality and operational cultures in HEIs and by increasing cooperation within the whole field of higher education ‒ Fostering a sense of community and a general commitment to the institution ‒ Inclining HEIs to learn from one another and share information and good practices with one another, also between the two higher education sectors – mutual understanding and collaboration between the sectors have been enhanced

15. Conclusions(2/2) Evident that the principle of enhancement-led evaluation should be fostered at least in some form and scale in the future, too The versatility of FINHEEC’s different evaluation types should be fostered – in addition to the process-oriented audits, there is a need for evaluations that deal with the quality of education itself Due to the on-going changes in higher education, further considerations needed to explicate the current roles and responsibilities of the actors in the national QA system: the Ministry of Education and Culture – HEIs – FINHEEC Finland a small country – a need for more cooperation and coordination between various evaluation organisations and which cross the boundaries between education sectors and between education and research.

16. Thank you for your attention! More information: kirsi.hiltunen@minedu.fi www.finheec.fi