1. From programme accreditation to quality assessment of study programme groups
Heli Mattisen

2. Agenda Higher education and quality assurance systems in Estonia
Principles of the new type of assessment - implementation and lessons learnt

3. ESTONIAN EDUCATION SYSTEM
Master’s programmes
ECTS credits
(1 to 2 years)
Integrated Bachelor’s and
ECTS or
(5 to 6 years)
Bachelor’s programmes
ECTS credits
(3 to 4 years)
Professional higher education programmes
ECTS credits
(3 to 4.5 years)
Doctoral programmes
Secondary education

4. Higher edaucation in Estonia
Higher education institutions (HEI)
No of HEIs
No of students
No of students per HEI
public universities 6
48 406
8 068
private universities 1
976
state institutions of prof HE 10
10 094
1 009
private institutions of prof HE 9
6075
675
state institutions of VET 2
1516
758
total 29
67 067
2 304
Tere 4

5. EKKA’s mission EKKA’s mission is, in cooperation with its partners,
to promote quality in the field of education
and thereby increase the competitiveness
of the Estonian society.

6. EKKA’s core values Equity: Competency: Cooperation: Openness:
applying the principles of equal treatment in all its activities
Competency:
commanding original and up-to-date know-how in its field; including experienced external assessors; engaging in continuous professional development and introducing the results of its activities internationally
Cooperation:
including partners in development activities, flexibly accommodating their needs and capabilities
Openness:
being open to new ideas, flexibly reacting to changes in the environment; maintaining the transparency of evaluation procedures and publicising the results

7. Estonian Higher Education Quality Agency -EKKA
Main activities
Development of principles and procedures of external evaluation
Assessment of the quality of a study program group (SPG) in HE
Institutional accreditation (IA) in HE
Accreditation of study program groups in VET
Training and counseling
Publications
International cooperation

8. Higher Education Quality Assessment System in Estonia
Accreditation of study programmes
STATE RECOGNITION
Quality assessment of a study programme group
at least once every 7 years
Institutional accreditation (voluntary)
Institutional accreditation
at least once every 7 years
TRANSITIONAL EVALUATION
Registration of study programmes by the Ministry of Education and Research
STATE RECOGNITION: The right to conduct studies in a study programme group

9. The need for the shift
The existing system was resource and time consuming for both HEIs and the state
Did not have much impact on the organizational learning and development of the university as an organization
The focus of the accreditations was too broad – they were simultaneously focused on controlling and on giving recommendations for further development
In the case of a negative decision, the result of the accreditation could be closure of the study programme, thus the honesty in analysing development needs was strongly challenged

10. Purpos and assessment areas of IA
The purpose of IA is to support the development of strategic management and culture of quality in HEIs, inform stakeholders of the outcomes of the main activities thereof, and enhance the reliability and competitiveness of Estonian higher education.
Areas of assessment:
organisational management and performance
teaching and learning
research, development and/or other creative activity
service to society.

11. Assessment areas of SPG QA
Quality assessment of SPGs focuses on study programmes, the content and organisation of studies and on instruction-related development activities within a study programme group. These aspects are not evaluated in the course of institutional accreditation.
Assessment areas:
study programme and study programme development
resources
teaching and learning
teaching staff
students

12. 1. Study programme and study programme development
The content and structure of the study programme are consistent with its objectives and learning outcomes.
Different parts of the study programme form a coherent whole.
The launch or development of the study programme is based on legislation, development plans, analyses (including labour market and feasibility analyses), and professional standards; and the best quality is being strived for.
The study programme development takes into account feedback from students, employers, alumni and other stakeholders.

13. 2. Resources
Resources (teaching and learning environments, teaching materials, teaching aids and equipment, premises, financial resources) support the achievement of objectives in the study programme.
There is a sufficient supply of textbooks and other teaching aids and they are available.
Adequacy of resources is ensured for changing circumstances (change in student numbers, etc.).
Resource development is sustainable.

14. 3. Teaching and learning
Modern teaching methods are used in teaching.
The process of teaching and learning is flexible, takes into account the specifics of the form of study and facilitates the achievement of learning outcomes.
The process of teaching and learning supports learning mobility.
Assessment of learning outcomes (including recognition of prior learning and work experiences) is relevant, transparent and objective.

15. 4. Teaching staff
Teaching staff with adequate qualifications exist to achieve the objectives and learning outcomes of the study programme, and to ensure quality and sustainability of the teaching and learning.
Overall student assessment on teaching skills of the teaching staff is positive.
Qualified foreign and visiting teaching staff and practitioners participate in teaching within the study programme.
The teaching staff is engaged in professional and teaching-skills development.

16. 5. Students
Student places are filled with motivated and capable students.
Dropout rates are low, the proportion of students graduating within the standard period of study is large.
Students study as foreign or visiting students at other Estonian and/or foreign higher education institutions as part of their studies.
Employment rate of alumni is high.
Alumni and their employers are satisfied with their professional preparation and social competencies.

17. Involvement of stakeholders
All principal stakeholders of HE provided their inputs during the process of developing the evaluation policies and procedures; educational institutions in particular were included in the development process.
Focus group interviews
All HEI vice-rectors
Steering group and working group

18. Piloting QA of SPG in 2012 and 2013
SPG Informatics and Information Technology
December 2012 – February 2013
Tallinn University of Technology (14 programmes)
University of Tartu (5 programmes)
Reports and decisions available on EKKA website
November 2013
Tallinn University
Estonian Information Technology College

19. Challenges: finding balance
The new system of QA is developed to give feedback on study programme groups, in this way supporting their development. This approach enables assessments of a more general nature than the evaluation of a single study programme.
Major challenge: to find a balance between two conflicting realities: on the one hand, HEIs expect feedback on every single study programme and; on the other hand the experts` limited time resources, which do not enable them to deal with all study programmes in depth.

20. Challenges: tailor-made
The external quality assessment system goes hand in hand with internal work on quality in universities.
The quality agency proposes the areas of assessment and (minimum) standards as well as questions to ask within each standard.
The questions in the standards are not compulsory
The HEIs are asked to specify the areas where they expect to receive more thorough feedback from experts
The study programmes to be assessed within an SPG are chosen in cooperation with the quality agency and the HEI.

21. Challenges: (self)assessment as a part of every-day life
The HEI is expected to provide as little extra data for evaluation as possible, and experts are provided with direct links to the available information sources.
The challenge is to find a balance between universities` workloads in preparing their self-evaluations and additional documentation for assessment, and assessment experts` workloads in analysing and contextualising that information.

22. Challenges: modern technology
A tailor-made assessment - structure of self-analyses varies as does that of the experts assessment reports. The more diverse the structure and content of the reports are, the harder it is to fulfil (even describe) or foresee future requirements while developing an electronic environment. The electronic evaluation platform that was intended for the SPG assessment was not that well perceived by HEI nor by experts.

23. Summary
The format and procedure of the SPG assessment was well perceived by piloting HEIs.
The reports were considered to be helpful in planning the development of SPGs.
Experts` workloads in processing the information as well as training of experts should carefully be planned.
The information (and cooperation) needs of HEI, quality agency and experts should be thoroughly studied to continue with the development of electronic assessment platform.