1. Indicators&Criteria in External Quality Assessment
Indicators&Criteria in External Quality Assessment
Lithuanian experience
2018
Almantas Šerpatauskas, Acting Director
Centre for Quality Assessment in Higher Education (SKVC)

2. Objectives
to contribute to creation of favorable conditions for free movement of persons
assessment of foreign qualifications
provision of info and consultations
oversight of HEI recognition practices
to encourage quality of activities of HEI
review and accreditation of HEI (institutional review)
evaluation and accreditation of HE study programs (ex-ante and ex-post)
assessment of (new) establishing HEI

3. Expertise of SKVC in QA of HE
SKVC as QA agency has knowledge and experience in application and interpretation of the ESG, and can assist in capacity building of relevant authorities in developing countries since we:
operate as external QA agency in Lithuania for already 23 years;
successfully underwent two external international reviews, organised by ENQA (in 2012 and 2016);
are listed as trustworthy QA agency on EQAR from 2012, did reviews abroad;
work as experts of QA in various capacities (serving on Boards of ENQA and CEENQA; as experts in ENQA reviews of agencies to determine their compliance to ESG; as Bologna experts; as experts in Twinning projects, as members of international and local working groups; as invited speakers to various international events; as consultants to various QA agencies which are in the process of development). 9

4. SKVC as academic information and recognition centre:
Our own quality
SKVC as academic information and recognition centre:
voluntary self-evaluation and external review by the international panel following SQUARE methodology (end of 2015 – beginning 2016).
SKVC as external quality assurance agency:
Mandatory self-evaluation and external review against Standards and Guidelines for Quality Assurance in the European Higher Education Area (ESG) by ENQA:
1st in 2012, 2nd – in 2016.

5. International relations
INQAAHE (International Network for Quality Assurance Agencies in Higher Education) – full member from 1997
ENIC/NARIC (European Network of National Information Centres / Network of National Academic Recognition Information Centres) – full member from 1999
CEENQA (The Network of Central and Eastern European Quality Assurance Agencies in Higher Education) – co-founder and full member from 2000
ENQA (associate member from 2000, full member since 14 Sept 2012)
Listed on EQAR as trustworthy QA agency from 2012

6. SKVC’s commitment to ENQA
ENQA GA in Vilnius (October 2013)
Former SKVC Director – ENQA Vicepresident
Two members of staff – on ENQA Board (in the past)
Two members of staff – among ENQA reviewers for QA agency reviews
Participation in working groups:
on impact of quality assurance
staff development
stakeholder involvement in quality assurance
excellence
review of the external review model for quality agencies
internal quality assurance and compliance of quality agencies with ESG requirements
links between quality assurance and qualifications‘ recognition

7. External Evaluation of study prorammes 2011-2018
All study programmes must be evaluated at least once every 6 years;
Evaluation and accreditation of new study programmes and study programmes in operation;
I & II cycle study programmes, non-degree programmes – evaluation by SKVC;
III cycle studies – evaluation by Research Council of Lithuania;
7/15/2019

8. Study programmes in operation
All study programmes are subject to evaluation and accreditation
Accreditation is based on study programme evaluation report.

9. Evaluation of study programmes in operation
Study programmes of the same (or similar) study field are evaluated at once;
SKVC informs HEIs when their programmes going to be evaluated at least a year before submittion of self-evaluation report;
7/15/2019

10. Evaluation process Follow-up Appeal
Consideration of the final report and publication
Writing an evaluation report
Site visit
Preparation for external evaluation
Self-evaluation
Planning of evaluation

11. Expert panels International or local
Composition of panels: academics, employers and students

12. Number of study programmes evaluated by …
Expert panels
Number of study programmes evaluated by …

13. Study programme evaluation areas
Programme aims and learning outcomes
Curriculum design
Teaching staff
Facilities and learning resources
Study process and student assessment
Programme management
7/15/2019

14. Evaluation criteria Eg. Programme aims and learning outcomes:
Each evaluation area must be evaluated according to set of criteria.
Eg. Programme aims and learning outcomes:
the programme aims and learning outcomes are well defined, clear and publicly accessible;
the programme aims and learning outcomes are based on the academic and/or professional requirements, public needs and the needs of the labor market;
the programme aims and learning outcomes are consistent with the type and level of studies and the level of qualifications offered;
the name of the programme, its learning outcomes, content and the qualifications offered are compatible with each other.

15. Evaluation area, criteria & indicators
the study programme is provided by the staff meeting legal requirements;
the qualifications of the teaching staff are adequate to ensure learning outcomes;
the number of the teaching staff is adequate to ensure learning outcomes;
teaching staff turnover is able to ensure an adequate provision of the programme;
the higher education institution creates conditions for the professional development of the teaching staff necessary for the provision of the programme;
the teaching staff of the programme is involved in research (art) directly related to the study programme being reviewed.
Must be analysed in SER:
the list of the teaching staff complete with information on each member’s academic rank and scientific degree (if any); teaching experience; research interests; subjects taught; practical work experience in the area of the subjects taught;
description of each member’s activities;
information on the teaching staff’s involvement in applied research, projects and research (artistic) activities directly related to the study programme being evaluated;
student/teacher ratio in the provision of the study programme;
data on the teaching staff exchange (visiting/outgoing teaching staff ratio).

16. Evaluation area, criteria & indicators
the study programme is provided by the staff meeting legal requirements;
the qualifications of the teaching staff are adequate to ensure learning outcomes;
the number of the teaching staff is adequate to ensure learning outcomes;
teaching staff turnover is able to ensure an adequate provision of the programme;
the higher education institution creates conditions for the professional development of the teaching staff necessary for the provision of the programme;
the teaching staff of the programme is involved in research (art) directly related to the study programme being reviewed.
It is recommended that the analysis of the teaching staff also includes:
data to prove compliance of the staff composition with legal requirements;
data on the teaching staff turnover;
data on the participation of the teaching staff in scientific conferences, wokshops, exchange programmes, long-term visits, etc.
methods of professional (educational, scientific, practical) development of the staff;
professional development areas and statistics of participation;
age profile of the academic staff;
workload of the academic staff (in the provision of the programme under evaluation and other programmes; time allocated for research and/or other (professional) activities, etc.)

17. Study programme accreditation
The study programme evaluation is positive or negative.
The decision on study programme accreditation is based on external evaluation report.
Study programme can be accredited for:
6 years
3 years
The programme can be given non-accreditation!
7/15/2019

18. Evaluation framework to be introduced since 2019
Evaluation and accreditation of study fields instead of individual study programmes. Use of data gathered from national registers, Monitoring agency, Research Council. Importance of scientific level in the field. Simplified vs full procedure

19. Indicators Threshold and monitoring indicators:
Scientific level in the field;
Student competitive score in admission;
Student drop-out in 3 years period;
Percentage of teaching staff working in HEI on full time basis;
Percentage of graduates employed in relevant positions 12 months after graduation;
Percentage of unemployed graduates 12 months after graduation;
Ratio of incoming and outgoing exchange students in 3 years period;
Percentage of international students in the study field in 3 years period;
Ratio of incoming and outgoing exchange teaching staff in 3 years period;
Percentage of study programmes accredited for 6 years;
Percentage of study programmes accredited for 3 years twice in a row;
Student satisfaction about quality of studies;
Number of students in the study field in 3 year period;
Percentage of students finishing their studies on time.

20. Agency’s point of view
Quantitative and qualitative indicators.
Data must be interpreted in wider context!

21. Thank you!