How to become full ENQA member? Tibor Szanto Vice-President, ENQA Sarajevo, 12 May 2010

ENQA - History , European pilot project for evaluating quality in HE 1998, Recommendation of the Council (98/561/EC) on European co-operation in QA in HERecommendation of the Council (98/561/EC) 2000, European Network for Quality Assurance in HE established to promote European co-operation in the field of QA (EU countries only) 2002, Widening eligibility for membership: EHEA countries November 2004, the General Assembly transformed the Network into the European Association for Quality Assurance in HE , Sarajevo Tibor Szanto 2

Involvement in ENQA Full members: 45 (24 countries + 1 internat.) Candidate memb.: 3 (+1 country + 1 internat.) Associates: 27 (+12 countries + 3Eu + 2non-Eu) Affiliates: 3 (CEEN, ECA, ECBE) , Sarajevo Tibor Szanto 3

Membership categories Full member QA agency / in the field of HE / from EHEA member state / has been operating and conducting actual evaluation activities for at least two years / meets the membership criteria (ESG) Independent external review report! Candidate member as above, except: agency does not meet the criteria for full membership, but is likely to be able to meet the criteria within two years (external report is due) , Sarajevo Tibor Szanto 4

Other types of involvement Associate status: An Associate body shall be a bona fide organisation or agency with a demonstrable interest in the quality assurance of higher education. (Regulations, Art. 9) Affiliate status: An Affiliate body shall be a network of bona fide quality assurance agencies or other bona fide umbrella organisation concerned with the quality assurance of higher education. (Regulations, Art. 10) , Sarajevo Tibor Szanto 5

1. Activities, regular external QA (Part 2 of ESG!) 2. Official status, legal basis 3. Resources (human, financial, + development) 4. Mission statement 5. Independence from HEIs and government 6. Criteria and processes 7. Accountability (internal QA of agency) 8. Miscellaneous ENQA membership criteria , Sarajevo Tibor Szanto 6

5. Independence operational independence from HEIs and governments procedures and methods, appointment of external experts and determination of outcomes of quality assurance processes are undertaken autonomously and independently = independent process final outcomes of the quality assurance processes remain the responsibility of the agency , Sarajevo Tibor Szanto 7

6. Criteria and processes predefined and publicly available processes used by the agency: ▸ self-assessment by HEI/programme ▸ external assessment by expert panel (student member ‘as appropriate’) ▸ publication of a report (decisions, recommend.) ▸ follow-up procedure ▸ appeals procedure (formal decisions) , Sarajevo Tibor Szanto 8

7. Accountability published policy for agency’s IQA no-conflict-of-interest mechanism internal feedback external feedback  internal reflection, improvememt cyclical external review of the agency’s activities (every five years, including a report on its conformity with the membership criteria of ENQA) , Sarajevo Tibor Szanto 9

ENQA criteria additional to ESG 1 ENQA Criterion 3 – Resources (ESG 3.4) Agencies should have adequate and proportional resources, both human and financial, to enable them to organise and run their external quality assurance process(es) in an effective and efficient manner, with appropriate provision for the development of their processes, procedures and staff , Sarajevo Tibor Szanto 10

ENQA criteria additional to ESG 2 ENQA Criterion 7 – Accountability (ESG 3.8) iii. a mandatory cyclical external review of the agency’s activities at least once every five years which includes a report on its conformity with the membership criteria of ENQA , Sarajevo Tibor Szanto 11

ENQA criteria additional to ESG 3 ENQA criterion 8 - Miscellaneous i. The agency pays careful attention to its declared principles at all times, and ensures both that its requirements and processes are managed professionally and that its judgements and decisions are reached in a consistent manner, even if the judgments are formed by different groups ii. If the agency makes formal quality assurance decisions, or conclusions which have formal consequences, it should have an appeals procedure. The nature and form of the appeals procedure should be determined in the light of the constitution of the agency. iii. The agency is willing to contribute actively to the aims of ENQA , Sarajevo Tibor Szanto 12

ENQA members reviewed Externally reviewed agencies: 34 Review reports and decisions on ENQA website (Reviews section) Guidelines for external reviews of QA agencies in the EHEA (Sept ) Information note (Febr. 2010) ENQA Board decisions on external reviews of quality assurance agencies , Sarajevo Tibor Szanto 13

Types and co-ordination of reviews Type A (ENQA memb. crit. / ESG) Type B (also other aspects) statutory functions ECA Code of Good Practice special context (e.g. bi-national character, NVAO) follow-up on previous review Co-ordination national international (ENQA or other organisation) , Sarajevo Tibor Szanto 14

Presentation of findings – the ideal case Best practice (what the Board makes happy:) ESG Parts II and III covered Each standard discussed separately Structure (for each standard): a) evidence („description”, „facts and data”) b) analysis („consideration”) c) conclusion (judgement on compliance) d) recommendation (if any) Don’t forget about ENQA additional criteria! , Sarajevo Tibor Szanto 15

Judgements by review panel The Agency is with the criterion/ESG standard fully compliant substantially compliant partially compliant non-compliant Overall judgement not required (but can be made if the panel wishes) , Sarajevo Tibor Szanto 16

ENQA decision making Preparation by Secretariat Board review committee (3 pers.) proposal Board decision General Assembly approval , Sarajevo Tibor Szanto 17

Board Review Committee (RC) Task: initial scrutiny of review Scrutiny form (template, see as Annex to Guidelines) Preparation by Secretariat Each RC member makes an analysis ▸ independence, integrity of review ▸ meeting the membership criteria RC joint conclusion and proposal to the Board , Sarajevo Tibor Szanto 18

Review Committee proposal 1.(re-)confirm Full Membership 2.request further information in relation to the evidence/findings 3.request further information in relation to the review 4.full membership is not (re-)confirmed/granted a) agency is given two years to conform to criteria,  Candidate member for this period b) agency’s Candidate membership terminated c) agency is granted Candidate membership 5.review is rejected as unacceptable , Sarajevo Tibor Szanto 19

Board decision The Board a) follows the recommendation of the RC b) disagrees with the recommendation of the RC and draws a (motivated) different conclusion The two questions: Does the information provided in the final report of the agency’s review satisfy the Board that the review was conducted to the required level of independence, integrity and robustness? Does the final report of the agency’s review provide sufficient, verified evidence that the agency meets the ENQA membership criteria and thereby the European Standards and Guidelines for Quality Assurance in Higher Education (ESG)? , Sarajevo Tibor Szanto 20

Notification of the agency Letter by the President: reasons for the Board’s decision optional: requesting a progress report outlining how the recommendations made in the final review report have been addressed (~ in 2 years time) When not agreeing with the decision, the agency may submit an appeal (within two months from the receipt of the letter from the President) , Sarajevo Tibor Szanto 21

ENQA Board approach Not rigid adherence, but ‘substantial compliance’, each and every standard has to be met in substance in order to fulfil the criteria Enhancement led approach, principle-based rather than rules-based , Sarajevo Tibor Szanto 22

Levels of compliance fully compliant: the agency is entirely in accordance with the criterion/ESG standard, which is implemented in an effective manner; substantially compliant: the agency is to a large extent in accordance with the criterion/ESG standard, the principle/spirit of which is followed in practice; partially compliant: some aspects or parts of the criterion/ESG standard are met while others are not. The interpretation of the criterion/ESG standard is correct, but the manner of implementation is not effective enough; non-compliant: the agency fails to comply with the criterion/ESG standard , Sarajevo Tibor Szanto 23

Statistics Reports considered by the Board 34 Reports withdrawn by agency 1 Additional actions before decision: 6 follow-up visit by panel chair 1 more info/clarification requested 5 Decisions on granting/reconfirming full memb. 34 Decisions with enhancement elements: 28 dec.+ recomm.+ progr.report requested 25 decisions + recomm. 3 Follow-up by agencies action plans sent to ENQA 2 progress reports sent to ENQA , Sarajevo Tibor Szanto 24

Thank you for your attention!