Debriefing/brainstorming meeting for the training of experts for agency reviews Decision making process on the reviews: experiences from the ENQA Board and challenges Barcelona 17 March 2009

The Board's decision making procedure and the problematic/difficult and “decisive” criteria (standards) Debriefing/brainstorming meeting for the training of experts for agency reviews Barcelona 17-03-09

Renewal/ Application membership Review report ENQA Secretariat (Board) Review Committee Debriefing/brainstorming meeting for the training of experts for agency reviews Barcelona 17-03-09

(Board)Review Committee ENQA Board decides General Assembly approves Debriefing/brainstorming meeting for the training of experts for agency reviews Barcelona 17-03-09

(Board) Review Committee 1. Composition 2. Appointment 3. Task 4 (Board) Review Committee 1. Composition 2. Appointment 3. Task 4. Procedures Debriefing/brainstorming meeting for the training of experts for agency reviews Barcelona 17-03-09

(Board) Review Committee 1. Composition (Board) Review Committee 1. Composition -the Review Committee includes three board members and one alternate -due to the growing number of review reports to examine , ENQA has since 2008 two Review Committees acting 2. Appointment - the committees are appointed by the board for one year, renewable Debriefing/brainstorming meeting for the training of experts for agency reviews Barcelona 17-03-09

(Board) Review Committee 3. Task (Board) Review Committee 3. Task - conduct the initial scrutiny of the review report of a member - make a recommendation to the Board based on its findings Debriefing/brainstorming meeting for the training of experts for agency reviews Barcelona 17-03-09

(Board) Review Committee 4. Procedures (Board) Review Committee 4. Procedures - each member of the Committee will complete a template (ENQA membership criteria and considerations on the independence, integrity and robustness of the review process) - the committee members will combine their analyses to form a conclusion and to decide what recommendation to make to the Board - the conclusion and recommendation will be presented in the form of Template and will fall in to one of five categories Debriefing/brainstorming meeting for the training of experts for agency reviews Barcelona 17-03-09

(Board) Review Committee 4. Procedures Recommendation to the Board a (Board) Review Committee 4. Procedures Recommendation to the Board a. (re)-confirm Full Membership b. request further information in relation to the findings c. request further information in relation to the evidence of the review process d. full membership is not re-confirmed -agency is given two years to conform to criteria -agency is loosing membership -applicant is granting Candidate membership e. review is rejected as unacceptable -agency can reapply within limited period -agency is not granting membership Debriefing/brainstorming meeting for the training of experts for agency reviews Barcelona 17-03-09

Board decision Board members receive copies of:   - the final report of the agency’s review - the findings and recommendation of the Committee - each of the three Committee members’ analyses On basis of the recommendation of the Review Committee the board take a decision - the board is following the recommendation - the board is disagreeing with the recommendation of the Committee and is drawing a (motivated ) conclusion of a different type. Debriefing/brainstorming meeting for the training of experts for agency reviews Barcelona 17-03-09

The review report analysis and board decision are based on two principles   - Does the information provided in the review report satisfy that the review was conducted to the required level of independence, integrity and robustness?   - Does the review report provide sufficient, verified evidence that the agency meets the ENQA membership criteria and thereby the European Standards and Guidelines for Quality Assurance in Higher Education (ESG)? Debriefing/brainstorming meeting for the training of experts for agency reviews Barcelona 17-03-09

Integrity of the review process and quality of the report   - Has the review been conducted in accordance with the Guidelines for reviews of ENQA member agencies or to an equivalent standard of independence, integrity and frankness? - Do all parts of the review report provide sufficient, clear and verified information/evidence? - Is there any discrepancy between the panel’s conclusions and the evidence brought forward in its report? Debriefing/brainstorming meeting for the training of experts for agency reviews Barcelona 17-03-09

Membership criteria ESG 3. 1, 3. 3 / ENQA Criterion 1: Activities Membership criteria   ESG 3.1, 3.3 / ENQA Criterion 1: Activities Attention: take into account the presence and effectiveness of the external quality assurance processes described in Part 2 of the European Standards and Guidelines (ESG) ESG 3.2 / ENQA Criterion 2: Official status Attention: formally recognised by competent public authorities in the EHEA Debriefing/brainstorming meeting for the training of experts for agency reviews Barcelona 17-03-09

Membership criteria ESG 3 Membership criteria   ESG 3.4 / ENQA Criterion 3: Resources Attention: adequate and proportional resources, both human and financial ESG 3.5 / ENQA Criterion 4: Mission statement Attention: have clear and explicit goals and objectives for its work, contained in a publicly available statement Debriefing/brainstorming meeting for the training of experts for agency reviews Barcelona 17-03-09

Membership criteria ESG 3 Membership criteria   ESG 3.6 / ENQA Criterion 5: Independence Attention: • its operational independence from higher education institutions and governments • procedures and methods , appointment of external experts and determination of outcomes of quality assurance processes are undertaken autonomously and independently = independent process Debriefing/brainstorming meeting for the training of experts for agency reviews Barcelona 17-03-09

Membership criteria. ESG 3 Membership criteria ESG 3.7 / ENQA Criterion 6: External quality assurance criteria and processes Attention: The processes, criteria and procedures used by the member should be pre-defined and publicly available.   Processes will normally be expected to include: -a self-assessment or equivalent - an external assessment by experts, including student - publication of a report - follow-up procedure - appeal procedure Debriefing/brainstorming meeting for the training of experts for agency reviews Barcelona 17-03-09

Membership criteria ESG 3 Membership criteria   ESG 3.8 / ENQA Criterion 7: Accountability procedures Attention: needs for - published policy for own IQA (on website) - a no-conflict-of-interest mechanism for external experts - ensure the quality of work of subcontractors - internal and external feedback mechanism - a mandatory cyclical external review every five years including a report on conformity with ENQA membership criteria Debriefing/brainstorming meeting for the training of experts for agency reviews Barcelona 17-03-09

Membership criteria   ENQA Criterion 8i, 8ii, 8iii: Miscellaneous (if applicable) Attention: - requirements and processes are managed professionally - judgements and decisions are reached in a consistent manner - having an appeals procedure - contributing to the aims of ENQA Debriefing/brainstorming meeting for the training of experts for agency reviews Barcelona 17-03-09

Finally   The agency will be notified in writing of the reasons for the Board’s decision, and subsequently will have one month from the receipt of the letter from the President to submit an appeal (if necessary) In the notification of its decision the Board can request that the agency should submit to ENQA a written update outlining how the recommendations made in the final review report have been addressed. Debriefing/brainstorming meeting for the training of experts for agency reviews Barcelona 17-03-09