1. Conclusions (Yours, not mine!) Fiona Crozier, QAA f.crozier@qaa.ac.uk

2. From the ESG… Section 4: Future perspectives and challenges: An EHEA with strong, autonomous and effective HEIs, a keen sense of the importance of Q & S, good peer reviews, credible QAAs, an effective register and increased co-operation with other stakeholders…is now possible and the proposals contained in this report will go a long way towards making that vision a reality.

3. From Peter Williams presentation National and ENQA reviews must share a level of credibility and trustworthiness that will ensure that all the Boards decisions on membership are based on sound and reliable evidence and can show a high level of consistency of outcome.

4. Possible conclusions in the following areas: Nationally organised/ENQA organised reviews; ENQA/EQAR membership: confusion/consistency? Documentation: the ESG, Guidelines for National Reviews, Briefing pack for…ENQA co-ordinated reviews Process: the self-evaluation process, provision of documentation, panel/site visit, reporting The ENQA/EQAR approval processes

5. Nationally organised/ENQA co- ordinated/ENQA-EQAR relationship Confusion? Consistency? (Of interpretation/process/decision- making/re: the ESG)

6. Documentation The ESG: Interpretation – but also translation (Rafa Llavori/Seamus Purseil/Thierry Malan) (Agency/panel/ENQA/EQAR(?)) Substantial compliance not rigid adherence In particular criterion 3.6 (independence) – common understanding/national context – but from ENQA/EQARs point of view, is there a bottom line? (Consistency) Agency approach (Seamus Purseil)/national glossary Need for reconsideration/review? Do they do the job? The Guidelines…and the Briefing pack

7. Process Self-evaluation process and provision of documentation: (ANECA and AQU) Training for panel members; what about training or briefing for agencies undergoing review? (Michael Kraft and Rafa Llavori) Use it to set out strengths and weaknesses (Seamus Purseil) Are the guidelines for the SED sufficient to produce a document that allows the agency to make its case and for the panel to do its job?

8. Process contd. Panel/site visit: Composition of the panel (Peter Williams): language ability but also the differentiation of roles? Pre-meetings? With/without the agency present? Guidelines on documentation/number of meetings etc?

9. Process contd. Reporting: Structure Purpose Balance between accountability and improvement

10. ENQA/EQAR approval processes Same need for clear process and criteria as for the SED and process of external review What do ENQA/EQAR need in order to fulfil their roles as consistent, credible and transparent decision-makers?

11. My own inconsistencies! All points need to be considered from the point of view of the Agency under review, the panel doing the reviewing and ENQA/EQAR who need to make a decision.

12. So… Were aiming to improve… the guidance, communication/transparency (were not just working with QAAs) and process in each of the three stages (self- evaluation/external evaluation/final approval or review by ENQA/EQAR) A big job? Or not really a problem? Have we been here before and had these conversations before?

13. In the words of Bob Dylan… He said his name was Columbus… And I just said GOOD LUCK.