Balancing Quality Enhancement and Accountability Reforming the Dutch and Flemish accreditation system Stephan van Galen

INTRODUCTION: HISTORICAL BACKGROUND 1985 Universities in the Netherlands and Flanders agree on external peer review of all degree programmes Programmes reviewed every 6 years Focus on Quality enhancement 2002 introduction of bachelor master structure 2003 introduction of programme accreditation Accreditation: Validation of external peer review by NVAO as an independent government body Focus on Accountability

1. REFORMING ACCREDITATION Point of departure: The balance between enhancement and accountability should be restored; The system should focus more on the content of the programmes and less on procedural aspects; There is a pressing need to reduce red tape; Academic ownership of the system should be increased. Conditions: Accreditation remains on the level of the programme; International acceptance of the system: Compliance with European Standards and Guidelines (ESG); An assessment framework with generic criteria: respecting institutional autonomy and diversity in degree programmes; Limiting the number of standards; Cutting down on bureaucracy; All decisions and reports are public and informative; NVAO appoints peer panels.

2. REFORMING ACCREDITATION: GOALS 1.Stimulating a Quality Culture in Higher Education Institutions (HEI) by introducing an Institutional Audit (IA); 2.Committing the professionals (teachers); 3.Focus on learning outcomes and the programme; 4.Stimulating HEI to aim above threshold quality; 5.Providing Accountability and Quality Enhancement; 6.Introducing a light touch approach based on earned trust; 7.Reducing administrative burdens with 25%.

3. OVERVIEW OF THE NEW SYSTEM (NL) Institutional Audit Positive Negative Comprehensive programme assessment Limited programme assessment Positive (initial ) accreditation decision Negative (initial) accreditation decision Yes No

4A. INSTITUTIONAL AUDIT Focus on the functioning of the (internal) Quality Assurance System of HEI with respect to teaching and learning Institutional audits involve five related questions: 1.What is the vision of the institution regarding the quality of the education it provides? 2.How does the institution intend to realise this vision? 3.How does the institution measure the degree to which this vision is realised? 4.How is the institution working towards effecting improvements? 5.Who is responsible for what?

4B. INSTITUTIONAL AUDIT

5. COMPREHENSIVE PROGRAMME ASSESMENT 16 topics; Judgements on a four point scale: Unsatisfactory, satisfactory, good, excellent; A detailed assessment of content, policy and procedures; Data to facilitate the dialogue between peers and programme; Peer panel selected by HEI, NVAO has to give its consent. Topics : 1.Intended learning outcomes. 2.Programme. 3.Staff. 4.Services. 5.Quality assurance system. 6.Achieved learning outcomes & Learning assessment.

6. LIMITED PROGRAMME ASSESSMENT 3 topics; Judgements on a four point scale: Unsatisfactory, satisfactory, good, excellent; Assessment of the content of the programme with a focus on performance; Data to facilitate the dialogue between peers and programme; Peer panel selected by HEI, NVAO has to give its consent. Topics: 1.Intended learning outcomes. 2.Programme and staff quality. 3.Achieved learning outcomes & Learning assessment.

7A. FRAMEWORK FOR THE LIMITED PROGRAMME ASSESSMENT Intended learning outcomes Standard 1 The intended learning outcomes of the programme have been set down in concrete terms and satisfy international requirements regarding content, level and orientation. Explanation In terms of level and orientation (bachelors or masters programmes; with a professional or academic orientation) the intended learning outcomes are in keeping with the Dutch or Flemish qualifications framework. Furthermore, they are in line with the current requirements set by the occupational field and the discipline from an international perspective in terms of the programme content. Decision Unsatisfactory, satisfactory, good, excellent.

7B. FRAMEWORK FOR THE LIMITED PROGRAMME ASSESSMENT Teaching and learning environment Standard 2 The programme, the staff and the programme-specific facilities make it possible for admitted students to realise the intended learning outcomes. Explanation The content and structure of the programme allow the admitted students to achieve the intended learning outcomes. The quality of the staff and of the programme-specific facilities are essential to this end. For the students, the programme, staff and facilities constitute a cohesive teaching and learning environment. Decision Unsatisfactory, satisfactory, good, excellent.

7C. FRAMEWORK FOR THE LIMITED PROGRAMME ASSESSMENT Learning assessment and learning outcomes realised Standard 3 The programme has an adequate system of exams which demonstrates that the learning outcomes are realised. Explanation Interim and final exams, final projects and the way in which graduates function in practice or in subsequent education demonstrate what level has been realised. Exams and assessments are valid, reliable and clear to the students. Decision Unsatisfactory, satisfactory, good, excellent. General conclusion The quality of the programme is: Unsatisfactory, satisfactory, good, excellent.

8A. ASSESSMENT PROCEDURE: DOCUMENTATION Critical Reflection: a self-reflection which follows the standards of the framework and describes strengths and weaknesses that also indicates any improvement measures that have been taken as a result of the previous assessment. Mandatory Quantitative information on the programme: 1.Intake, transfers and graduates, if possible for the last 6 cohorts; 2.Realised teacher-student ratio; 3.Average number of face-to-face teaching hours per phase of the programme

8B. ASSESSMENT PROCEDURE: DOCUMENTATION Mandatory appendices: 1.Domain-specific reference framework and the learning outcomes of the programme; 2.Overview of the curriculum; content description of the components of the curriculum, stating the learning outcomes, learning objectives, teaching method(s), testing methods, prescribed literature, teacher and credits; 3.Literature list (mandatory / recommended); 4.Tuition and examination regulations; 5.Overview of the relevant staff listing name, position, nature of the appointment, grade and expertise; 6.List of the most recent 25 final theses or the final projects that demonstrate the level achieved by the students).

8C. ASSESSMENT PROCEDURE: DOCUMENTATION Documents for inspection during the site visit: 1.Reports of consultations within the relevant bodies; 2.Testing statements with accompanying assessment criteria and standards (answer models) and a representative selection of actual completed tests (e.g. presentations, work experience placements, assessments or portfolio) and evaluations; 3.A representative selection of final projects from the past two years chosen by the panel together with the accompanying assessment criteria and standards; 4.Manuals and other course material; 5.Summary and analysis of recent evaluation results and relevant management information.

9. ASSESSMENT PROCEDURE: SITE VISIT The mandatory site visit for the limited programme assessment takes one day; Prior to the visit, the assessment panel studies several final projects in order to gain insight into the exit level achieved in the programme. To this end, the panel makes a selection from an overview drawn up by the programme; During the site visit, the assessment panel speaks to at least: 1.Programme management; 2.Members of the examinations board; 3.Teachers; 4.Students; 5.Alumni; 6.Representatives from the occupational field if relevant.

10A. INDEPENDENT PEER PANELS Programme accreditation: NVAO sets the rules for panel composition and independence. Peers have to sign a declaration of independence and a code of conduct set by NVAO: The panel consists of at least four members, one of whom is a student; The panel comprises at least two authoritative subject-matter experts. At least one of these members has teaching experience at the relevant level of education; The panel is aware of the latest international developments in the discipline and, in as far as applicable, has expertise in the occupational field; The panel also has educational expertise and assessment or audit expertise; The panel is independent. The members have had no ties with the institution offering the programme for at least five years;

10B. INDEPENDENT PEER PANELS The panel is supported by an independent, external secretary who has been trained and certified by NVAO; Panel composition and declarations of independence will be published and made public; The parties involved in the assessment, such as panel members, staff or students, can report any matters that could affect the independence of the assessment panel to NVAO. Institutional audit and initial programme assessment: NVAO selects and appoints peer panels in accordance with the rules that apply for panel composition for programme accreditation. In Flanders the rules for panel composition will not be changed.

11. COMPLIANCE WITH ESG No changes in the position of NVAO as an independent organization established by treaty between Flanders and the Netherlands; NVAOs role in guaranteeing the independence and quality of peer panels for the accreditation of programmes is strengthened: HEIs provide suggestions for peers but NVAO has to give its consent. At present this process is in the hands of independent but commercial quality assurance agencies; NVAO selects and appoints peers for the Institutional Audit. Extensive consultations with stakeholders ensures that the new system is more in tune with day-to-day realities in HEIs and should be more effective in fostering a quality culture; The introduction of an institutional audit allows NVAO to take into account the effectiveness of the internal quality assurance processes of HEIs as a whole; System wide analyses is added to NVAOs tasks.