Irish Universities Quality Board Quality Review at Irish Universities - A tool for empowerment and reform? Dr Padraig Walsh Chief Executive Irish Universities Quality Board “Leadership through Change” DCU, 3 November 2005

Irish Universities Quality Board “continuous internal institutional reform” “the reform agenda” “refocusing of the organisation’s culture” “changing the organisation” “Leadership through Change”

Irish Universities Quality Board Quality Assurance? QA in Irish Universities –Legislation –Practice Role of QA in University Reform –Ireland –Europe Outline of Presentation

Irish Universities Quality Board established 2003 Website:

Irish Universities Quality Board Independent, non-statutory body –17 member Board 7 nominees from IUA (universities) 10 other nominees –HEA, NQAI, IBEC, ICTU, USI –European, American university system Funded by –Universities, HEA Currently being incorporated

Irish Universities Quality Board ??? Quality

Irish Universities Quality Board Quality Review (Evaluation) –Improvement –Enhancement –Formative Accreditation –Judgement-based The Quality Assurance Lexicon

Irish Universities Quality Board Accreditation is likened to … –doing an examination Quality Review is likened to … –going to confession The Quality Assurance Lexicon

Irish Universities Quality Board Quality as ‘fitness for purpose’ Balancing Improvement and Accountability –“Navigating between Scylla and Charybdis” Quality Review

Irish Universities Quality Board Principles of Quality Assurance System should be built on excellence System should be built on trust

Irish Universities Quality Board Universities Act (1997) Quality Assurance in Irish Universities

Irish Universities Quality Board Legislation that requires the University to put in place “quality assurance procedures aimed at improving the quality of education and related services provided by the university” Universities Act 1997

Irish Universities Quality Board “The (QA) procedures shall include the evaluation of.. each department (of the university).. and each faculty (of the university).. and any service provided by the university by employees of the university and by persons.. who are competent to make national and international comparisons on the quality of teaching and research and the provision of services at university level” Universities Act 1997

Irish Universities Quality Board “The procedures shall include assessment by those, including students, availing of the teaching, research and other services provided by the university” Universities Act 1997

Irish Universities Quality Board System of QA based on primary legislation that defines basis for: Internal QA procedures for individual universities External QA of individual universities –Periodic audits (organised by IUQB) of the effectiveness of QA procedures in individual universities External QA of university sector –Review (organised by HEA) of QA procedures in university sector Universities Act 1997

Irish Universities Quality Board Principles of Irish QA Legislation Primary responsibility for Quality Assurance rests with the University Internal QA: with the active contribution of students, universities must evaluate all their activities, including service departments External QA: should focus on checking through institutional audit that internal monitoring has been effectively done

Irish Universities Quality Board What happens in practice? Quality Assurance in Irish Universities

Irish Universities Quality Board Self-assessment –of Academic/Administrative Departments –assessment by students/stakeholders Peer Review (from list of nominees) –National experts –International experts Publication of Findings Implementation of Recommendations –Follow-up Legislation to Methodology

Irish Universities Quality Board Self-Assessment / Self-Evaluation Quality Assurance in Irish Universities

Irish Universities Quality Board What are we trying to do? How are we trying to do it? How do we know it works? How do we change to improve? Self Assessment – 4 questions

Irish Universities Quality Board Academic Department Review Quality Assurance in Irish Universities

Irish Universities Quality Board Easily identified “products” –Teaching Programmes –Research Inputs and Outputs –Service (Committees / Societies ) Easily identified “stakeholders” Familiarity with Peer Review –Research Grants –Manuscipts for Journals –Examination Papers Academic Department Review

Irish Universities Quality Board Administrative Department Review Quality Assurance in Irish Universities

Irish Universities Quality Board Not easy to identify “products/processes” More complex stakeholder system –Student Affairs (mainly students) –Human Resources (mainly staff) –Computer Services (everybody) Less familiar with Peer Review Must be reviewed themselves Are reviewed by everybody else!! Administrative Department Review

Irish Universities Quality Board In Ireland, we use the same methodology as when reviewing academic departments In other countries, Quality reviews of HEIs tend to focus on: –Subject review –Accreditation of programmes In such systems, Administrative Departments get evaluated only insofar as they serve or support the academic function Self-Assessment Methodolgy

Irish Universities Quality Board Report should provide a succinct, comprehensive statement of Unit activities Report should be a self-criticial analysis of the Unit, including how the Unit sees itself contributing to the Mission of the Organisation and the Strategic Plan Principle of Self Assessment

Irish Universities Quality Board Lessons learned since 1995 Self Assessment of Units

Irish Universities Quality Board A statement consisting solely of functions, activities and outputs is not a self-assessment Report A good self-assessment report implies no surprises for Review Team Review visit is a validation of the self-assessment report Self Assessment Report

Irish Universities Quality Board Co-ordinating committee of Unit representative of all staff at ALL grades Involvement of the Head Self Assessment (of unit) recipes for success

Irish Universities Quality Board Have a small but representative Quality committee Regular communication with all staff not on the Quality Committee Self Assessment (of unit) recipes for success

Irish Universities Quality Board Good Peer Review Report! Good Self Assessment

Irish Universities Quality Board Quote from a (good) self- assessment report “... during the course of the two days of meetings, it was notable that no significant areas of criticism were identified which had not already been noted in the (self-assessment) report... ” Good Self Assessment Good Peer Review

Irish Universities Quality Board Benchmark. Aim high! Aim high! Nominate first-class reviewers Good Self Assessment Good Peer Review

Irish Universities Quality Board Follow-up / Implementation Quality Assurance in Irish Universities

Irish Universities Quality Board Demonstrate leadership! Circulate Review Report to all staff Agree a plan of action (Quality Improvement) with timescales and deliverables –1 year –5 year Follow up at Unit level

Irish Universities Quality Board Demonstrate leadership! Publish the Review Report (visibly!) Publish the Quality Improvement Plan Agree to report on progress with the 1 year plan 1 year later –Perhaps invite members of review group back Follow up at Unit level

Irish Universities Quality Board Remember who you are assuring –Students –Staff –External stakeholders QR process is not simply a management tool Each level is being judged on its response to recommendations –Department –Faculty –Management Team Follow-up Lessons for Leadership

Irish Universities Quality Board Review and reform Reviews of Irish Higher Education

Irish Universities Quality Board OECD Review of Higher Education in Ireland (2004) –By Politicians/Civil Servants for Politicians/Civil Servants! –Diagonal slice look through Irish Higher Education Reviews of Irish HE

Irish Universities Quality Board EUA Review of Quality Assurance in Irish Universities ( ) –By universities for universities! –Individual reports on 7 Irish Universities –Cross-cutting report on university sector Reviews of Irish HE

Irish Universities Quality Board European University Association (EUA) were jointly commissioned by IUQB and the Higher Education Authority to: Review the quality assurance procedures in the Irish University sector –1 report Review the effectiveness of quality assurance procedures in the individual Irish Universities –7 reports EUA Review of QA in Irish Universities

Irish Universities Quality Board “... in the context of (each university’s) overall institutional decision making and strategic planning” EUA Review of QA in Irish Universities

Irish Universities Quality Board Strengths EUA Review of QA

Irish Universities Quality Board “(the) systematic organisation and promotion of QA at the initiative of the universities themselves.. is unparalleled in any other country in Europe, or indeed in the United States and Canada” “A common framework for QA has been agreed collectively by all the Irish Universities through IUQB” EUA Review of QA - strengths

Irish Universities Quality Board Irish Universities Quality Board (IUQB) IUQB was established voluntarily by the Irish Universities in 2003 with the aim of increasing the level of inter-university co- operation in developing quality assurance processes

Irish Universities Quality Board “The system would appear to strike the right tone and combination of public interest, accountability and university autonomy” “The agreed IUQB focus on quality improvement is particularly to be commended” EUA Review of QA - strengths

Irish Universities Quality Board “(system) encourages a greater focus on quality and improvement than some systems worldwide, while.. being less intrusive than some other systems in Europe” “(this activity) has taken place without the existence of a government agency” EUA Review of QA - strengths

Irish Universities Quality Board “not linked to the distribution of core governmental funding” “concept of quality is being built into discussions at many levels across each university campus” EUA Review of QA - strengths

Irish Universities Quality Board QA in a university Required Practice v Good Practice Required by lawA university Departmental Review Every 10 years6 year cycle Institutional Audit Every 15 yearsWithin 7 years Publication“Outcome of evaluations” Full PRG Report Full QuIP Synthetic Evaluation 1 year follow-up

Irish Universities Quality Board Weaknesses EUA Review of QA

Irish Universities Quality Board “There is no system in place in the Irish Universities for the performance appraisal of individual staff members, with the possibility of incentives or sanctions attached to this” EUA Review of QA - weaknesses

Irish Universities Quality Board “Given that widespread and sustainable quality improvement in an organisation is unlikely without the opportunity regularly to review and discuss the quality and performance of individuals, the (review) teams were pleased to learn that such a system has recently been agreed with the labour unions and will soon be introduced across the universities. “This potentially powerful tool should be closely articulated with the QA framework” EUA Review of QA - weaknesses

Irish Universities Quality Board “Focusing the review process on academic departments (tends) to reinforce existing academic structures and boundaries” “Quality review cycle is not linked to (for example) the strategic planning (cycle)” EUA Review of QA - weaknesses

Irish Universities Quality Board “QA process has until recently not appeared consistently to have enjoyed the necessary support of senior management and governance at all universities” “Little substantial evidence of systematic follow-up or monitoring at university level to the quality improvement plans of the various units” EUA Review of QA - weaknesses

Irish Universities Quality Board Lessons learned EUA Review of QA

Irish Universities Quality Board A review of the effectiveness of QA procedures must begin with a critical institutional self-evaluation Unit level reviews are a convenient way of “covering all the bases”, not ends in themselves Lessons learned from EUA Institutional Reviews

Irish Universities Quality Board Similar recommendations in many unit review reports Responses in unit quality improvement plans feed into institutional strategic planning and institutional change Lessons learned from EUA Institutional Reviews

Irish Universities Quality Board Bologna! University Reform in Europe

Irish Universities Quality Board Bologna Declaration (1999) Bologna Process ( ) University Reform in Europe

Irish Universities Quality Board Now signed by 45 European Countries –Adoption of a system of easily readable and comparable degrees based on two main cycles: UG and PG –Establishment of a system of credits –Promotion of mobility for students and staff –Promotion of European co-operation in quality assurance Bologna Declaration 1999

Irish Universities Quality Board EUA ‘Trends’ Surveys –Biannual survey of state of the Bologna Process Bologna Process

Irish Universities Quality Board EUA “Trends IV” survey 2005 Prepared by the European University Association (EUA) to evaluate the ways in which many higher education institutions are responding to the implementation of the Bologna Reforms Based on 62 site visit to HEIs in 29 European Countries (including Ireland)

Irish Universities Quality Board EUA “Trends IV” survey 2005 Findings in relation to Quality

Irish Universities Quality Board EUA Trends IV survey - findings “Among a considerable number of HEIs, which had developed a reform agenda, the most limiting factors to their quality enhancement was not the nature of their internal quality processes but the limits of available resources and the scope of their autonomy, i.e. the extent to which they can decide and plan their own institutional future”

Irish Universities Quality Board EUA Trends IV survey - findings “Institutions with the most systematic approach to quality are also those that benefit from the greatest institutional autonomy” “Institutions find that a well developed internal quality culture should be associated with a light external quality control”

Irish Universities Quality Board EUA Trends IV survey - findings “Generally, institutions had experienced internal quality processes to be more improvement oriented than external quality assurance processes, which were felt to be either more geared towards control and compliance or less attuned to the aims, priorities and conditions under which the institutions or evaluated unit was developing”

Irish Universities Quality Board Final thoughts EUA Sectoral Report on Irish Universities

Irish Universities Quality Board “While the Irish universities are pioneers in developing and implementing a systematic quality review and quality improvement system by themselves,.. they are not yet leaders in using the outcomes of the system to stimulate strategic change inside the universities” “This should be a priority during the next phase of developing the QA system”

Irish Universities Quality Board