Taught Postgraduate Program Review On-line briefing for Program Directors and Staff

TOPICS Background Key Features Self-Evaluation Review by External Panel Review Report

1: BACKGROUND #1 Audit Report November 2010 Recommendation 5 “The QAC recommends that HKUST introduce a system of periodic reviews of all taught programmes, with defined frequency, terms of reference and requirements for action and follow-up on review recommendations.”

1: BACKGROUND #2 Report of Provost’s Task Force on Educational Quality and Learning Outcomes of TPg Programs: March 2013 Recommendation The Task Force suggests that there should be a comprehensive review of the TPg programs periodically at a minimum frequency of five to six years, with defined terms of reference and requirements for action and follow-up on review recommendations. The periodic review should include benchmarking against similar programs, and some external element, such as inviting faculty from other local institutions as external reviewers and/or employer bodies to review and provide a report on the programs.

2: KEY FEATURES Senate approval of policy: 3 December 2013 TPg programs to be reviewed every 5 years Individual programs or cognate groups 4 stages: Internal review and production of SED External Panel review of documentation and visit Production of TPg Program Review Report/Action Plan Consideration of Report and Action Plan Pilot in 2014 (4 programs); full implementation from 2015

3: SELF-EVALUATION #1 Self-Evaluation Document (SED) - core areas: Program Delivery, Management, Quality Assurance The Curriculum, Benchmarking, Quality Enhancement Students: Intake; Performance; Support Stakeholder Feedback Assessment Resources and Risk Other Review Components Plans for Development Include (in 7) aspects of review specific to individual programs

3: SELF-EVALUATION #2 Program Director collates documentation and data, and drafts SED SED to be an evidential-based self-critical evaluation of the relative success of the program during the last five years in regard to the core areas, incorporating data analysis and references to documentation as appropriate. SED and documentation + data discussed and analysed by program teaching team and administrators

4: REVIEW BY EXTERNAL PANEL #1 Panel Visit to be determined by Program Director Ideally 1.5-2 days, to allow scrutiny of documentation, meetings (see below) and discussion of review outcomes and recommendations Visit Program   External Review Panel meetings to include: The Dean (and, where appropriate, the responsible Department Head) The Program Director, the program teaching team and the program’s administrators Support, including technical, staff as appropriate Current students (both full-time and part-time, where appropriate) The program’s graduates, if available Local employers of the program’s graduates The Dean (and, where appropriate, the responsible Department Head) and Program Director at the conclusion of the visit External Review Panels also may wish to tour the facilities and review the learning resources available to the program and its students

4: REVIEW BY EXTERNAL PANEL #2 Documentation   SEDs to be discussed with External Review Panels during visits. Additionally, the documentation and data that underpin internal review should be made available to External Review Panels during their visits and is likely to include the following: the program specification, to include the course structure and program and course objectives, as approved by the University; course outlines/other documentation provided to students for all courses in the academic year of review; a summary of approved changes to the program since its inception, with the rationale; the program management structure, including details of the teaching team and committees concerned with the quality assurance of the program; details of any benchmarking (local or international) of the program and/or the assessments; the Program Annual Report on Taught Postgraduate Education for each year of the review period, with a critical commentary on actions taken arising from the reports; Minutes of the Staff Student Liaison Committee and other committees associated with the program, for the year of review and the previous two years; critical commentaries on feedback from stakeholders during the review period and on the relative success of any actions taken arising from such feedback (to include results of Student Feedback Questionnaires (SFQ), the TPg Exit Survey and any employer feedback; and feedback from Staff Student Liaison Committees); a range of assessments and associated marking schemes, model answers and student work from each year of the review period, across a sample of courses; Intake data, for each year of the review period, with a critical analysis - Section 3 of the Self-Evaluation Document refers; marks/grades and program award data for each year of the review period, with a critical analysis - Section 3 of the Self-Evaluation Document refers

4: REVIEW BY EXTERNAL PANEL #3 External Reviewers’ Report covers: Nature of Engagement Syllabus and Curriculum Assessment and Academic Standards Student and Graduate Support and Feedback Additional Information Good Practice Overall Report Additional comments To be submitted to AP (T&L) within two weeks of Panel Visit (forwarded to Program Director/Dean (and responsible Department Head))

5: REVIEW REPORT #1 1. External Subject Specialist and Review Panel Indicate the extent of engagement with the External Subject Specialist during the review period; Summarise the major findings of the External Review Panel, with particular reference to any proposed changes to the program, the appropriateness of student assessments and grade/award distributions, and the relative standards of student achievement in relation to international benchmarking.   2. Overall Evaluation and Forward Look Taking account of the suggestions of the External Review Panel on the self-evaluation document and following their visit, provide a critical commentary on the relative success of the program, with specific reference to strengths and areas for improvement. 3. Recommendations and Action Plan List the Recommendations arising from the review. Append an Action Plan to indicate how the recommendations and related actions would be taken forward, the timescales for implementation, milestones to allow monitoring and those responsible for each action. [example action plan templates @ http://qa.ust.hk/AnnualReports/ActionPlan.pdf]

5: REVIEW REPORT #2 TPg Program Review Reports to be submitted, within four weeks of receipt of External Reviewers’ Reports, to: Dean (via responsible Department Head where appropriate); CTLQ and CPS (for comment and subsequent monitoring, as deemed appropriate); Executive Vice-President and Provost (for information) Append External Reviewers’ Report and Action Plan Deans comment on TPg Program Review Reports in School Annual Reports Program Annual Reports on Taught Postgraduate Education will not be required in the years in which Taught Postgraduate Program Reviews are undertaken Program Annual Reports in the years following Taught Postgraduate Program Reviews should be accompanied by progress reports on implementation of action plans, indicating reasons for any delay in implementation, where appropriate

6: NEXT STEPS Program Director to: Identify Panel members (academic/subject specialist from peer institution outside HK preferably with QA experience for example as a Program Director, Head of Department, Dean, etc.; local alumnus or employer; senior HKUST academic from another School primarily to comment on and provide advice on the program’s management and quality assurance arrangements in relation to the University’s procedures and quality assurance framework, and to guide the external members of the Panel accordingly) Seek Dean’s approval and submit nominations to Associate Provost (T&L) for CTLQ approval Write to nominees, to offer appointment and agree date for Panel visit (draft letter to be provided by AQUS, to include all relevant details, including requirements, fees etc.) Undertake internal self review (up to 2 months): Collate and critically analyse documentation and data sets Draft SED Arrange meetings of Program Team, undertake further review/analysis and revisions of SED Send final SED to Panel for consideration, in advance of Panel Visit Agree program and documentation for visit with Panel Chair and make arrangements for visit, including organising and briefing those who are to meet the Panel