1. Handbook for Internal Subject Review Team Members 2013/14 1

2. Contents Purpose of this handbook and ISR training4 The Purpose of ISR8 Key features of the process14 Team Roles and Responsibilities18 Managing the process31 How to ask questions38 How to make commendations and recommendations40 2

3. Contents (cont.) Appendices: 1.List of key documents about the ISR process 2.ISR Strategic Themes for 2012/13 3.Suggested ISR Review Visit schedule 4.ISR Timeline 5.Template for meeting preparation 6.Commentary exercise 7.How to ask questions during a review Visit 8.Commendations and Recommendations 9.Recommendations exercise 10.Previous examples of features of exemplary practice Appendices can be found at:- http://www.ncl.ac.uk/quilt/assets/documents/qsh-isr-handbook_appendices.pdf 3

4. Purpose of the Training and Handbook After reading this handbook and/or attending training ISR team members should be able to: Articulate the purpose of ISR Appreciate what the key features of the process are Understand how ISRs are planned, and, with support, how to carry out and follow up an ISR Understand the role of the team members 4

5. Purpose (cont.) Understand the special roles of the Chair and the Secretary Understand how to use the commentary and other evidence Pose suitable questions in an appropriate way Word commendations for good and exemplary practice and recommendations appropriately 5

6. Accentuate the positive We can learn more from what works well than from what works badly What works well in one context can often be transferred to another Messages which are constructive and positive in tone are likely to be better received Makes the event a more positive experience for everyone 6

7. Key documents The key documents are in the ISR section of the Quality and Standards Handbook at http://www.ncl.ac.uk/quilt/resources/monitoring/inter nal See also Appendix 1 7

8. The Purpose of ISR 8

9. Institutional responsibility Primary responsibility for academic standards and quality in UK HE rests with individual universities, each of which is independent and self-governing QAA checks how well they meet their responsibilities, identifying good practice and making recommendations for improvement 9

10. QAA requirements Formal and effective procedures for the design, approval, monitoring and review of programmes Processes should foster creativity, and encourage a culture of continuous enhancement of provision Institutions should have means of assessing the effectiveness of their programme design, approval, monitoring and review practices 10

11. How we fulfil QAA requirements Mechanisms for annual and periodic review (Annual Monitoring and Review – AMR; and Internal Subject Review – ISR). Two parts of the same process of continuous improvement Programme Approval Internal Audit 11

12. How ISR can add value Self-reflection on whether programmes remain current and are effectively taught External and student input Constructive and positive engagement Opportunity to bring matters to the attention of Faculty and University Highlighting areas of exemplary practice 12

13. Outcomes of the ISR process A positive experience! Confirmation that standards are secure, there is a high quality learning experience Re-approval of degree programmes for a further six year period Formal report considered by faculty and University Recommendations to assist the subject area with continuous improvement of provision 13

14. ISR – key features 14

15. The key features are... Enhancement-led - emphasis on ISR as a developmental process which encourages enhancement of teaching and learning processes as well as disseminating exemplary practice A Review Team visit to the School/subject area of one day The inclusion of a student representative on the Review Team 15

16. Key features(cont.) Role for the Chair as critical friend Compliance review - the secretary to the Review Team conducts a review of local implementation of quality assurance and enhancement processes, and of programme documentation Priority review themes for each year (Appendix 2) 16

17. Key features (cont.) Training for Review Team members Annual evaluation of the process involving Review Team members 17

18. The role of the members of the team 18

19. What does a team consist of? Chair – usually an academic from another faculty (ULTSEC nominee) An academic from another faculty (ULTSEC nominee) An academic from the same faculty, but a different school (FLTSEC nominee) A student rep nominated by the Students’ Union An external expert nominated by the subject ISR Secretary from QuILT There may be other observers 19

20. Team’s responsibility The whole team is responsible for the conduct of the ISR and agreement to the report This is a collective task However, some members of the team have special roles 20

21. Shared tasks Reading the commentary and reviewing other evidence before the visit Preparatory meeting (external remotely) Preparing for meetings Debriefing after meetings Agreeing exemplaries, commendations and recommendations Commenting on the draft report 21

22. Role of the external Only subject specialist in the team Of key importance in judging the curriculum, standards, currency, compliance with benchmarks and PSRB requirements, and how the provision compares with that available elsewhere Vitally important and required by QAA Will lead on curriculum issues 22

23. Role of the Chair Also vitally important, but in a different way BEFORE THE VISIT Meeting the subject area before the visit Chairing the preparatory meeting Ensuring the schedule is agreed (Appendix 3) 23

24. Role of the Chair (cont.) DURING THE VISIT Keeping to schedule Keeping focused Ensuring there is preparation for meetings and debriefing afterwards at which the key points are recorded Giving oral feedback at the end of the day 24

25. Role of the Chair (cont.) AFTER THE VISIT Final say on the draft report 25

26. Role of the Student Member A full member of the ISR team Selected from another area and not there to represent the students in the subject Likely to bring some different perspectives Will take part in preparatory meeting Will ask questions in all meetings May chair a session 26

27. Role of other academic members Full members of the team Will ask questions in all sessions May chair a session 27

28. Secretary GENERAL Take a proactive approach to advising the Review Team Help to foster a collaborative culture within the team and to support an enhancement-led approach Ensure that all Review Team members and the subject area understand the purpose of ISR and how the process works BEFORE THE VISIT Make arrangements and liaise with the subject starting many months before Liaise over appointment of external and fixing the date of the visit Carry out compliance review Negotiate with the subject area over the schedule and other logistical arrangements (See Appendix 3) 28

29. Role of the secretary (cont.) DURING THE VISIT Liaise with School contact about arrangements, document requests etc. Take notes during sessions Help team devise questions for sessions and debrief afterwards, recording agreed key points Helps chair and team agree exemplaries, recommendations and feedback 29

30. Role of the secretary (cont.) AFTER THE VISIT Draft the report Liaise with team members Send draft report to subject for factual checking Send final report to subject and to relevant FLTSEC secretary 30

31. Managing the process 31

32. Planning the visit and designing the questions ISR – Timeline (Appendix 4) The commentary and other documents provided (e.g. handbooks; externals’ reports; programme data; programme specifications etc.) form the initial basis for identifying areas to follow up and planning the visit The compliance review carried out by the ISR secretary may also contribute useful pointers The meeting with students may reveal new issues 32

33. Evidence-based process This is an evidence-based process Where the documentation provides sufficient evidence to reach a conclusion, then there is no need for oral questions Sometimes additional documents may (within reason) be requested Oral evidence from one source ideally needs confirmation 33

34. Using scarce time profitably One day is an incredibly short period of time to gather evidence It has to include time for reviewing some documents and for discussion and reflection as well as for meetings Questioning has to be very strategic and focused on the most important issues emerging from the team’s review 34

35. Using time profitably Sessions must be prepared for identifying a chair, topics to pursue, who will ask what and rough time scales Sometimes you have to drop something in a session because time runs out, so it is important to know what matters most Session chair needs to keep team and interviewees focused and prevent digression Useful to have a rough schedule (see Appendix 5i/5ii) 35

36. Follow up After each session the team should agree a small number of bullet points relating to the key issues identified in the session Be on the look-out for commendations and recommendations 36

37. Exercise 1 Exercise on reviewing providers’ commentaries - See Appendix 6 37

38. How to ask questions Clarity of purpose “Etiquette” of questioning Structuring lines of questioning – what are you trying to do? – why are you doing it? – how are you doing it? – why is that the best way to do it? – how do you know it works? – how can you improve it? See also appendix 7 38

39. Exercise 2 In the meeting with students, concerns were raised by some PGT student representatives about their experience of personal tutoring. They indicated that they had raised this at the Staff Student Committee but had not received a response. This does not seem to support the assertion made by the School that in the last three years they have reformed personal tutoring and have fully implemented the new student representation policy. In groups, construct a line of questioning that explores this with members of staff 39

40. Commendations and Recommendations What is exemplary? What is a commendation? Recommendations SMAR(T) goals See also appendix 8 40

41. Exercise 3 Exercise on wording a recommendation using scenarios provided and a review of some actual exemplaries to identify which ones work better [See appendices 9 and 10] Feedback at the end 41

42. And finally... …thank you for attending this workshop Questions Contacts Feedback 42