Review visit Heli Mattisen

Visit to one HEI – 1-3 days Committee has a working room where, among others, pre-requested documents/materials Reflection slots between interviews (incl. lunch and coffee breaks) Duration of interviews min Recommended: not to have more than 6 interviewees at a time Final meeting to staff and students: presentation of preliminary conclusions (without questions/discussion) General 2

access statutes and normative documents which provide for and govern the activities of the higher education institution and its structural units; interview employees and students of the higher education institution at the choice of experts; access information related to teaching, learning and students; educational materials; and the study information system; access information related to the teaching staff concerning their CVs, work load, methodological work and research activities; Rights of the assessment committee 3

examine the internal quality assurance system of teaching and learning; examine the condition of the infrastructure of the higher education institution; access subject syllabuses/programmes as well as instructional documents related to the content of teaching and learning; access students’ final papers; access information related to financial activities of the higher education institution; visit any forms of contact learning (lectures, seminars, laboratory work, etc.). Rights of the assessment committee (cont.) 4

examine documents regulating accreditation and complete assessment training provided; review the self-evaluation report of HEI, pre-fill in the assessment report according to the tasks divided between the committee members, and prepare and submit to the coordinator a list of topics/questions to be focused on in the course of assessment visit by assessment areas as well as a list of additional materials they would like to get from the institution. Before the visit 5

Agree among the committee what questions to ask from whom. Ask similar questions from different people (head of department, teachers, students, …) Divide the questions among the committee members Decide who chairs the interviews Decide how to take notes Adhere to the agreed division of tasks and committee deadlines Before the visit (cont.) 6

Drafting and asking questions Helka Kekäläinen Baku

Site visit questions  Principal way of getting information - the teams are also encouraged to use other methods to complement the picture.  Themes or prepared questions? Prepared questions / question lists, because:  To ensure that the different aspects related to the criteria are covered.  To create a common understanding within the team of the topics discussed during the site visit.  Provides support for the interviews, functions as a check list.  However, remember to use question lists flexibly. Give room for interview dynamics and follow-up questions

How to ask good questions?

Formulation is everything

Open question How do you gather feedback from students? With questionnaires after each course.

Direct question Do students have access to the intranet? Yes they do.

Leading question Do you have a system with which you gather feedback, which is then analysed and used to improve operations ? Absolutely.

Two-part question Do you gather feedback from students and are your indicators linked to your strategy ? Yes, they are linked to the strategy.

Bad questions create empty answers

How?Why? Please assess… Please provide examples of … Good questions – some examples

(Introduce yourselves) and try to reduce strain at the beginning of each interview! (We are here to support your development etc.). NB! Very important as it has a strong impact on the success or failure of an interview! Opening an interview 17

Pose short and easily understandable questions, one question at a time! Use „quality-cycle“ questions: What have you done? Why have you done so? What are the results? Are they good or bad? Why do you think so? How do you use them? What have you learned from the process?... Directing (advising) questions should be avoided (Wouldn’t it be better if you...? Why don’t you...?) Let everyone talk. Address questions to specific persons if necessary. Reduce domination. Interrupt if necessary. Core of an interview 1 18

Reflect back what has been said in order to make sure that you have understood the point Always remember your goals (what we want do know and why)! Keep the interview within the time frames you set Collect evidences! Show your interest with appropriate body-language Keep eye-contact Listen! Avoid the tendency of filling the interview with your own voice It is not polite to eat or drink during an interview Core of an interview 2 19

Thank interviewees for their responses and time If possible, leave 2-3 minutes for additional question „Is there anything we did not ask but you would like to tell us“? Avoid making any premature conclusions („You have a fabulous/horrible institution!“ If you think you still need further information, agree on next steps Ending an interview 20

Discuss and decide whom you want to meet during an interview (up to 2 people) and which questions to ask. An interview lasts max 10 minutes – please keep the time! Decide the strategy of an interveiw: who will chair it, how you will start, how you give word to each other, how you will end the interview etc. Teamwork 21

Examples of follow-up questions  That’s interesting. Can you tell me more about …  We haven’t got a clarity about… Can you please explain?  Can you give a practical example of how … works in practice?  Is this a standard approach?  How has this been carried forward/followed-up?  What is the institution planning to do about this?  How does this affect you/your department/student group…?  How would you go about this?