Assessment and Examination Dianne Ford Graduate School Faculty of Medical Sciences University of Newcastle Chairman, Ladies and Gentlemen. I would like to describe a series of experiments performed by our group to determine how acute inflammation might contribute directly to the development of chronic renal allograft nephropathy.

Assessment Why? Monitor student progress Identify problems with Project Supervisor(s)-student relationship Completion rate ‘exit’ strategy for struggling students Student re-registration Increased focus on the timeline

Assessment When? http://www.ncl.ac.uk/fms/postgrad/documentation/documents/Restud2012-13_final.pdf

Assessment When? http://www.ncl.ac.uk/fms/postgrad/documentation/documents/Restud2012-13_final.pdf

Assessment What? First assessment

Assessment Intermediate progress reviews

Assessment Final progress review

Before the interview Read all the documents! Read the student’s research report Read the report from the student for the progress panel

Read the report from the supervisor

Your report

Exit strategy If you feel a student is struggling do not give him or her the benefit of the doubt! Students who fail to complete often show weak 1st reviews But in some cases a prod in the first year can be very beneficial! If a student “exits” within 12 months, they don’t appear on our overall completion statistics. And some (not all) grant bodies regenerate the funds for a replacement student

The Viva Voce

Purpose of an examination For the University To assess and maintain quality To mark ‘completion’ of the degree programme For the Student Potentially leads to award of a degree Is an important and memorable life event can be a real emotional roller coaster (for everyone involved)

Who needs an oral examination? All PhD and MD candidates Students need one internal and one external examiner All staff candidates require two external examiners and an internal ‘moderator’ Not all MPhil candidates Same criteria for examiners as above Oral examination held at examiners request Not only for weak students With a good candidate can be fun for everyone!

Appointment of examiners See nomination forms on Graduate School http://medical.faculty.ncl.ac.uk/postgraduate/internal/staff.html Completed by supervisor and Head of school/Institute Examiners must: be cognisant of standards have subject knowledge (need CV) command authority not have played a role in the research (if external) not have been a member of Newcastle staff for at least 3 years be able and willing to examine

Internal examiners Do NOT organise the exam, food, accommodation, travel, etc This is the supervisors responsibility. Ensure the examination complies with Newcastle University protocol Ensure appropriate report forms are completed and submitted in a timely manner Provide balance, fairness and ensure good examination conduct Potentially play a role during any appeals process

External examiners Often is the scientific specialist Is an experienced examiner Chicken and egg…. Often thought to have the ‘casting vote’ But there are procedures for disagreement Maintains inter-university quality Report provides important feedback to Graduate School (and QA etc) May not be completely ‘up to speed’ with local regulations!

What if 2 external examiners? Need to appoint an internal moderator This person need not read or understand the thesis Role is to provide advice on Newcastle examination process May be required to present records of the exam if the result is questioned (appeal process).

Criteria – all theses Should be: Authentic Scholarly Professional Well-structured, written and presented

MPhil candidates Should Theses need not Demonstrate advanced knowledge Have good knowledge of literature Theses need not Demonstrate consistent originality Be worthy of publication

PhD/MD candidates Should provide evidence of adequate industry demonstrate ability for originality understand relationship with wider field thesis should contain material worthy of publication

Types of thesis ‘Standard’ By publication Divided into chapters with results and interpretations By publication A review and a series of ~5 related papers Can be difficult to examine as papers have already satisfied external referees! Staff candidates only

Examiners handbook http://www.ncl.ac.uk/students/progress/staff-resources/pg-research/examiners.htm

Reading a Thesis Are you a proof reader or a scientist? You will need to provide a list of corrections if you require them For a good thesis, I (and most colleagues) usually stick “post-it” notes to the margin to localise my questions within the thesis Be sensitive though; hundreds of these can look very scary! If the thesis is poor, it might be better to have a more detailed critique with lists of specific questions and problems.

Preliminary Report Regulations vary between institutions -read them! Many (but not all) institutions require examiners to independently produce reports before the examination Some need these to be submitted (well) before the examination to flag up potential problems Some don’t require submission of these reports until after the examination! But they should be exchanged with the other examiner’s report on the day.

Newcastle

Imperial college

How to conduct the examination - 1 Arrange the room Often good to have pencils and paper to draw on At the start candidates can be very nervous! Put them at their ease if possible with a soft start (but don’t anticipate the result!) Remember to arrange refreshment breaks Consider the candidate’s bladder

How to conduct the examination - 2 Agree a plan with your co-examiner Remember: Oral examination of a good candidate can/should be one of the most pleasurable academic experiences for all involved Examination of a poor thesis/candidate can be truly awful! You need to devise different strategies for both situations

Questioning The soft start “what result in your thesis are you most proud of?” “what led you to choose this study” “what are you doing now?”

How long should the exam last No fixed duration but >3 hours is exhausting for everyone. Often examination of good students will last longer! Use your judgement

At the end (if all has gone well) Ask the candidate to withdraw for a few minutes REMEMBER you do not award the degree!! Work out what you wish to say, then invite the candidate back Tell the candidate what recommendation you will be making to the higher degrees committee (or other appropriate authority). Smile and shake hands

Examination Results 2011

Can you do more to recognise excellence? In many countries, 1st rate PhD students can be defined. the French system allows PhDs to be awarded as "honourable (not very good)", "very honourable" (average) and "very honourable with felicitations" (top 5%). there is no similar recognition in the UK. In Newcastle we ask examiners (on a separate report sheet) to indicate whether they consider the thesis to be in the top 10% of theses they have examined. A committee considers this recommendation and prizes are awarded. We hope this will be good for the student’s CV.

At the end (if it has NOT gone well) - 1 Make absolutely sure you know what your options are (read the regulations) You are not obliged to tell the candidate anything (although you will feel some pressure to do so) the candidate will receive written confirmation of the outcome in due course You may wish to speak to the supervisor You might need clarification of the regulations (Graduate School)

At the end (if it has NOT gone well) - 2 You will need to fill in the report form with very comprehensive details of any changes you require This outcome results in much more effort in the future for the examiners! All report forms look different. Make sure you know the precise significance of a tick in every optional box! For example, ticking box 3 (pass) at Newcastle can produce a very different outcome from box 3 at Imperial College (fail)!

The appeal process www.ncl.ac.uk/spo/AppealsForm.pdf

Invitation to examine This is not necessarily an honour! Not even a valued career move? Why you and not someone else? Are you a ‘soft touch’? Are you the supervisor’s best friend? You will/should see the abstract of the thesis at the time of invitation Think about the thesis You have the right to REFUSE!

Case study X agreed to act as external examiner for Y’s PhD thesis. The thesis was of marginal quality but X’s preliminary report suggested that with modification and a satisfactory oral examination it was likely that a positive recommendation could be made. Close scrutiny of the thesis immediately before the examination revealed references to an earlier MD thesis by the same candidate (at the same university). X contacted the internal to request that a copy of this thesis should also be available for scrutiny (it is reasonable to see all reference sources cited in a thesis). Before meeting the candidate on the day of the oral, X and the internal discovered that the earlier thesis reported about 70% of the data presented in the PhD thesis (about 45 pages of the introduction to both works was identical). X and the internal examiner commenced the examination. Discuss the potential outcomes.