Examining a Higher Degree: The View From The Other Side Professor Max Bramer University of Portsmouth United Kingdom www.maxbramer.org

Choosing an External Examiner Nominated by the Director of Studies Student can express a preference Must not be too close to project Academic in UK university (usually) Fairly senior (preferably) Experienced examiner (normally) Internal Examiner is chosen mainly for availability IE and EE are equal but EE is primus inter pares

Choosing an External Examiner (2) EE is an expert in areas relating to yours but not necessarily precisely your area You are the leading expert in your area (temporarily) Rule 1 (for Director of Studies) Avoid the ‘partisan’ EE opposed to research in any subfield and by any research group but their own

Awarding a Higher Degree The Student’s View 3 years (or more) of hard work and false starts 150-200 pages or more of text Frequently many drafts Summary justice in a morning or afternoon after a sleepless night

Awarding a Higher Degree The Examiner’s View Examining fitted into a busy schedule Short period of intensive reading (say 2-3 days), plus a lifetime’s experience Personal interest in the topic Onerous travelling in many cases Small fee, high responsibility Rule 2 Make the thesis as interesting and as clear as possible for someone in a hurry

What’s at Stake? Student Future career and income Three year’s work potentially wasted Status with friends, family etc.

What’s at Stake? Examiner Professional reputation (‘who was your external examiner?’) May not be asked again (?) Rule 3 Make the external examiner confident enough to pass you (see later)

Possible Outcomes: PhD (First Attempt) Pass with no changes (<5%) Pass with minor changes checked by int. examiner (75%) Major changes and resubmission after a year with second viva (20%) Compensatory MPhil (<1%) Fail (<1%)

Possible Outcomes: PhD (Second Attempt) Pass with no changes Compensatory MPhil Fail No changes permitted at this stage No further resubmission possible It is best to avoid a resubmission at the first attempt – all effort needs to be devoted to this

What Happens at the Viva: The Official Agenda Present: the two examiners, the candidate and the Director of Studies (by invitation only) as observer The two examiners ask questions in turn (2-3 hours) The examiners confer in private The candidate and observer return to hear the examiners’ recommendation

What Happens at the Viva: The Hidden Agenda Negotiation of which items to include in the list of changes (there are almost always some) Negotiation whether the changes needed are minor or major Most importantly, the examiners are trying to decide whether they are confident to pass the candidate (perhaps with minor changes) If they are not confident, the easiest and safest course is to request a resubmission and a further viva

The Hidden Agenda (2) A Rite of Passage

Starts the day you begin your PhD Preparing for the Viva Starts the day you begin your PhD

Relationship with the Supervisor A good supervisor will see you regularly, put you in contact with the field, help you get through the bureaucracy and do everything to ensure you pass BUT is likely to be busy, so the onus is on you to arrange meetings, send papers for discussion beforehand etc. A good relationship between student and supervisor is essential to the success of any project. If the relationship breaks down irrevocably, consider divorce!

Writing a Thesis Make Chapter 1 a mini-thesis Signposting in each chapter Repetition and redundancy are helpful Order of chapters is important (not necessarily chronological) A thesis is not a diary! Don’t mention lack of time

Writing a Thesis (2) Spelling, punctuation and grammar matter! Clear expression matters (a lot) Non-native English speakers need to arrange for a native speaker to read drafts through carefully

Critiquing Thesis Drafts One of the supervisor’s key tasks, so make sure he/she has time to do the job properly Expect several points marked per page (some trivial, others important) even in ‘near-final’ drafts If this step is not done thoroughly, the examiners will be sure to do it!

Assessing Your Progress Submit your work to public scrutiny (both internal and external) as often as possible Seminars Workshops Working papers Conference and Journal Papers Discuss your work with people outside your field Start doing all this as early as possible

Preparing for the Viva Rehearse answers to standard questions “What is the original contribution to knowledge of this work?” “What are the limitations of your approach?” “How would you propose to develop your work further?” Anticipate likely lines of criticism and prepare your response [your supervisor can help with this]

The Viva is an Oral Examination Unless the regulations demand it, do not make a presentation If you are asked to make one, do not read it out! One of the reasons for an oral examination is to confirm that YOU did and fully understand the work presented

The Viva: How to Build the Examiners’ Confidence (1) Expert Systems always take 18 months to build [reference 27]. Rule 4 Do not defend the indefensible

The Viva: How to Build the Examiners’ Confidence (2) Less is more. A modest but accurate and well substantiated claim is enough. Very strong claims to have solved everything are not necessary, will not be believed and are almost certainly untrue. Rule 5 Do not claim too much

The Viva: How to Build the Examiners’ Confidence (3) The discussion will focus on two topics: (a) Your work (b) The broader picture The examiner does not want to believe all you know about is your own work. Rule 6 Make sure you can put your work in context. Why is it important? Why did you not use a different method? How does your method relate to other approaches?

Defending a Thesis Examiners will have already formed an opinion based on the written thesis. A good viva can improve it. A bad one can ruin it. Most examiners would much prefer to pass the candidate and want to be persuaded to do so. Vital that answers make the examiners more confident not less. An open mind and a proper scientific approach is far more important than trying to justify every word of the thesis (pass mark is not 100%).

Rule 7: Make Sure Your Conclusions Follow From Your Premises

What to Avoid (1): Mind Your Language Everyone knows that … There is no doubt that … I have proved that … X is well-known to be the best method of … It is clear that … No-one can dispute that …

What to Avoid (2): Do Not Criticise Other Researchers All previous researchers have made the false assumption that … Einstein had the mistaken view that … No-one else has ever worked on …. Just say what you have done and examine the evidence for its being a small improvement on previous work. That is all that is needed.

What to Avoid (3): Do Not Criticise Other Researchers Especially not the examiner’s friends…. Dr. X clearly does not understand the work of John Stuart Mill. Summary of 1 to 3: Do not waste the examiners’ goodwill by unsupported and unscientific comments on fringe issues.

What to Avoid (4): Small Examples … … that do not scale up e.g. model-based reasoning for a circuit with 5 components (6 possible solutions) … that could easily be handled by other methods

What to Avoid (5): Missing/'Optimistic' Evaluation 'Fully evaluated' but no results in the thesis 'Best results in the world' (see reference 24).

What to Avoid (6): Unsound Evaluation Two data mining examples … Evaluating a trained model on the training data Using a model on unseen data needs a human expert to be present

What to Avoid (7): Plagiarism If you must plagiarise don't use Wikipedia! Phrases such as 'the following account is based on Brown and Jones (2003) and Williams, Smith and Wilson (2008)' can save a lot of pain.

What to Avoid (8): Horror Stories X’s algorithm runs 1000 times faster than C4.5 A method which chooses between 2 equally likely possibilities with 30% accuracy A method that predicts rare events less accurately than chance It is not necessary to be an expert to spot problems like these. Any competent reader should be able to do it. So why do mistakes like these keep being made?

‘You make your own luck’ And Finally …… Good luck! But remember … ‘You make your own luck’

Examining a Higher Degree: The View From The Other Side Professor Max Bramer University of Portsmouth United Kingdom www.maxbramer.org