Examining a Higher Degree:The View From The Other Side

Professor Max BramerUniversity of Portsmouth

United Kingdom

www.maxbramer.org

Choosing an External Examiner

• Nominated by the Director of Studies• Student can express a preference• Must not be too close to project• Academic in UK university (usually)• Fairly senior (preferably)• Experienced examiner (normally)• Internal Examiner is chosen mainly for

availability• IE and EE are equal but EE is primus inter pares

Choosing an External Examiner (2)

• EE is an expert in areas relating to yours but not necessarily precisely your area

• You are the leading expert in your area (temporarily)

Rule 1 (for Director of Studies)

Avoid the ‘partisan’ EE opposed to research in any subfield and by any research group but their own

Awarding a Higher DegreeThe Student’s View

• 3 years (or more) of hard work and false starts• 150-200 pages or more of text• Frequently many drafts• Summary justice in a morning or afternoon after

a sleepless night

Awarding a Higher DegreeThe Examiner’s View

• Examining fitted into a busy schedule• Short period of intensive reading (say 2-3 days),

plus a lifetime’s experience• Personal interest in the topic• Onerous travelling in many cases• Small fee, high responsibility

Rule 2

Make the thesis as interesting and as clear as possible for someone in a hurry

What’s at Stake?

Student

Future career and income

Three year’s work potentially wasted

Status with friends, family etc.

What’s at Stake?

Examiner

Professional reputation (‘who was your external examiner?’)

May not be asked again (?)

Rule 3

Make the external examiner confident enough to pass you (see later)

Possible Outcomes: PhD(First Attempt)

• Pass with no changes (<5%)• Pass with minor changes checked by int.

examiner (75%)• Major changes and resubmission after a

year with second viva (20%)• Compensatory MPhil (<1%)• Fail (<1%)

Possible Outcomes: PhD(Second Attempt)

• Pass with no changes• Compensatory MPhil• Fail

No changes permitted at this stageNo further resubmission possible

It is best to avoid a resubmission at the first attempt – all effort needs to be devoted to this

What Happens at the Viva:The Official Agenda

Present: the two examiners, the candidate and the Director of Studies (by invitation only) as observer

The two examiners ask questions in turn (2-3 hours)

The examiners confer in privateThe candidate and observer return to hear

the examiners’ recommendation

What Happens at the Viva:The Hidden Agenda

Negotiation of which items to include in the list of changes (there are almost always some)

Negotiation whether the changes needed are minor or major

Most importantly, the examiners are trying to decide whether they are confident to pass the candidate (perhaps with minor changes)

If they are not confident, the easiest and safest course is to request a resubmission and a further viva

The Hidden Agenda (2)

A Rite of Passage

Preparing for the Viva

Starts the day you begin your PhD

Relationship with the Supervisor

A good supervisor will see you regularly, put you in contact with the field, help you get through the bureaucracy and do everything to ensure you passBUT is likely to be busy, so the onus is on you to arrange meetings, send papers for discussion beforehand etc.

A good relationship between student and supervisor is essential to the success of any project. If the relationship breaks down irrevocably, consider divorce!

Writing a Thesis

• Make Chapter 1 a mini-thesis• Signposting in each chapter• Repetition and redundancy are helpful• Order of chapters is important (not necessarily

chronological)• A thesis is not a diary!• Don’t mention lack of time

Writing a Thesis (2)

Spelling, punctuation and grammar matter!

Clear expression matters (a lot)

Non-native English speakers need to arrange for a native speaker to read drafts through carefully

Critiquing Thesis Drafts

One of the supervisor’s key tasks, so make sure he/she has time to do the job properly

Expect several points marked per page (some trivial, others important) even in ‘near-final’ drafts

If this step is not done thoroughly, the examiners will be sure to do it!

Assessing Your Progress

Submit your work to public scrutiny (both internal and external) as often as possible

- Seminars

- Workshops

- Working papers

- Conference and Journal Papers

Discuss your work with people outside your field

Start doing all this as early as possible

Preparing for the Viva

Rehearse answers to standard questions

“What is the original contribution to knowledge of this work?”

“What are the limitations of your approach?”

“How would you propose to develop your work further?”

Anticipate likely lines of criticism and prepare your response [your supervisor can help with this]

The Viva is an Oral Examination

• Unless the regulations demand it, do not make a presentation

• If you are asked to make one, do not read it out!

• One of the reasons for an oral examination is to confirm that YOU did and fully understand the work presented

The Viva: How to Build the Examiners’ Confidence (1)

Expert Systems always take 18

months to build [reference 27].

Rule 4

Do not defend the indefensible

The Viva: How to Build The Examiners’ Confidence (2)

Less is more. A modest but accurate and well substantiated claim is enough. Very strong claims to have solved everything are not necessary, will not be believed and are almost certainly untrue.

Rule 5

Do not claim too much

The Viva: How to Build the Examiners’ Confidence (3)

The discussion will focus on two topics

(a) Your work

(b) The broader picture

The examiner does not want to believe all you know about is your own work.

Rule 6

Make sure you can put your work in context. Why is it important? Why did you not use a different method? How does your method relate to other approaches?

Defending a Thesis

• Examiners will have already formed an opinion based on the written thesis. A good viva can improve it. A bad one can ruin it.

• Most examiners would much prefer to pass the candidate and want to be persuaded to do so

• Vital that answers make the examiners more confident not less

• An open mind and a proper scientific approach is far more important than trying to justify every word of the thesis (pass mark is not 100%)

Rule 7: Make Sure Your Conclusions Follow From Your Premises

What to Avoid (1):Mind Your Language

Everyone knows that …

There is no doubt that …

I have proved that …

X is well-known to be the best method of …

It is clear that …

No-one can dispute that …

What to Avoid (2):Do Not Criticise Other Researchers

All previous researchers have made the false assumption that …

Einstein had the mistaken view that …

No-one else has ever worked on ….

Just say what you have done and examine the evidence for its being a small improvement on previous work. That is all that is needed.

What to Avoid (3):Do Not Criticise Other Researchers

Especially not the examiner’s friends….

Dr. X clearly does not understand the

work of John Stuart Mill.

Summary of 1 to 3: Do not waste the examiners’ goodwill by unsupported and unscientific comments on fringe issues.

What to Avoid (4):Small Examples …

… that do not scale up

e.g. model-based reasoning for a circuit with 5 components (6 possible solutions)

… that could easily be handled by other methods

What to Avoid (5):Missing/'Optimistic' Evaluation

'Fully evaluated' but no results in the thesis

'Best results in the world' (see reference 24).

What to Avoid (6):Unsound Evaluation

Two data mining examples …

Evaluating a trained model on the training data

Using a model on unseen data needs a human expert to be present

What to Avoid (7):Plagiarism

If you must plagiarise don't use Wikipedia!

Phrases such as 'the following account is based on Brown and Jones (2003) and Williams, Smith and Wilson (2008)' can save a lot of pain.

What to Avoid (8):Horror Stories

X’s algorithm runs 1000 times faster than C4.5

A method which chooses between 2 equally likely possibilities with 30% accuracy

A method that predicts rare events less accurately than chance

It is not necessary to be an expert to spot problems like these. Any competent reader should be able to do it. So why do mistakes like these keep being made?

And Finally ……

Good luck!But remember …

‘You make your own luck’

Examining a Higher Degree:The View From The Other Side

Professor Max BramerUniversity of Portsmouth

United Kingdom

www.maxbramer.org