assessment and examination dianne ford graduate school faculty of medical sciences university of...
Post on 28-Dec-2015
216 Views
Preview:
TRANSCRIPT
Assessment and Examination
Dianne FordGraduate School
Faculty of Medical SciencesUniversity of Newcastle
Assessment
• Why?– Monitor student progress– Identify problems with
• Project• Supervisor(s)-student relationship
– Completion rate• ‘exit’ strategy for struggling students• Student re-registration• Increased focus on the timeline
Assessment
• When? http://www.ncl.ac.uk/fms/postgrad/documentation/documents/Restud2012-13_final.pdf
Assessment
• When? http://www.ncl.ac.uk/fms/postgrad/documentation/documents/Restud2012-13_final.pdf
Assessment• What?
– First assessment
Assessment
– Intermediate progress reviews
Assessment
– Final progress review
Before the interview• Read all the documents!
– Read the student’s research report– Read the report from the student for the progress panel
1. Project/Thesis Title:
2. Progress to date
Yes No
(a) Do you consider that you have made satisfactory progress during the academic year and will be on track
to complete according to your project plan?
(b) Have you had regular contact with your supervisor (approximately monthly or equivalent for part-time students)?
(c) Have you discussed your research training needs with your supervisor?
(d) Have you attended research training events as discussed with your supervisor?
(e) Do you feel that your training needs have been met?
(f) Are there any problems or issues that you would like to draw to the Progress Panel's attention (for example regarding supervision or resources)?
(g) If yes, have you discussed these problems fully with your supervisor, Postgraduate Director/Tutor or Head of School?
Please comment on any individual aspects of your progress with which you are either particulary satisfied or dissatisfied. Please outline any concerns or problems you may have.
Evidence
Current Evidence
– Read the report from the supervisor
1. Progress to Date:
Yes No N/A
(a) Has the student's attendance been satisfactory with respect to the following?
i. Supervisory meetings
ii. Laboratory/Fieldwork (if appropriate)
(b) Have the student's research training needs been discussed?
(c) Has the student attended the training sessions as you agreed?
(d) If the student is a non-native speaker of English, is the language competence satisfactory?
(e) Do you consider that the research facilities and resources are adequate?
(f) Has the student completed their minimum candidature and are they now, in your opinion, eligible for writing up status and reduced fees?
(g) In your view is the student's progress satisfactory and should they be permitted to continue to the next stage? Please indicate below;
i. Yes, without reservation
ii. Yes, with some concerns (noted in Comments)
iii. No (for reasons noted in Comments)
Please detail your concerns and provide general comments and feedback. You should comment on any particular points you wish to commend the student or on any areas of deficiency of which you would like to bring to the attention of the
Progress Panel.
i
Highlight any major risk to the on-time completion of this research project and submission of a satisfactory thesis.
Your report
Exit strategy• If you feel a student is struggling do not
give him or her the benefit of the doubt!• Students who fail to complete often
show weak 1st reviews– But in some cases a prod in the first year
can be very beneficial!• If a student “exits” within 12 months,
they don’t appear on our overall completion statistics.– And some (not all) grant bodies regenerate
the funds for a replacement student
The Viva Voce
Purpose of an examination
• For the University– To assess and maintain quality– To mark ‘completion’ of the degree programme
• For the Student– Potentially leads to award of a degree– Is an important and memorable life event
• can be a real emotional roller coaster (for everyone involved)
Who needs an oral examination?
• All PhD and MD candidates– Students need one internal and one external
examiner– All staff candidates require two external examiners
and an internal ‘moderator’
• Not all MPhil candidates– Same criteria for examiners as above– Oral examination held at examiners request
• Not only for weak students • With a good candidate can be fun for everyone!
Appointment of examiners• See nomination forms on Graduate School
http://medical.faculty.ncl.ac.uk/postgraduate/internal/staff.html
– Completed by supervisor and Head of school/Institute
• Examiners must:– be cognisant of standards– have subject knowledge (need CV)– command authority– not have played a role in the research– (if external) not have been a member of Newcastle
staff for at least 3 years– be able and willing to examine
Internal examiners• Do NOT organise the exam, food,
accommodation, travel, etc– This is the supervisors responsibility.
• Ensure the examination complies with Newcastle University protocol
• Ensure appropriate report forms are completed and submitted in a timely manner
• Provide balance, fairness and ensure good examination conduct
• Potentially play a role during any appeals process
External examiners• Often is the scientific specialist• Is an experienced examiner
– Chicken and egg….• Often thought to have the ‘casting vote’
– But there are procedures for disagreement• Maintains inter-university quality
– Report provides important feedback to Graduate School (and QA etc)
• May not be completely ‘up to speed’ with local regulations!
What if 2 external examiners?• Need to appoint an internal moderator• This person need not read or
understand the thesis• Role is to provide advice on Newcastle
examination process– May be required to present records of the
exam if the result is questioned (appeal process).
Criteria – all theses
• Should be:– Authentic– Scholarly– Professional– Well-structured, written and presented
An Independent Chair
An Independent Chair makes sure the University’s procedures with regard to the examination of
research degrees are followed. They take no part in the assessment process, but ensure that the
examination process is conducted fairly and equitably.
An Independent Chair MUST be appointed in the following circumstances:
Where two External Examiners are appointed.
Where the Internal examiner has no previous experience of examining doctoral degrees.
An Independent Chair MAY be appointed in the following circumstances:
Where the Examiners of the thesis require the assistance of an independent authority to
conduct the examination process.
When the Dean deems an independent authority is needed to ensure the examination
process is conducted fairly.
An Independent chair MUST:
Be an academic member of staff at the University at Senior Lecturer / Reader level or
above.
Be familiar with University examination processes for research degrees.
Have substantial experience of postgraduate research and examination.
An Independent Chair MUST NOT:
Be a member of the supervisory team, or have played any part in the research under
examination.
MPhil candidates
• Should– Demonstrate advanced knowledge– Have good knowledge of literature
• Theses need not– Demonstrate consistent originality– Be worthy of publication
PhD/MD candidates
• Should – provide evidence of adequate industry– demonstrate ability for originality– understand relationship with wider field– thesis should contain material worthy of
publication
Types of thesis
• ‘Standard’– Divided into chapters with results and
interpretations
• By publication– A review and a series of ~5 related papers– Can be difficult to examine as papers have already
satisfied external referees!– Staff candidates only
• Examiners handbookhttp://www.ncl.ac.uk/students/progress/staff-resources/pg-research/examiners.htm
Reading a Thesis
• Are you a proof reader or a scientist?• You will need to provide a list of corrections if
you require them• For a good thesis, I (and most colleagues)
usually stick “post-it” notes to the margin to localise my questions within the thesis – Be sensitive though; hundreds of these can look
very scary!
• If the thesis is poor, it might be better to have a more detailed critique with lists of specific questions and problems.
Preliminary Report
• Regulations vary between institutions -read them!
• Many (but not all) institutions require examiners to independently produce reports before the examination– Some need these to be submitted (well) before the
examination to flag up potential problems– Some don’t require submission of these reports
until after the examination!• But they should be exchanged with the other examiner’s
report on the day.
Newcastle
8. The examiners should independently write a preliminary report indicating their provisional assessment of the thesis and of the issues to be explored in the oral examination. It is expected that, if the criteria for the award of the degree have clearly been met, the preliminary reports will be very brief (a single paragraph). If, on the other hand, the examiners have serious concerns about whether the criteria have been met, fuller reports will be expected. Each examiner's preliminary report should be sent to the relevant graduate school administrator in advance of the oral examination taking place. The reports will be forwarded to the relevant dean of postgraduate studies. They must not be shown to the candidate or the supervisory team in advance of the oral examination. But examiners should be aware that preliminary reports will be made available to candidates after the oral examination if they request them under the provisions of the Data Protection Act.
Imperial college
How to conduct the examination - 1• Arrange the room
– Often good to have pencils and paper to draw on
• At the start candidates can be very nervous!– Put them at their ease if possible with a
soft start (but don’t anticipate the result!)– Remember to arrange refreshment breaks– Consider the candidate’s bladder
How to conduct the examination - 2
• Agree a plan with your co-examiner• Remember:
– Oral examination of a good candidate can/should be one of the most pleasurable academic experiences for all involved
– Examination of a poor thesis/candidate can be truly awful!
• You need to devise different strategies for both situations
Questioning• The soft start
• “what result in your thesis are you most proud of?”• “what led you to choose this study”• “what are you doing now?”
How long should the exam last
• No fixed duration but >3 hours is exhausting for everyone.
• Often examination of good students will last longer!
• Use your judgement
At the end (if all has gone well)• Ask the candidate to withdraw for a few
minutes• REMEMBER you do not award the degree!!• Work out what you wish to say, then invite the
candidate back• Tell the candidate what recommendation you
will be making to the higher degrees committee (or other appropriate authority).
• Smile and shake hands
RESEARCH DEGREE EXAMINATION JOINT REPORT OF EXAMINERS
Student Candidates for Doctoral Awards (including Integrated PhD and Professional Doctorates)
Name of Candidate:
Title of Thesis:
Name and Address of External Examiner:
Name of Internal Examiner:
Date of Oral Examination:
SECTION A - RECOMMENDATIONS (tick the relevant box to indicate overall recommendation)
The Candidate be admitted to the degree
(a)(i) That the candidate be admitted immediately to the degree of Doctor of Philosophy.
(a)(ii) That the candidate be admitted to the degree of Doctor of Philosophy subject to minor corrections of the text made to the satisfaction of the internal examiner, normally within a period of one month of receiving formal notification of the corrections to be made.
(a)(iii) That the candidate be admitted to the degree of Doctor of Philosophy subject to minor revisions being made to the satisfaction of the internal examiner, normally within a period of six months of receiving formal notification of the revisions to be made.
The Candidate be permitted to resubmit for the degree
(b)(i) That the candidate’s thesis be deemed to be of a satisfactory standard, but that the candidate be adjudged to have failed to satisfy the examiners in the oral examination and that the candidate therefore be required to attend within six months either for a second oral examination or for a written examination, as the examiners shall determine in their written report.
(b)(ii) That the candidate be adjudged to have failed to satisfy the examiners and the candidate be permitted to revise and re-submit the thesis within twelve months without a further oral examination.
(b)(iii) That the candidate be adjudged to have failed to satisfy the examiners and the candidate be permitted to revise and re-submit the thesis within twelve months and be re-examined orally.
The Candidate be recommended for the Master’s degree
(c)(i) That the candidate has reached the standard required for the appropriate Master’s Degree and should immediately be awarded that degree instead.
(c)(ii) That the candidate has reached the standard required for the appropriate Master’s Degree and should be awarded that degree instead subject to minor corrections of the text made to the satisfaction of the internal examiner, normally within a period of one month of receiving formal notification of the corrections to be made.
(c)(iii) That the candidate has reached the standard required for the appropriate Master’s Degree and should be awarded that degree instead subject to minor revisions being made to the satisfaction of the internal examiner, normally within a period of six months of receiving formal notification of the revisions to be made.
The Candidate be permitted to resubmit for the Master’s degree
(d) That the candidate be permitted to revise and re-submit the thesis for the appropriate Master’s Degree within twelve months and be re-examined orally if the examiners so require by indication in their written report.
The Candidate be adjudged to have failed to satisfy the examiners
(e) That no degree be awarded and that the candidate be adjudged to have failed.
Examination Results 2011
Ai
Aii
Aiii
B
C
Out of Time
On Time
Assessment Irregularities
Doctoral Prize Nominations
Total submissions
0 20 40 60 80 100 120 140
Can you do more to recognise excellence?
• In many countries, 1st rate PhD students can be defined. – the French system allows PhDs to be awarded as
"honourable (not very good)", "very honourable" (average) and "very honourable with felicitations" (top 5%).
– there is no similar recognition in the UK.• In Newcastle we ask examiners (on a separate
report sheet) to indicate whether they consider the thesis to be in the top 10% of theses they have examined.– A committee considers this recommendation and prizes
are awarded.• We hope this will be good for the student’s CV.
At the end (if it has NOT gone well) - 1
• Make absolutely sure you know what your options are (read the regulations)
• You are not obliged to tell the candidate anything (although you will feel some pressure to do so)– the candidate will receive written confirmation of the
outcome in due course
• You may wish to speak to the supervisor• You might need clarification of the regulations
(Graduate School)
At the end (if it has NOT gone well) - 2
• You will need to fill in the report form with very comprehensive details of any changes you require– This outcome results in much more effort in the
future for the examiners!
• All report forms look different.– Make sure you know the precise significance of a
tick in every optional box!– For example, ticking box 3 (pass) at Newcastle
can produce a very different outcome from box 3 at Imperial College (fail)!
The appeal process
www.ncl.ac.uk/spo/AppealsForm.pdf
Invitation to examine
• This is not necessarily an honour!– Not even a valued career move?
• Why you and not someone else?– Are you a ‘soft touch’?– Are you the supervisor’s best friend?
• You will/should see the abstract of the thesis at the time of invitation
• Think about the thesis– You have the right to REFUSE!
Case study • X agreed to act as external examiner for Y’s PhD thesis.
• The thesis was of marginal quality but X’s preliminary report suggested that with modification and a satisfactory oral examination it was likely that a positive recommendation could be made.
• Close scrutiny of the thesis immediately before the examination revealed references to an earlier MD thesis by the same candidate (at the same university). X contacted the internal to request that a copy of this thesis should also be available for scrutiny (it is reasonable to see all reference sources cited in a thesis).
• Before meeting the candidate on the day of the oral, X and the internal discovered that the earlier thesis reported about 70% of the data presented in the PhD thesis (about 45 pages of the introduction to both works was identical).
• X and the internal examiner commenced the examination.
Discuss the potential outcomes.
top related