Evaluations for Senior Classes

John S. Walton

Animal & Poultry Science, OAC

University of Guelph

(in support of supervised examinations)

Outline

1. Why am I here?

2. What I do

3. Why I think supervised examinations are important

4. How I think these can be improved

5. What I am really trying to ‘sell’ today?

Why am I here?

1. Seemed like a good time for a ‘rant’

2. Pure reaction to reports from the “Academic Integrity” sessions last year

I am somewhat concerned for the future of final examinations ….

1. Insufficient time allowed for grading (nothing new)

2. Students with conflicting final exam times (relatively new, on the increase)

Potential Solutions to Exam Conflicts:

1. Abandon exams

2. Ask the other professor to hold an alternate

3. Create an alternate

The Problem of Alternate Exams:

1. Time consuming

2. Ensuring equivalent course – coverage and rigor (“Fairness”)

3. Insufficient novel, meaningful and substantive questions for multiple sittings

For me # 3 is the major problem

Potential Outcomes:

1. Fewer final examinations

2. “Final” examinations that represent a minor (trivial) part of the overall grade

3. More unsupervised methods of evaluation with the attendant risk of… “FACILITATED ACADEMIC MISCONDUCT”

What I do:I teach –

ANSC*3120 Animal Reproduction (3-3) W

ANSC*4130 Reproductive Management and Technology (3-3) W

ANSC*3120 96 students (‘04)

is the prerequisite for

ANSC*4130 38 students (‘04)

Evaluations:ANSC*3120

Midterm examination 40% (2 h) 20 questions (wk 1-6)

Final examination 40% (2 h) 20 questions (wk 7-12)

Laboratory books 20%

ANSC*4130 (the course I’ll focus on primarily)

*Midterm examination 30% (2 h) 3 questions (wk 1-6)

*Final examination 30% (2 h) 3 questions (wk 7-12)

Laboratories/Hands-on 40%

*(6 questions, 3 are selected)

My students: ANSC*3120 W’04

0

5

10

15

20

25

# St

uden

ts

Grade

D+ C- C C+ B- B B+ A- A A+

BSc. (Agr) BSc.

My written exams tended to (and still do) focus on “Do they know anything?”

For example: ANSC*3120 Results

High proportion of ‘A’ grades despite the fact no student really demonstrated an ‘A’ performance (examination is heavy on retention of factual material)

The Correct Distribution?

“An outstanding performance in which the student demonstrates a superior grasp of the subject matter AND AN ABILITY TO GO BEYOND THE GIVEN MATERIAL IN A CRITICAL AND CONSTRUCTIVE MANNER ….”

UoG (2003)

What is an ‘A’ Grade?80-100 ‘A’ Excellent

ANSC*3120

20 questionsPrerequisite

WHAT DO THEY WHAT DO THEY KNOW?KNOW?

CAN THEY THINK?

easy

ANSC*4130

3 questionsElective

WHAT DO THEY KNOW?

CAN THEY THINK?CAN THEY THINK?

more difficult – but necessary

They provide an accurate method of assessing each student

The risk of academic misconduct is relatively low (well-defined boundaries)

Unsupervised examinations/assignments are relatively susceptible to plagiarism (“facilitated” academic misconduct)

Why I think supervised examinations are important

How I think supervised examinations can be improved:

My problem has been:

1. Too many sittings required

2. Not enough novel questions

3. Asking the wrong sorts of questions

My current solution is:

Open questions/closed exam

Mid-term

Final

Identical format

Both are a 2 h written exam

(during wk 7 and during Final Exams)

For ANSC*4130, therefore

Open Questions

Questions published: week 3 (Midterm)

week 9 (Final)

6 questions/exam (1 question per week)

In pairs (answer either 1 or 2, etc)

At the exam:

1. Questions to be answered (3 of 6) chosen by flipping a coin

2. Students write their answers without access to their notes or sample answers

3. Student identify their answer booklets using their STUDENT NUMBER ONLY

Closed Exam

Sample Question:

After graduation, you are contracted by Health Canada to assist with submissions to the Veterinary Drugs Directorate for approval of the CIDR-B* device for use in lactating dairy cattle. The request for the label-claim is for the regulation of ovulation in dairy cattle, including problem breeders. Review the currently available published evidence (North America) and develop an appropriate recommendation [i.e. prohibit, permit use, permit use with restrictions (milk withdrawal/slaughter)]. Be sure to frame this according to the approval criteria: 1) efficacy; 2) human safety; and 3) potential impact on the cow (short and long-term).

*CIDR-B is an intra-vaginal insert that contains the hormone progesterone, ovulation is blocked while the insert is in place.

1. Prepare sample answers

2. Work together if they wish

3. Seek help from me (facilitated by an in-class overview in week 5 and week 11 and a post-mortem of the mid-term in week 9)

4. Incorporate material beyond lectures and laboratories (identifying key authors or their locations)

Students are encouraged to:

Open Questions:

1. Less paranoid students:

a) Predictable situation (no real surprises)

b) They have time to seek advice

c) Almost zero risk of academic misconduct

2. Allows for more complex and wide ranging questions that are problem-based (and give a better evaluation of understanding)

3. Improved precision and accuracy (vs. assignments)

Advantages:

4. Exam can be written at almost any time – avoids composing alternate exams

5. I can assign an ‘A’ with more of a clear conscience

Advantages: (cont’d)

1. Helps with, but does not totally solve the need for novel questions

2. Somewhat unfair to introverted students

3. Not all students respond to the opportunity (depressing)

4. Time commitment (instructor)

Disadvantages:

Given the 2 courses I teach – I’d suggest a maximum of ~ 100 students (perhaps more, if it is the only course to be graded)

Applicable in any course that lends itself to essay-type questions

Scalability and transferability:

1. If we don’t want academic misconduct, we should prevent it through supervised evaluations

2. Consider publishing your examination questions in advance (and then expecting ‘better’ answers). Open questions/closed examinations appear to be working for me

The Sales ‘Pitch’