ATTENDANCE AND PERFORMANCE IN MEDICAL MICROBIOLOGY & IMMUNOLOGY

Medical Microbiology and Immunology : All lectures were audio recorded

This is a three quarter course with around 120 contact hours. It has varied a little in terms of contact hours over the time period we are talking about today. Until 2010-2011, the format mostly involved covering each microbe individually, but in 2010-2011, we switched to a system based format. There has been between 184-192 students each year. All are M2 medical students. The students were graded with a T-score grading system.

Each student could get a T-score point as a grading bonus for taking the national shelf exam in Microbiology and Immunology. Over 95% of the students took the shelf exam. We get the individual scores on the shelf exam and have the individual scores from our tests.

Each student could get a second T-score point as a grading bonus if they came to class 65% of the time and used the electronic response devices(clickers). Between 40-60% of the students received this bonus on the time period covered.

Medical Microbiology and Immunology Exam Averages

2010-2011 2009-2010 2008-2009# of questions 332 374 371# of questions

correct282 317 313

Average 84.95% 84.74% 84.48%

  10-11 09-10 08-09 07-08 06-07 05-06

Nat'l Mean 490.0 490.0 490.0 490.0 500.0 490.0

RFUMS Mean 578.3 611 583.5 539.5 524.7 523.4

SD 129.3 123.6 136.6 136.9 108 122.7

High Score 990 990 980 920 840 780

Low Score 210 300 140 170 190 10

# of Students 184 174 184 183 177 135

# of Students Passing

181 (98.37%

)

173 (99.43%

)

177 (96.20%

)

172 (93.99%

)

172 (97.12%

)

133 (98.52%

)

Passing Score 340 340 340 340 300 300

SHELF EXAM SCORES

For 05-06 and 06-07, passing was set at 2 SDs below the national mean, but in 07-08, this was changed to 1.5 SDs below the national mean.

Comparison of 2010- 2011 Shelf Exam Scores

and Final T-Scores

T-Scores

Shelf Scores

Passing: 340 Nat’l Mean: 490 Class Mean: 578 Students: 184

Clicker Use vs Average % and Shelf Score

Avg % before bonus Avg Shelf score

Clicker Use 10-11 09-10 08-09 07-08 06-07 10-11 09-10 08-09 07-08 06-07

> 65% 85.3% 85.6% 84.6% N/A 83.6% 562 618.7 586.2 555.5 537.3

< 65% 85.1% 83.2% 84.0% N/A 80.6% 612 598.3 572.7 517.9 478.7

Standard Deviation : 129.3 123.6 136.6 136.9 108.0

So, there was no clear improvement in student performance on our tests or the shelf exams by attending and using the clickers.

Student Attendance at the M2 Clinical Neuro Course

• A large proportion of our M2 Clinical Neuro course consists of interactive lectures in which we show brief videotapes of us and our colleagues interviewing patients, and then asking the class to discuss its observations.

• In the fall of the 2008-09 academic year, our M2s persuaded us to let them audiotape and videotape these interactive lectures, using Camtasia.

6

Major Problems with the Taping• To preserve patient privacy, it took countless hours of

Medical Media time and our time to blur patients’ eyes and delete references to names and other identifying data.

• Around 3 weeks into the course, the day the taping began, class attendance dropped from 120 to 30, and remained at 30 for the duration of the course (humiliating!)

• Despite the drop in attendance being unacceptable to the course directors and faculty, the course was highly rated by the students, and the Behavioral Science and Psychiatry shelf and Steps 1 and 2 scores improved from the prior year.

7

Our Solutions to the Attendance and Taping Problem

• During 2009-10, and 2010-11, we stopped Camtasing the lectures.

• In 2009-10, we developed a token economy (positive reinforcement) system for attendance.

8

Token Economy for Attendance, AY 2009-10

• Token economy for attendance– If 80 students attended 90% of the time in fall term,

and – 60 students attended 90% of the time in winter term,

and– 40 students attended 90% of the time in the spring

(just before USMLE)– The entire class would receive two bonus points.– The students in attendance would, in essence, be

“covering” for their classmates, just as doctors “cover” for fellow doctors in emergency rooms or walk-in clinics.

9

2009-10 Attendance Outcome, with Bonus Points Given for Minimum Number of Students (Class size 185) in Attendance

10

How do you interpret this?

How We Interpreted This• In fall and winter of 2009-10, they responded literally to the way we

rewarded them: – In the fall, we required 80, and 78-85 (out of 185) attended.– In the winter, we required 60, and 57-70 showed up.

• Rumor has it that soon after the class began, they texted their buddies with something like, “We need another 10,” or “They want another 10,” and approximately 10 walked into class, late.

• In the spring, they were so preoccupied with studying for finals and USMLE that the token economy didn’t work. – In the spring, we required 40, and 3-20 were present.

• Although this still wasn’t satisfactory, the faculty felt better about attendance compared to 2008-09.

• Once again, the course was highly rated by the students, and the Behavioral Science and Psychiatry shelf and Steps 1 and 2 scores improved from 2008-09.

11

Token Economy for 2010-11• Based on the 2009-10 results, we modified the

2010-11 token economy considerably, limiting it to the spring, as follows:– In the fall and winter, there was no attendance

rule. Whoever attended did so with no external reward or penalty.

– Hoping to improve spring attendance despite students studying intensively for finals and USMLE, our spring rule was if 40 or more students attended 90% of the time, the entire class would receive 3 bonus points.

12

2010-11: Bonus Points Given Only in the Spring if 40 or More Students Attend

13

How do you interpret this?

During academic year 2010-11, when asked to state—narratively—why why they chose to attend class rather

than rely on other sources...

• Their modal response was that they – Felt they learned more by attending class than by

relying on PowerPoints and other sources.– Enjoyed multiple aspects of the interactive

lectures, including the videotapes and the discussions that followed.

• Only a tiny number wrote that they attended because of bonus points.

14

Overall Course Outcomes

• Once again, the course was highly rated by the students, and the Behavioral Science and Psychiatry shelf and Steps 1 and 2 scores improved from 2009-10.

15

Our Conclusions

• In a course for which there is no audiotaping-videotaping (Camtasing) of lectures, if the lecture quality is good,– There is no need to give bonus points for

attendance.– A token economy based on a minimum

attendance requirement contributes nothing of value.

16

THE END