aamc annual meeting san antonio, texas november 1, 2008
DESCRIPTION
GSA Transition Forum: Is the MSPE Dead?. AAMC Annual Meeting San Antonio, Texas November 1, 2008. Chair. Molly Osborne, M.D., Ph.D. Associate Dean for Student Affairs Oregon Health & Science University Chair-Elect, AAMC Group on Student Affairs. Speakers. Richard P. Wheeler, M.D. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/1.jpg)
Annual Meeting 2008
Creating a Better Tomorrow
GSA Transition Forum:
Is the MSPE Dead?
AAMC Annual MeetingSan Antonio, TexasNovember 1, 2008
![Page 2: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/2.jpg)
Creating a Better Tomorrow2008 Annual Meeting
Chair
Molly Osborne, M.D., Ph.D.
Associate Dean for Student Affairs
Oregon Health & Science University
Chair-Elect, AAMC Group on Student Affairs
![Page 3: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/3.jpg)
Creating a Better Tomorrow2008 Annual Meeting
Speakers
Richard P. Wheeler, M.D.
Executive Associate Dean for Academic Affairs, College of Medicine
University of Arkansas for Medical Sciences
Lanny Garth Close, M.D.
Professor and Chair, Department of Otolaryngology
Columbia University College of Physicians & Surgeons
![Page 4: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/4.jpg)
Creating a Better Tomorrow2008 Annual Meeting
Speakers
Neil H. Parker, M.D.
Senior Associate Dean, Student Affairs & GME
University of California Los Angeles Geffen SOM
Angela Nuzzarello, M.D., MHPE
Associate Dean, Student Programs & Professional Development
Northwestern University Feinberg School of Medicine
![Page 5: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/5.jpg)
Creating a Better Tomorrow2008 Annual Meeting
Speakers
Maria C. Savoia, M.D.
Vice Dean for Medical Education
Professor of Medicine
University of California San Diego School of Medicine
![Page 6: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/6.jpg)
Creating a Better Tomorrow2008 Annual Meeting
Question # 1
Which of the choices below is your primary affiliation?
1. Student Affairs (GSA)
2. Undergraduate Medical Education (GEA)
3. Graduate Medical Education (Program Directors)
4. Student (OSR)
5. Resident (ORR)
6. Other
![Page 7: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/7.jpg)
Creating a Better Tomorrow2008 Annual Meeting
Question # 2
If you are involved with GSA what part are you primarily involved with?
1. MAS
2. COSR
3. COSA
4. COSFA
5. OSR
6. All
![Page 8: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/8.jpg)
Creating a Better Tomorrow2008 Annual Meeting
Question # 3
How long have you worked in this position?
1. < 1 year
2. 1-5 years
3. 5-10 years
4. >10 years
![Page 9: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/9.jpg)
Creating a Better Tomorrow2008 Annual Meeting
Question # 4
Are you the sender, recipient or subject of the MSPE?
1. Sender
2. Recipient
3. Subject
4. None of the Above
![Page 10: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/10.jpg)
Creating a Better Tomorrow2008 Annual Meeting
Richard P. Wheeler, M.D.
Executive Associate Dean for Academic Affairs, College of Medicine
University of Arkansas for Medical Sciences
![Page 11: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/11.jpg)
Creating a Better Tomorrow2008 Annual Meeting
History of the Dean’s Letter
AKA – M S P E
AKA – MISSPEEEEEEEE
AKA – “It’s not time yet to do those darn things again this year, is it?”
![Page 12: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/12.jpg)
Creating a Better Tomorrow2008 Annual Meeting
National Library of Medicine
4th or 5th Century AD
![Page 13: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/13.jpg)
Creating a Better Tomorrow2008 Annual Meeting
National Library of Medicine
4th or 5th Century AD
![Page 14: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/14.jpg)
Creating a Better Tomorrow2008 Annual Meeting
National Library of Medicine
4th or 5th Century AD
![Page 15: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/15.jpg)
Creating a Better Tomorrow2008 Annual Meeting
This was too damn hard. Have the Associate Dean do it from now on!!
![Page 16: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/16.jpg)
Creating a Better Tomorrow2008 Annual Meeting
National Library of Medicine
4th or 5th Century AD
![Page 17: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/17.jpg)
Creating a Better Tomorrow2008 Annual Meeting
National Library of Medicine
4th or 5th Century AD
![Page 18: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/18.jpg)
Creating a Better Tomorrow2008 Annual Meeting
National Library of Medicine
4th or 5th Century AD
![Page 19: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/19.jpg)
Creating a Better Tomorrow2008 Annual Meeting
Henry M. Sondheimer, M.D.November 1, 300
![Page 20: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/20.jpg)
Creating a Better Tomorrow2008 Annual Meeting
No rules or standardization.
Personal letters written to individual program directors.
Many times a real letter of recommendation rather than
reference.
![Page 21: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/21.jpg)
Creating a Better Tomorrow2008 Annual Meeting
![Page 22: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/22.jpg)
Creating a Better Tomorrow2008 Annual Meeting
UAMS Dean’s Letters = 1 to 2 pages
![Page 23: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/23.jpg)
Creating a Better Tomorrow2008 Annual Meeting
1989
![Page 24: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/24.jpg)
Creating a Better Tomorrow2008 Annual Meeting
UAMS Dean’s Letters = 2 to 3 pages
![Page 25: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/25.jpg)
Creating a Better Tomorrow2008 Annual Meeting
November 1 release date approved through a complex
process.
Intended to slow down the process - and it worked. Few interviews before November.
![Page 26: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/26.jpg)
Creating a Better Tomorrow2008 Annual Meeting
ERASSeptember 1 release established by ERAS Committee but not
through complex approval process like the November 1 release date
1995
![Page 27: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/27.jpg)
Creating a Better Tomorrow2008 Annual Meeting
![Page 28: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/28.jpg)
Creating a Better Tomorrow2008 Annual Meeting
Camel
![Page 29: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/29.jpg)
Creating a Better Tomorrow2008 Annual Meeting
2002
![Page 30: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/30.jpg)
Creating a Better Tomorrow2008 Annual Meeting
UAMS Dean’s Letters = 8 to 9 pages
![Page 31: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/31.jpg)
Creating a Better Tomorrow2008 Annual Meeting
TODAY
![Page 32: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/32.jpg)
Creating a Better Tomorrow2008 Annual Meeting
OR
![Page 33: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/33.jpg)
Creating a Better Tomorrow2008 Annual Meeting
Lanny Garth Close, M.D.
Professor and Chair, Department of Otolaryngology
Columbia University College of Physicians & Surgeons
![Page 34: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/34.jpg)
Creating a Better Tomorrow2008 Annual Meeting
THE PROGRAM DIRECTOR’S POINT OF
VIEW
The Medical Student Performance Evaluation
![Page 35: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/35.jpg)
Creating a Better Tomorrow2008 Annual Meeting
MSPE
2004 Survey of Oto/HNS PD’s
Rank Order of ERAS Candidate Information
• USMLE step 1 score
• Performance as sub-I
• LOR from fellow Chairs/PD’s
• AOA status
• Candidate’s Personal statement
• Dean’s Letter (MSPE)
![Page 36: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/36.jpg)
Creating a Better Tomorrow2008 Annual Meeting
MSPE
The PD’s dream MSPE………
“ Josephine Smith’s class rank at the completion of the third year of medical school is **/***.”
![Page 37: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/37.jpg)
Creating a Better Tomorrow2008 Annual Meeting
MSPE
The PD’s nightmare MSPE………..
“Although the Dean’s Letter Advisory Committee
of the AAMC recommends that the MSPE
contain comparative performance statements,
………..’s evaluation system was not designed to
provide information comparing one student to
another. For that reason, we encourage review
of this evaluation in it’s entirety.”
![Page 38: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/38.jpg)
Creating a Better Tomorrow2008 Annual Meeting
MSPE
Why is the non-revealing “stealth” MSPE a problem for the PD in a highly competitive specialty for which large numbers of highly qualified candidates are applying?
450 applicants
4 positions
40 interview slots
One PD
![Page 39: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/39.jpg)
Creating a Better Tomorrow2008 Annual Meeting
MSPE
What does the PD want from the MSPE
Earlier release
Clearly stated, uniform performance scoring
Clearly stated and illustrated performance comparisons for
• All subjects (rotations) for all 3 years
• An overall ranking of performance at the end of the third year (at least by quartiles)
• As few adjectives and adverbs as possible
![Page 40: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/40.jpg)
Creating a Better Tomorrow2008 Annual Meeting
MSPE
Where does the PD turn when the MSPE does not provide a meaningful comparison of performance for the candidate?
Other schools
USMLE scores
AOA status (when available)
LOR from colleagues
College transcript
![Page 41: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/41.jpg)
Creating a Better Tomorrow2008 Annual Meeting
MSPEExample of scored/illustrated
performance by course
Nu
mb
er
of
Stu
den
ts
![Page 42: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/42.jpg)
Creating a Better Tomorrow2008 Annual Meeting
MSPE
Example of scored/illustrated overall
performance at completion of third year
%
Stu
den
ts
![Page 43: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/43.jpg)
Creating a Better Tomorrow2008 Annual Meeting
MSPE
A Review of 2007 MSPEs from
62 U.S. Medical Schools
![Page 44: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/44.jpg)
Creating a Better Tomorrow2008 Annual Meeting
MSPE
SCORING CRITERIA
3 CATAGORIES
MS performance scored
MS performance compared
MS performance comparison illustrated
![Page 45: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/45.jpg)
Creating a Better Tomorrow2008 Annual Meeting
MSPE
MS Performance Scored
0- not done
1- poorly/incompletely done
2- average (acceptable)
3- well done
4- uniformly done (excellent)
![Page 46: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/46.jpg)
Creating a Better Tomorrow2008 Annual Meeting
MSPE
MS Performance Compared
0- not done
1- poorly/incompletely done
2- average (acceptable)
3- well done
4- uniformly done (excellent)
![Page 47: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/47.jpg)
Creating a Better Tomorrow2008 Annual Meeting
MSPE
MS Performance Comparison Illustrated
0- not done
1- poorly/incompletely done
2- average (acceptable)
3- well done
4- uniformly done (excellent)
![Page 48: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/48.jpg)
Creating a Better Tomorrow2008 Annual Meeting
MSPE
Overall Score for Institution
0- No attempt to follow AAMC guidelines
6- Acceptable presentation of MS performance
12- Outstanding (Gold Standard)
![Page 49: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/49.jpg)
Creating a Better Tomorrow2008 Annual Meeting
MSPE
SCORES
# o
f In
stit
uti
on
s
![Page 50: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/50.jpg)
Creating a Better Tomorrow2008 Annual Meeting
MSPE Study Results
20/62 (32%) U.S. Medical Schools failed to present MS Performance in acceptable format
42/62 (68%) U.S. Medical Schools present MS Performance in acceptable format
6/62 (10%) U.S. Medical Schools present MS Performance in outstanding format
![Page 51: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/51.jpg)
Creating a Better Tomorrow2008 Annual Meeting
Timing of MSPE
Present date is November 1st
• Applications reviewed from mid-September until mid-October
• First batch of interview invitations go out 1st week of October
• Interview slots filled by 2nd to 3rd week of October
• MSPE used by PD to help rank candidates after interviews
![Page 52: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/52.jpg)
Creating a Better Tomorrow2008 Annual Meeting
Timing of MSPE
Proposed date is October 1st
• Applications reviewed mid-September until mid-October
• Interview slots filled by 2nd or 3rd week of October
• MSPE used by PD to select resident applicants for interview
![Page 53: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/53.jpg)
Creating a Better Tomorrow2008 Annual Meeting
Student Affairs Dean Prospective on the MSPE
Neil H. Parker, M.D.
Senior Associate Dean, Student Affairs & GME
University of California Los Angeles Geffen SOM
![Page 54: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/54.jpg)
Creating a Better Tomorrow2008 Annual Meeting
Student Affairs Dean Prospective on the MSPE
Neil H. Parker, M.D.
Senior Associate Dean, Student Affairs & GME
University of California Los Angeles Geffen SOM
![Page 55: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/55.jpg)
Creating a Better Tomorrow2008 Annual Meeting
MSPE – One Dean’s PerspectiveAccurate assessment of student’s performance
Perspective over three plus years
Preclinical coursework minor component and not very useful
Clinical coursework major component and should be very useful
Summary provides perspective on student, potential and overall assessment
Predictive of success as a houseofficer
![Page 56: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/56.jpg)
Creating a Better Tomorrow2008 Annual Meeting
MSPE – Unique CharacteristicsProvides insight into what makes this student different, special, unique….
Documents the Road Travelled
Different from Student’s Activity and Background sections of ERAS as seen through the eyes of the Student Affairs Dean and Educator
Non-cognitive variables important for choosing residents - Programs three to seven years in length
![Page 57: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/57.jpg)
Creating a Better Tomorrow2008 Annual Meeting
MSPE – Education DemographicsImportant for years of school
Leaves of absence important but very variable in how schools represent leaves and reasons
Academic difficulty and disciplinary issues
Predictive of future problems in licensure, etc
![Page 58: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/58.jpg)
Creating a Better Tomorrow2008 Annual Meeting
MSPE – Years One and TwoToo short to be meaningful
All students about the same
Little information from course chairs
Can be left out without / not very different from transcript grades
Trumped by USMLE Step 1 score
![Page 59: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/59.jpg)
Creating a Better Tomorrow2008 Annual Meeting
MSPE – Clinical Years
Not same as clinical transcript
Critical information on student’s performance in required clerkships
Best predictor of abilities, attitudes and future potential and problems
This is the meat of the MSPE
Essentially same as evaluations of residents by faculty in programs
Needs faculty and resident development
![Page 60: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/60.jpg)
Creating a Better Tomorrow2008 Annual Meeting
MSPE – SummaryShort and puts residency applicant into perspective
Not specialty oriented
Not a recommendation
Would be better if was recommendation and uniform as to prediction of success as a resident
![Page 61: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/61.jpg)
Creating a Better Tomorrow2008 Annual Meeting
MSPE – Graphs & AppendixIn Pass Fail Schools, totally useless
What does it predict
Adds multiple pages that make MSPE seem long
Appendix E – Does anyone read in selection or only when there is a problem in residency
![Page 62: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/62.jpg)
Creating a Better Tomorrow2008 Annual Meeting
MSPE – One Dean’s PerspectiveWritten by Student Affairs Dean
169 Students
Follow AAMC MSPE recommendations
Use Automated System for Processing
Senior reviews for accuracy
1 ½ - 2 hours per student
![Page 63: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/63.jpg)
Creating a Better Tomorrow2008 Annual Meeting
MSPE – Production Time / CostAt UCLA, production of MSPE reduced by 50% through automation
Students complete web-based survey on their activities, background and other unique characteristics – 2+ hours
Individual meeting with students 20-30 minutes, SAD
Production of Unique characteristics and Summary – SAD ( Student Affair Dean – and is sad) 1 – 1 1/2 hours
Electronic formulation of clerkships information from evaluation system – HAL
Cost $300-400 per student x 169 students = $ 45,000 +
![Page 64: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/64.jpg)
Creating a Better Tomorrow2008 Annual Meeting
TimingStudents complete questionnaire on Activities, Background and Characteristics - August
Letters written between August and October
Students review letter for accuracy between September and mid October
Uniform release November 1
This allows for all the Third Year electives and first Sub I’s of fourth year to be received and place in the MSPE
![Page 65: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/65.jpg)
Creating a Better Tomorrow2008 Annual Meeting
MSPEGeffen School of Medicine
Pass/Fail all Years
Nu
mb
er
of
Stu
den
ts
0
20
40
60
80
100
120
140
160
180
PASS FAIL
![Page 66: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/66.jpg)
Creating a Better Tomorrow2008 Annual Meeting
MSPE - ProblemsPD’s and faculty increasingly busy with clincial & research productivity and ACGME requirements
Extremely useful but not appreciated by PD’s
Decisions on Interviews often decided before MSPE released
Undervalued
Clerkship evaluations require 6-8 weeks to collect
Senior electives evaluations difficult to collect and late
![Page 67: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/67.jpg)
Creating a Better Tomorrow2008 Annual Meeting
MSPE – Perspective Cont.MSPE really only 2-3 Pages
5 -6 pages of graphs, Appendices, etc
Written comments 2-3 pages
Has the MSPE gone the way of the Physical Exam and the Medical Write-up?
Should it be reduced to Consumer Report style?
Is complete, valuable information overruled by need for bottom lines and byte size information?
![Page 68: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/68.jpg)
Creating a Better Tomorrow2008 Annual Meeting
MSPE – Thoughts from the Weird sideConsider component release – everything short of Senior electives by October 1st
Hold ERAS process till two weeks after release of MSPE – lite
No interviews scheduled till after MSPE lite
Senior Elective evaluations supplement Dec 1st
Transcripts folded into MSPE’s
![Page 69: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/69.jpg)
Creating a Better Tomorrow2008 Annual Meeting
MSPE - From the Far SideDon’t change, they will come around
Go back to letter of recommendation
Clerkship evaluation development
Do away with MSPE
Just send unabridged clerkship evaluations
Develop electronic MSPE that PD’s can sort for the parts they want
![Page 70: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/70.jpg)
Creating a Better Tomorrow2008 Annual Meeting
Are we dealing with MillennialsNot the Students but PD’s – younger and have tenure of 3 years on average
Want everything electronic
Information must be in Byte size
Want it their way and don’t except No as an answer
Multitaskers who don’t have time to read 8 pages
Just give me the bottom line
![Page 71: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/71.jpg)
Creating a Better Tomorrow2008 Annual Meeting
Angela Nuzzarello, M.D., MHPE
Associate Dean, Student Programs & Professional Development
Northwestern University Feinberg School of Medicine*
*As of January 1, 2009 – Oakland University William Beaumont School of Medicine
![Page 72: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/72.jpg)
Creating a Better Tomorrow2008 Annual Meeting
What are we duplicating?
Unique Characteristics
• Leadership, research, community service, already in personal statement, CV, and LoRs
Academic History
• Grades already on transcript
Summary
• If summary statements are included, information has already been presented in LoRs or unique characteristics
![Page 73: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/73.jpg)
Creating a Better Tomorrow2008 Annual Meeting
What information is unique to the MSPE?
Explanation of any extension of time
Recipient of any adverse action
Assessment of professional performance
Clerkship narratives
Comparative performance (ranking, or quartiles, etc)
Grade percentages for each clerkship
![Page 74: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/74.jpg)
Creating a Better Tomorrow2008 Annual Meeting
The Problem with Creating MSPEs
Information from multiple sources required a significant amount of “cutting and pasting”
Students previewed MSPE as PDF and suggested changes through email or in person
Changes to document were required every time a new grade was finalized
Setting up comparative charts and tables was time consuming
Creating each student’s graphical record was also time consuming.
![Page 75: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/75.jpg)
Creating a Better Tomorrow2008 Annual Meeting
Solutions
Web-based system allows students easy access to MSPE document
Can make suggestions for changes through web interface which decreases number of emails
Any information that is stored in a database can be put in to the MSPE template
New information (i.e. grades) continually updated
![Page 76: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/76.jpg)
Creating a Better Tomorrow2008 Annual Meeting
Unique Features of Web-based MSPE
Email notification to students when the draft is ready; when student comments have been addressed
Forum-style system for collaboration on various sections of MSPE letter
Real-time clerkship narrative and grade inclusion
Dynamic chart and table generation
MS Word document generation
Allows tracking of progress of individual MSPEs
![Page 77: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/77.jpg)
Creating a Better Tomorrow2008 Annual Meeting
![Page 78: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/78.jpg)
![Page 79: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/79.jpg)
Medical Student Performance Evaluation for
John Smith November 1, 2008
Identifying Information [Comment] # of Comments: 0
John is a fourth-year student at Northwestern University Feinberg School of Medicine in Chicago, Illinois.
Unique Characteristics EDIT [Comment] # of Comments: 0
Over the course of his medical training, John has participated in a variety of activities and gained experiences that will add to his strengths as a physician. Although all of his pursuits have been listed elsewhere on his ERAS application, several pursuits have been particularly meaningful to his development. John has done……..
Academic History EDIT [Comment] # of Comments: 0
Date of Expected Graduation from Medical School: May 2009
Date of Initial Matriculation in Medical School: July 2005
Please explain any extensions, leave(s) of absence, gap(s), or break(s) in the student’s educational program.
Not applicable
For transfer students:
Date of Initial Matriculation in Prior Medical School: Not applicable
Date of Transfer from Prior Medical School: Not applicable
For dual/joint/combined degree students:
Date of Initial Matriculation in other degree program: Not applicable
Date of Expected graduation from other degree program: Not applicable
Type of other degree program: Not Applicable
Was this student required to repeat or otherwise remediate any coursework during his/her medical education?
No
Was this student the recipient of any adverse action(s) by the medical school or its parent institution?
No
![Page 80: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/80.jpg)
Professional Performance [Comment] # of Comments: 0
We have assessed all students’ capabilities to meet standards for: accountability, self-improvement and adaptability, appropriate relationship with patients, relationship with healthcare team, initiative, professional demeanor, and behavior under stress. Unless otherwise noted, all students have met the stated objectives for professionalism at Feinberg.
Academic Progress EDIT [Comment] # of Comments: 0
John earned grades of Pass in all preclerkship courses and recorded a score of 239 on Step I of the USMLE exam.
Preclinical/Basic Science Curriculum [Comment] # of Comments: 0
(Details are provided in Appendix E.)
Core Clinical Clerkships EDIT [Comment] # of Comments: 0
Following are the unedited narrative evaluations of John’s performance on the successive clerkships and electives. Appendix B summarizes this information graphically. NEUROLOGY: PASS 7/2/2007 - 7/27/2007
John did a very good job during his neurology rotation. This was his first clinical clerkship. He demonstrated good history and examination skills. His histories were thorough and he was able to pay attention to detail. He also demonstrated appropriate techniques in the neurological examination. His oral presentations were well organized and clear. He appeared to have a good fund of knowledge based on his clinical interactions and discussion; he scored below the mean on his written shelf examination. He was motivated and eager to learn. He was hard working, organized and responsible in his work. He had great interpersonal skills and interacted very well with his team members and patients. For his overall performance on this rotation, he is awarded the grade of PASS.
PRIMARY CARE: HONORS 7/30/2007 - 8/24/2007 John is a bright medical student who did an outstanding job in all aspects of the Primary Care Clerkship. He has excellent interactions with patients and staff alike. His history and physical exam taking skills are on par with a 4th year medical student. His oral case presentations are concise and well organized. He has a solid knowledge base and very good clinical judgment. In Weekly Report he was an active participant with frequent insightful comments. His Learner Centered Learning Goal presentation on depression screening was outstanding. The preceptor wrote, "Not
![Page 81: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/81.jpg)
APPENDICES
Appendix A: Comparative Performance in Preclinical/Basic Science Coursework [Comment] # of Comments: 0
During the first and second years of the curriculum, the only grades assigned to students are Pass or Fail. Students must receive a grade of “Pass”, in each preclinical course before beginning the clerkships.
Appendix B: Graphic Representations of Comparative Performance in Core Clinical Clerkships [Comment] # of Comments: 0
Grades in Required M3 Clerkships/Courses
John’s grade for each clerkship is indicated inside the bracket beside the clerkship name.
Table 1- Grade Percentages for Each Required M3 Clerkships/Courses
IDM3 MED NEURO OBGYN PEDIA PRIMCARE PSYCH SURG
H 25 36 34 22 23 31 24 31
HP 16 27 30 34 29 26 40 18
P 59 37 36 43 46 43 36 48
F 0 0 0 1 2 0 1 2
![Page 82: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/82.jpg)
Creating a Better Tomorrow2008 Annual Meeting
Comments from students
“I think it worked well. The ability to send e-comments was very helpful and response time was quick.”
“I think it was great that we could view our MSPEs and any comments I had were immediately addressed.”
“I feel that the bi-directional feedback and electronic format made this quite efficient.”
“I liked that everything was web based.”
![Page 83: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/83.jpg)
Creating a Better Tomorrow2008 Annual Meeting
Maria C. Savoia, M.D.
Vice Dean for Medical Education
Professor of Medicine
University of California San Diego School of Medicine
![Page 84: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/84.jpg)
Creating a Better Tomorrow2008 Annual Meeting
““BOTH SIDES NOWBOTH SIDES NOW””
A VIEW ABOVE THE FRAYA VIEW ABOVE THE FRAY&&
A REVIEW OF THE RELEVANT A REVIEW OF THE RELEVANT LITERATURELITERATURE
![Page 85: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/85.jpg)
Creating a Better Tomorrow2008 Annual Meeting
Do MSPEs Accurately Reflect Do MSPEs Accurately Reflect Medical School Performance?Medical School Performance?
Are they based on accurate data?
Are data accurately reflected in letters?
![Page 86: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/86.jpg)
Creating a Better Tomorrow2008 Annual Meeting
Based On Accurate Data?Based On Accurate Data?
Evaluation often done by the least experienced
› Experience important in preceptor’s ability to assess cognitive levels Taylor C., Lipsky MS Fam Med 1990 Jul-Aug;
(2294)296-8
![Page 87: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/87.jpg)
Creating a Better Tomorrow2008 Annual Meeting
Based On Accurate Data?Based On Accurate Data?
For data to be reliable, either many observations or many observers
For data to be valid, they have to measure what they say they measure
› “Knowledge and skill” in a Psychiatry clerkship correlated with personality characteristics and not with NBME Subject exam or OSCE scores Chibnall JT, BlaskiewiczRJ. Acad Psychiatry 2008
May-Jun;32(3); 199-205
![Page 88: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/88.jpg)
Creating a Better Tomorrow2008 Annual Meeting
Based On Accurate Data?Based On Accurate Data?
“Objective” data often maligned
Although many recent studies have not demonstrated gender or ethnic biases… it is better to be good looking and thin & to work closely with the person who evaluates you
› Supervisors grade higher than control graders McKinstry BH, et. al. BMC Medical Education 2004,
4;28
![Page 89: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/89.jpg)
Creating a Better Tomorrow2008 Annual Meeting
Based On Accurate Data?Based On Accurate Data?The great group of “Very Good”
› In Pass/Fail curriculum, how do you distinguish among those who excel &those who scrape by?
› P/F grading puts students at a disadvantage in applying to General Surgery programs Dietrick JA, et. al., Am J Surg 1991 Jul;162(1);63-6
› In a survey of 110 program directors across specialties, only 8 favored a P/F system Provan JL, Cuttress L, Can Med Assoc J Oct.1, 1995;
153 (7)
![Page 90: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/90.jpg)
Creating a Better Tomorrow2008 Annual Meeting
1992 55%
2001 65%
2005 75%
“Adequate MSPEs”
Are Data Accurately Reflected?Are Data Accurately Reflected?
Hunt DD, et. al., Acad Med 2001 Jul;76(7): 727-33
Shea JA, et. al., Acad Med 2008 Mar; 83(3):284-91
![Page 91: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/91.jpg)
Creating a Better Tomorrow2008 Annual Meeting
Are Data Accurately Reflected?Are Data Accurately Reflected?
We live in a “fantasy land” where all students “will make a fine physician,” & are “above average” “with excellent interpersonal skills.”
Dean’s Letter for Adolph Hitler: “A natural leader…good communication skills…assisted in the development of technological advances…likes to find solutions to problems”
Friedman RB, NEJM 1983 308;651-3
![Page 92: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/92.jpg)
Creating a Better Tomorrow2008 Annual Meeting
Are Data Accurately Reflected?Are Data Accurately Reflected?
Negative information found on transcript not mentioned in Dean’s Letter 34% of time.
› Conclusion: “Some deans suppress negative information in their letters and potentially obfuscate the residency selection process.” Edmond M, et. al. 1999 Acad Med 74(9);1033-5.
![Page 93: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/93.jpg)
Creating a Better Tomorrow2008 Annual Meeting
Are Data Accurately Reflected?Are Data Accurately Reflected?
Excellent: of superior merit, exceptionally good
56/75 schools used “Excellent” as a category.
For >50% of schools, a student described as excellent might be in bottom half of class. For 2 schools, all in “Excellent” category were in the bottom half of the class
Naidich JB, et. al., Acad Rad 2007 14; 1121-6.
![Page 94: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/94.jpg)
Creating a Better Tomorrow2008 Annual Meeting
Are Data Accurately Reflected?Are Data Accurately Reflected?
Professionalism comments evaluated in 293 MSPEs for 2005 graduates.
70% mentioned professionalism, with fewer than 1% of comments that could be considered negative.
› Conclusion: “Most professionalism comments in MSPE’s are generic and somewhat bland…” Shea JA, et. al., Acad Med 2008 Oct. 83(10
Suppl);S1-4
![Page 95: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/95.jpg)
Creating a Better Tomorrow2008 Annual Meeting
What Factors Accurately Predict What Factors Accurately Predict Residency Performance?Residency Performance?
Univ. Rochester looked at 2 graduating classes, divided into:• “Outstanding” (upper quartile),• “Excellent” (second quartile),• “Very Good” (lower 2 quartiles)• “Good” (lowest few percentiles)
Found “Dean’s Letter rankings were a significant predictor of later performance.”
Lurie SJ, et. al., Teach Learn Med 2007 19(3);251-6
![Page 96: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/96.jpg)
Creating a Better Tomorrow2008 Annual Meeting
What Factors Accurately Predict What Factors Accurately Predict Residency Performance?Residency Performance?
At the Uniform Services University, appearing before Standings & Promotions Committee (for any reason) was predictor of below average performance during internship.
Durning SJ, et. al., Teach Learn Med 2008 20(3);267-72
![Page 97: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/97.jpg)
Creating a Better Tomorrow2008 Annual Meeting
What Factors Accurately Predict What Factors Accurately Predict Residency Performance?Residency Performance?
At the Medical University of South Carolina, GPA (preclinical + clinical) + interpersonal score on Step 2CS best predicted internship performance but only accounted for 30% of variance.
Taylor ML, et. al., Acad Med 2005 80(5); 496-501
![Page 98: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/98.jpg)
Creating a Better Tomorrow2008 Annual Meeting
What Factors Accurately Predict What Factors Accurately Predict Residency Performance?Residency Performance?
Otolaryngology: “excellent academic performance in medical school”
Calhoun KH, et. al., Otolaryngol Head Neck Surg 1997 116(6 pt1); 647-51
Emergency Medicine: medical school attended, MSPE and “distinctive factors”
Hayden SR, et. al., Acad Emerg Med 2005 12(3); 206-10
![Page 99: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/99.jpg)
Creating a Better Tomorrow2008 Annual Meeting
What Factors Accurately Predict What Factors Accurately Predict Residency Performance?Residency Performance?
Radiology: “Noncognitive factors were as important as cognitive factors,” “Objective measures, such as NBME scores, failed to adequately predict residents’ performance.”
Wood PS, et. al., Invest Radiol 1990 25(7);855-9
Radiology: Success on ABR exam predicted by grades and NBME scores. No good predictors of clinical performance.
Boyse TD, et. al., Acad Radiol. 2002 9(4);437-45
![Page 100: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/100.jpg)
Creating a Better Tomorrow2008 Annual Meeting
What Factors Accurately Predict What Factors Accurately Predict Residency Performance?Residency Performance?
PM&R: 205 residents at the University of Washington. Clinical residency performance predicted by clerkship honors grades (p=.0001). Probation was predicted by failing a basic science course (p=.0001)
Amos DE, Massagli TL, Acad Med 1996 71(6);678
![Page 101: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/101.jpg)
Creating a Better Tomorrow2008 Annual Meeting
What Factors Accurately Predict What Factors Accurately Predict Residency Performance?Residency Performance?
Orthopedics: “Academic score,” using only objective measures, made screening process more objective but didn’t correlate with outcomes of the training program.
Dirschl DR, et. al., Clin Orthop Relat Res 2006 449;44-9
Surgery: “Dean’s letter seems to give the most accurate picture of applicant’s suitability for residency training.”
Naylor RA, et. al., Arch Surg 2008 143(7); 647-52
![Page 102: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/102.jpg)
Creating a Better Tomorrow2008 Annual Meeting
PD’s: Ignore at your own risk!With an edge of annoyance, Hunt asked (Barrows) about Swango. “What kind of guy did you send us?”
Barrows said that Hunt should have seen plenty of warning flags in Swango’s dean’s letter. “Well,” Hunt retorted, “I don’t read dean’s letters.”
….Soon after, Hunt called back: he’d found the letter
“Oh, my God,” Hunt said. “You’re right. You did tell me.”
Stewart, JB, Blind Eye , 1999, pps. 59-60
![Page 103: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/103.jpg)
Creating a Better Tomorrow2008 Annual Meeting
What Factors Accurately Predict What Factors Accurately Predict Residency Performance?Residency Performance?
Surgery: Only Step 2 scores (and not Step 1, overall GPA, AOA status, or 3rd year clerkship grade) predicted internship performance for 87 graduates from Wash U.
Andriole DA, et. al., Am J Surg 2004 188;161-4
Anesthesia: Best predictor of academic performance number of years spent in another specialty. Step 1 score inversely correlated with in service exam score
Warrick SS, Crumrine RS, J Med Ed 1986 61(7); 591-5
![Page 104: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/104.jpg)
Creating a Better Tomorrow2008 Annual Meeting
What Factors Accurately Predict What Factors Accurately Predict Residency Performance?Residency Performance?Anesthesia: “Selection committee score,” which correlates with Step 1 score, doesn’t correlate with resident performance.
Metro DG, et. al., Anesth Analg 2005 100;502-5
Pediatrics: In the case of 69 residents at UVA, medical school grades, performance on standardized exams, interviews and match list rankings did not predict clinical performance during residency.
Borowitz SM, et. al., Arch Pediaatr Adolesc Med 2000 154;256-60
![Page 105: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/105.jpg)
Creating a Better Tomorrow2008 Annual Meeting
In Conclusion:In Conclusion:
We need better measures of performance during medical school.
Multiple types of evaluation may provide a more accurate picture of performance.
It’s about the Match. The best program may not be the best program for a particular student.
Honesty is the only policy.
![Page 106: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/106.jpg)
Creating a Better Tomorrow2008 Annual Meeting
In Conclusion:In Conclusion:
More categories of grading may help students in the Match
The MSPE is as accurate a measure as any in predicting residency performance.
NBME Step 2 but not NBME Step 1 scores have occasionally been found to predict clinical performance
![Page 107: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/107.jpg)
Creating a Better Tomorrow2008 Annual Meeting
In Conclusion:In Conclusion:
![Page 108: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/108.jpg)
Creating a Better Tomorrow2008 Annual Meeting
Question # 5
Do you favor continuing the MSPE in some form?
1. Yes
2. No
3. Unsure
![Page 109: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/109.jpg)
Creating a Better Tomorrow2008 Annual Meeting
Question # 6
If we keep the MSPE, do you favor changing the current format of the MSPE?
1. Yes
2. No
![Page 110: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/110.jpg)
Creating a Better Tomorrow2008 Annual Meeting
Question # 7
Would you remove the unique characteristics section of the MSPE?
1. Yes
2. No
![Page 111: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/111.jpg)
Creating a Better Tomorrow2008 Annual Meeting
Question # 8
Would you remove the clerkship narratives from the MSPE?
1. Yes
2. No
![Page 112: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/112.jpg)
Creating a Better Tomorrow2008 Annual Meeting
Question # 9
Do you favor changing the MSPE release date?
1. Yes
2. No
![Page 113: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/113.jpg)
Creating a Better Tomorrow2008 Annual Meeting
Question # 10
If you favor changing the MSPE release date, which would be the optimal date?
1. September 1st
2. September 15th
3. October 1st
4. October 15th
5. November 1st
6. January 11th
![Page 114: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/114.jpg)
Creating a Better Tomorrow2008 Annual Meeting
Question # 11
Would you favor a single date for MSPE release and the start of ERAS upload?
1. Yes
2. No
![Page 115: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/115.jpg)
Creating a Better Tomorrow2008 Annual Meeting
Question # 12
Would you favor a uniform release date of
Oct 1st for transcripts?
1. Yes
2. No
![Page 116: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/116.jpg)
Creating a Better Tomorrow2008 Annual Meeting
Question # 13
Do you think the GSA steering committee should propose a change in the MSPE, develop a survey on the MSPE, or should we a new advisory group be convened?
1. Propose an MSPE change
2. Develop a survey
3. Convene a new advisory group
![Page 117: AAMC Annual Meeting San Antonio, Texas November 1, 2008](https://reader035.vdocuments.us/reader035/viewer/2022062803/5681486c550346895db5793b/html5/thumbnails/117.jpg)
Creating a Better Tomorrow2008 Annual Meeting
Thank you for coming!!