introduction: the importance of board certification
DESCRIPTION
American Board of Surgery Examinations: Can We Identify Surgery Residency Applicants Who Will Pass the Examinations on the First Attempt?. John L. Shellito, M.D., Jacqueline S. Osland, M.D., Stephen D. Helmer, Ph.D., Frederic C. Chang, M.D. Department of Surgery - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Introduction: The Importance of Board Certification](https://reader034.vdocuments.us/reader034/viewer/2022051517/56815a65550346895dc7af58/html5/thumbnails/1.jpg)
American Board of Surgery Examinations: Can We Identify Surgery Residency Applicants
Who Will Pass the Examinations on the First Attempt?
John L. Shellito, M.D., Jacqueline S. Osland, M.D., Stephen D. Helmer, Ph.D., Frederic C. Chang, M.D.
Department of SurgeryUniversity of Kansas School of Medicine – Wichita
![Page 2: Introduction: The Importance of Board Certification](https://reader034.vdocuments.us/reader034/viewer/2022051517/56815a65550346895dc7af58/html5/thumbnails/2.jpg)
Introduction: The Importance of Board Certification
• One of the key demonstrations of the effectiveness of a residency’s training program
• ACGME Program Requirements-Surgery– 65% first attempt pass rate for both the
qualifying and certifying exam
![Page 3: Introduction: The Importance of Board Certification](https://reader034.vdocuments.us/reader034/viewer/2022051517/56815a65550346895dc7af58/html5/thumbnails/3.jpg)
Objective
This study was performed to identify and evaluate factors which predict successful completion of both the American Board of Surgery Qualifying Exam (QE) and Certifying Exam (CE) on the first attempt
![Page 4: Introduction: The Importance of Board Certification](https://reader034.vdocuments.us/reader034/viewer/2022051517/56815a65550346895dc7af58/html5/thumbnails/4.jpg)
Methods
• Retrospective review – University of Kansas SOM-Wichita
Surgery Residency Program– Included residents entering the program from
1990-2001
![Page 5: Introduction: The Importance of Board Certification](https://reader034.vdocuments.us/reader034/viewer/2022051517/56815a65550346895dc7af58/html5/thumbnails/5.jpg)
Pre-Admission Measures
• Age and gender• Class rank• AOA status• Honors in third year surgery clerkship• Interview scores• National Board of Medical Examiners (NBME)
Part 1 and Part 2 scores or United States Medical Licensing Examination (USMLE) Step 1 and Step 2 scores
• Rank list number
![Page 6: Introduction: The Importance of Board Certification](https://reader034.vdocuments.us/reader034/viewer/2022051517/56815a65550346895dc7af58/html5/thumbnails/6.jpg)
Post-admission Measures
• American Board of Surgery In Training Examination (ABSITE) scores
• Resident awards (Research, Teaching, Outstanding Senior Resident)
• Faculty evaluations of the 5th year residents
![Page 7: Introduction: The Importance of Board Certification](https://reader034.vdocuments.us/reader034/viewer/2022051517/56815a65550346895dc7af58/html5/thumbnails/7.jpg)
Statistical Analysis
• Univariate analyses were performed using Fischer’s exact test for the dichotomous variables (i.e. gender, AOA status, honors)
• Two-sample Wilcoxon rank-sum test was used for continuous variables (i.e. scores, age) and ordinal data
• Logistic regression analysis was performed on the pre-admission variables
![Page 8: Introduction: The Importance of Board Certification](https://reader034.vdocuments.us/reader034/viewer/2022051517/56815a65550346895dc7af58/html5/thumbnails/8.jpg)
Results
• 77 medical students and residents– 15 subjects excluded
• 5 NBME or USMLE scores not available • 8 Left the program• 2 never attempted the QE (both plastic surgeons)
• 62 subjects remained after exclusions
![Page 9: Introduction: The Importance of Board Certification](https://reader034.vdocuments.us/reader034/viewer/2022051517/56815a65550346895dc7af58/html5/thumbnails/9.jpg)
UKSM-W Surgery Residency Program 1990-2001
• >90% of our graduates attended medical school in the Midwest (50% from KU)– One foreign medical graduate
• 16% (10/62) AOA• Pass rates
– First time: 77.4%– Eventual: 96.8%
• 2/62 (plastic surgery)
• Majority entered private practice (73%)• No difference in first time pass rates over the
study period
![Page 10: Introduction: The Importance of Board Certification](https://reader034.vdocuments.us/reader034/viewer/2022051517/56815a65550346895dc7af58/html5/thumbnails/10.jpg)
Results
• No significant difference based on gender• 50 (80.7%) were male• First time pass rate
– 74% for males/ 91.7% for females (p=0.179)
• Residents who successfully passed were slightly younger at time of application
• 27.1 years vs. 28.6 years (p=0.047)
![Page 11: Introduction: The Importance of Board Certification](https://reader034.vdocuments.us/reader034/viewer/2022051517/56815a65550346895dc7af58/html5/thumbnails/11.jpg)
Effect of Class Rank on First Attempt Pass Rates
Class Rank Group Number Percent
Top 1/3rd 22/23 95.7%
Middle 1/3rd 14/22 63.6%
Bottom 1/3rd 3/5 60.0%
P=0.0073
![Page 12: Introduction: The Importance of Board Certification](https://reader034.vdocuments.us/reader034/viewer/2022051517/56815a65550346895dc7af58/html5/thumbnails/12.jpg)
Effect of Step 1 scores on First Attempt Pass Rates
Step 1 Group
Number Percent
Top 50% (>200)
36/42 85.7%
Bottom 50% (<200)
12/20 60.0%
P=0.0247
![Page 13: Introduction: The Importance of Board Certification](https://reader034.vdocuments.us/reader034/viewer/2022051517/56815a65550346895dc7af58/html5/thumbnails/13.jpg)
Effect of Step 2 scores on First Attempt Pass Rates
Step 1 Group Number Percent
Top 3 Quartiles (>186)
44/55 80.0%
Bottom Quartile (<186)
2/5 40.0%
P=0.0447
![Page 14: Introduction: The Importance of Board Certification](https://reader034.vdocuments.us/reader034/viewer/2022051517/56815a65550346895dc7af58/html5/thumbnails/14.jpg)
Results
• Cumulative scores for step 1 and 2– >382 (top 2/3rds)
• 81.1% first time pass rate
– <382 (bottom 1/3rd)• 42.9% first time pass rate
![Page 15: Introduction: The Importance of Board Certification](https://reader034.vdocuments.us/reader034/viewer/2022051517/56815a65550346895dc7af58/html5/thumbnails/15.jpg)
Results
Logistic regression analysis showed Step 1 scores to be the only independent predictor from the group of pre-admission factors for first time pass rates
![Page 16: Introduction: The Importance of Board Certification](https://reader034.vdocuments.us/reader034/viewer/2022051517/56815a65550346895dc7af58/html5/thumbnails/16.jpg)
Effect of ABSITE on First Attempt Pass Rates
ABSITE Group Number Percent
PGY-1 >50th percentile 41/50 82.0%
<50th percentile 5/10 50.0% P=0.044
PGY-3
>50th percentile 36/40 90.0%
<50th percentile 12/22 54.5% P=0.002
![Page 17: Introduction: The Importance of Board Certification](https://reader034.vdocuments.us/reader034/viewer/2022051517/56815a65550346895dc7af58/html5/thumbnails/17.jpg)
Effect of ABSITE on First Attempt Pass Rates
ABSITE Group Number Percent
PGY-4
Top 1/3rd 19/22 86.4%
Middle 1/3rd 23/26 88.5%
Bottom 1/3rd 6/14 42.9%P=0.009
PGY-5
Top 1/3rd 23/24 95.8%
Middle 1/3rd 15/21 71.4%
Bottom 1/3rd 10/17 58.8%P=0.004
![Page 18: Introduction: The Importance of Board Certification](https://reader034.vdocuments.us/reader034/viewer/2022051517/56815a65550346895dc7af58/html5/thumbnails/18.jpg)
Results
• Residents winning an award during residency had a higher first time pass rate – 92% versus 67%
• Faculty evaluations of 5th year residents did not predict board success
![Page 19: Introduction: The Importance of Board Certification](https://reader034.vdocuments.us/reader034/viewer/2022051517/56815a65550346895dc7af58/html5/thumbnails/19.jpg)
Results
• Pre-admission factors that failed to predict success for first time pass rates– Honors in 3rd year surgery clerkship– AOA status
• 10/10 first time pass rate, p=0.061
– Interview score– Ranking in the match
![Page 20: Introduction: The Importance of Board Certification](https://reader034.vdocuments.us/reader034/viewer/2022051517/56815a65550346895dc7af58/html5/thumbnails/20.jpg)
Discussion
• Clerkship grades• Honors grade in third year surgery
clerkship• Dean’s letters• Letters of recommendations• Interview Score• Class rank• USMLE scores
![Page 21: Introduction: The Importance of Board Certification](https://reader034.vdocuments.us/reader034/viewer/2022051517/56815a65550346895dc7af58/html5/thumbnails/21.jpg)
Discussion
• Xu 1998– Positive correlation between eventual board
certification status and past academic performance– Included other surgical specialties in addition to
general surgery
• Silber 2005– Step 2 score single most important predictor for
eventual OB/GYN board certification– Graduates from a single medical school, multiple
residency programs
![Page 22: Introduction: The Importance of Board Certification](https://reader034.vdocuments.us/reader034/viewer/2022051517/56815a65550346895dc7af58/html5/thumbnails/22.jpg)
![Page 23: Introduction: The Importance of Board Certification](https://reader034.vdocuments.us/reader034/viewer/2022051517/56815a65550346895dc7af58/html5/thumbnails/23.jpg)
Summary
This is first study to examine factors predicting first time board certification rates for general surgery residency graduates for both the QE and CE
![Page 24: Introduction: The Importance of Board Certification](https://reader034.vdocuments.us/reader034/viewer/2022051517/56815a65550346895dc7af58/html5/thumbnails/24.jpg)
Limitations
• Small sample size
• Bias injected from our selection process– No information about the success or failure of
applicants who were excluded by our selection committee
• Single residency training program
![Page 25: Introduction: The Importance of Board Certification](https://reader034.vdocuments.us/reader034/viewer/2022051517/56815a65550346895dc7af58/html5/thumbnails/25.jpg)
Summary
• Academic aptitude by itself does not predict the multifaceted state of competence of a surgeon
• Professionalism, manual dexterity, communication abilities, and integrity are not measured by standardized tests
• Sorting applicants that are above a certain threshold of intellectual capacity solely by test scores may not be a good way to obtain the best residents
![Page 26: Introduction: The Importance of Board Certification](https://reader034.vdocuments.us/reader034/viewer/2022051517/56815a65550346895dc7af58/html5/thumbnails/26.jpg)
Conclusion
Our study indicates a positive correlation between class rank, USMLE scores, ABSITE scores and first time pass rates for ABS certification
![Page 27: Introduction: The Importance of Board Certification](https://reader034.vdocuments.us/reader034/viewer/2022051517/56815a65550346895dc7af58/html5/thumbnails/27.jpg)