can a brief on-line education tool improve surgical resident operative dictations? a prospective...
TRANSCRIPT
Can a Brief On-line Education Tool Improve Surgical Resident Operative Dictations?
A Prospective Evaluation
Alicia Kieninger, MD, Yi Wei Zhang, MD, Anna Dowdle, Amy Braddock, MD, Diego Hernandez, MD, Amy Kirby, MD, Robert
Robinson, MD
Background
• Surgical education encompasses much more than clinical and technical education– Patel, et. al, 2005
• Survey of ENT program directors and recent graduates• Topics lacking in current training programs
– Coding and documentation
– Office management
– Reimbursement
– “Business of medicine”
Background
• Proper documentation is the key to achieving adequate coding and reimbursement– Fakhry, et al., 2007
• Survey of surgical trainees and attendings regarding knowledge of coding and proper documentation
• Residents scored 54% correct vs. 77% for attendings
– Novitsky, et al., 2005• Deficiencies identified in resident operative notes would
have led to $18,200 in reduced revenues compared to attending notes
Background
• How do we teach residents the skills they need in less time?– Impact of the 80 hour work week
• Limited time spent in hospital must focus on clinical and technical skills
• Jones, et al., 2008– Implementation of a practice management program
improved resident coding accuracy from 36% to 88%
• Eichholz, et al., 2004– Brief teaching sessions improved ob-gyn resident dictation
scores
Background
• The operative note– Communication of details important for
patient care– Proper documentation for appropriate
billing– A skill important for all surgical trainees
Purpose
• To determine if a brief on-line education module can improve the quality and content of surgical resident operative dictations
Methods
• Participants– General surgery residents at a community based teaching
hospital
• IRB approval and informed consent obtained• Resident operative dictations collected for a 1 month
period before and after intervention– Blinded review by 2 of 4 participating attending surgeon
reviewers• Evaluation tool adapted from previously published, validated
tool (Vergis, et al., 2008)
• Dictations scored for content, quality, overall
Methods
• Statistical Analysis– Before and after scores compared using paired t-
test (SPSS v. 21.0)• Residents serve as their own controls
– p0.05 significant
Methods
• Module created for hospital-wide on-line compliance testing program– Brief power-point presentation
• Outlining keys to content and quality of operative dictations
– Several multiple choice questions
Results
Total Junior(PGY 1-3)
Senior(PGY 4-5)
Residents 11 7 4
Dictations (pre/post)
144 (86/58)
76 (46/30)
68 (40/28)
ResultsJunior vs. Senior Residents
0
0.2
0.4
0.6
0.8
1
Overall Content Quality
Junior (PGY 1-3)
Senior (PGY 4-5)
p<0.0001*p<0.0001* p<0.0001*
ResultsPaired t-Test
0
0.2
0.4
0.6
0.8
1
Overall Content Quality
Before
After
p= 0.314 p= 0.163 p= 0.230
Conclusions
• Senior resident dictations scored significantly higher than junior resident dictations
• There was a trend toward improved dictations after participation in the educational module for the group as whole
• Using the paired t-test, individual residents showed improvement after our intervention when compared to their own pre-intervention performance
Future Directions
• Expand information in education module based on deficiencies in operative notes
• Focus evaluations on single operative procedure to improve validity
• Expand to other residency programs• Add a component of direct one-on-one
feedback to residents to improve readability
References
• Patel, A.T., R.M. Bohmer, et al. (2005). “National assessment of business-of-medicine training and its implications for the development of a business-of-medicine curriculum.” Laryngoscope 115(1): 51-55.
• Fakhry, S.M., L. Robinson, et al. (2007). “Surgical residents’ knowledge of documentation and coding for professional services: an opportunity for a focused educational offering.” Am J Surg 194(2): 263-267.
• Novitsky, Y.W., R.F. Sing, et al. (2005). “Prospective, blinded evaluation of accuracy of operative reports dictated by surgical residents.” Am Surg 71(8): 627-631.
• Jones, K., R.A. Lebron, et al. (2008). “Practice management education during surgical residency.” Am J Surg 196(6): 878-881
• Eichholz, A.C., B.J. Van Voorhis, et al. (2004). “Operative note dictation: should it be taught routinely in residency program?” Obstet Gynecol 103(2): 342-346.
• Vergis, A., L. Gillman, et al. (2008). “Structured assessment format for evaluating operative reports in general surgery.” Am J Surg 195(1): 24-29.