prelimcheck: software to facilitate follow-up of resident preliminary reports
DESCRIPTION
PrelimCheck: Software to Facilitate Follow-Up of Resident Preliminary Reports. Phillip Cheng, MD MS Assistant Clinical Professor of Radiology. Objective. - PowerPoint PPT PresentationTRANSCRIPT
PrelimCheck: Software to Facilitate Follow-Up of
Resident Preliminary Reports
Phillip Cheng, MD MSAssistant Clinical Professor of Radiology
Objective
The purpose of this exhibit is to illustrate an open source desktop tool for follow-up of resident preliminary notes, allowing residents to rapidly evaluate their notes for discrepancies compared to final dictated reports.
Patient and study identifiers shown in this presentation have been obscured, altered or synthesized.
Introduction
• At our teaching institution, radiology residents begin taking independent call at night in their second year of training.
• During the night, their preliminary interpretations are recorded as notes in the PACS (Fuji Synapse). Typically 50-100 notes are placed by a resident per call night.
• The attending dictates the final reports in the morning and reviews preliminary notes by the resident.
Resident Notes in PACS
Resident Notes in PACS
Resident preliminary note at night
Attending note in the morning, resident now post-call at home
A Feedback Problem
• Independent call experience is considered by our residents and faculty as one of the most important educational experiences of their residency.
• However, feedback is an essential part of this education, and is not immediately available since the resident leaves in the morning before the attendings dictate final reads for the overnight studies.
• In particular, there is no mechanism in the PACS for discrepancies between preliminary and final reads to be flagged for later review.
Reviewing Discrepancies
We have reviewed several unsatisfactory options for reviewing overnight discrepancies with residents:
• Morning readout with attending: post-call resident is typically tired and not receptive to feedback
• Attending e-mail to residents: requires extra effort by attending especially for minor discrepancies, so such feedback is sporadic
• Residents reviewing own reads from a previous call night– Attractive since the residents are usually motivated to review
their call performance; however, identification and sorting of the preliminary and final reads from a call night is tedious and prone to incompleteness.
?PrelimCheck
PrelimCheck: Facilitating Feedback
Resident
Attending
Notes in PACS Notes in PACS,Dictated reports
PrelimCheck is a tool that completes the communication loop between attendings and residents.
PrelimCheck
• Open source, portable Windows desktop application
• Written in C#, requires .NET Framework 2.0
• Communicates directly with Fuji Synapse PACS
PrelimCheck - Usage
• Easy-to-use tool for residents to compare their preliminary notes with attending follow-up notes and final reports
• A resident specifies the location and time period of the call shift and clicks the Download button:
LocationTime Period
PrelimCheck - Usage
Download– Performs NTLM authentication to Synapse PACS server– SQL query over network for notes performed in
requested time period– Retrieve text of each note, and any dictated report for
the corresponding study– Typical download time for 12 hr shift: approx 1 minute
PrelimCheck - Usage
All preliminary notes for highlighted patient study
Dictated report, automatically scrolled to impression section
Sortable list of studies with notes
placed in the specified
time frame
Search list by keywords or
save to comma separated value
(CSV) file
PrelimCheck - Usage
• Requires approximately 5-10 minutes for review of overnight call cases
• No built-in intelligence to detect and flag discrepancies between preliminary and final report– Ease of rapid review partially obviates this
problem
Survey
• Use of the program is encouraged but optional in our residency
• One year after release, residents of all training levels were anonymously surveyed on usage, survey response rate 83%
Survey - Usage
Most respondents used the program. The main stated reason for non-usage was lack of familiarity with program, or lack of call experience.
Survey – Usability
Most respondents rated the program high in usability / ease of use.
Survey - Effectiveness
Despite lack of automated flagging of discrepancies, most respondents indicated that the program improved discrepancy detection.
Sample of Anonymous Comments
Limitations and Future Work
• Lack of automated discrepancy detection– Attendings can use standardized code words in notes to
accurately flag and rate discrepancies
• Slow network client-based implementation– Requires approximately 1 minute to download notes for a
call shift– Faster response possible with a server-side implementation
• Inability to filter results during network retrieval– Since notes are not indexed on PACS server, all notes
within time period are retrieved and then filtered on the client side
– Solvable with server-side implementation
Source Code Availability
• PrelimCheck source code is available under the Apache 2.0 open source license.
• All source available at Github:http://github.com/pmcheng/prelimcheck
Summary
PrelimCheck, an open source desktop application, facilitates rapid resident self-evaluation of on-call preliminary radiologic interpretations.
http://hsc.usc.edu/~phillimc/prelimcheck