zabar final report cg

Picker Institute/ACGME Challenge Grants

Project Name: Emergency Medicine Resident Training in Inter-professionalism Skills

Evaluating a Needs-Based Curriculum

FINAL REPORT (February 29, 2007 – April 15, 2008)

Date of Report: April 15, 2008 Grant Number: 16 Grantee Institution: New York University School of Medicine Principal Investigator Information: Sondra Zabar, MD Associate Professor of Medicine New York University School of Medicine 550 First Avenue, OBV D401 New York, NY 10016 (212) 263-1138 [email protected] Co-Investigator Information: Linda Regan, MD Assistant Professor of Emergency Medicine New York University School of Medicine [email protected]

TABLE OF CONTENTS

A. EXECUTIVE SUMMARY (ABSTRACT)........................................................................................................2 B. INTRODUCTION (BACKGROUND)............................................................................................................3 C. METHODS (PROJECT IMPLEMENTATION AND ADMINISTRATION) ...........................................4 D. RESULTS............................................................................................................................................................9 E. DISCUSSION ...................................................................................................................................................14 F. DISSEMINATION ..........................................................................................................................................16 G. FINANCIAL REPORT ...................................................................................................................................16 H. ATTACHMENTS ............................................................................................................................................17

ATTACHMENT – SAMPLE CASE AND CHECKLIST (MEDICAL ERROR).........................................................................18 ATTACHMENT – SAMPLE REPORT CARD ..................................................................................................................28 ATTACHMENT – SESSION OBJECTIVES .....................................................................................................................34 ATTACHMENT – SAMPLE POCKET CARD ..................................................................................................................35 ATTACHMENT – GOLD FOUNDATION ABSTRACT.....................................................................................................36

EM RESIDENT TRAINING IN INTERPERSONAL AND PROFESSIONALISM SKILLS (Final Report) Page 2 PI: Sondra Zabar, MD NYU School of Medicine

A. EXECUTIVE SUMMARY (ABSTRACT)

Since the 1960’s, Emergency Medicine (EM) researchers’ efforts have worked to demonstrate the importance of patient-centered doctor-patient communication, only acknowledging decades later that advancing such patient-centered care will require increased and effective provider education. Having had experience with the development and implementation of a controlled study on the impact of comprehensive, integrated clinical communication skills curriculum on student patient-centered skills, the Section of Primary Care faculty at New York University School of Medicine’s were prepared and eager to partner with Emergency Medicine faculty on this very important topic. With the commitment of NYUSOM-Bellevue Emergency Medicine Residency leadership, we created the Emergency Medicine Professionalism and Communication Training (EMPACT) Project.

EMPACT aimed to improve EM resident competency in communication and professionalism through the development, implementation, and evaluation of new curriculum and assessment measures. Our objectives were to: 1) design, implement and evaluate patient-centered healthcare curriculum for all 60 EM residents; 2) evaluate predictive validity of Objective Structured Clinical Examinations (OSCEs) by assessing correlation of OSCE performance with actual resident performance in emergent care setting for cohort of PGY2 residents (n=15); and 3) disseminate this Patient-Centered Care educational program to EM programs nationally. We conducted EMPACT in four phases: Phase I) established baseline competency of EM interns using a 5 station OSCE; Phase II) integrated an interactive skills-based series of five workshops focusing on interpersonal and professionalism skills—into monthly required EM seminar series; Phase III) conducted post-curriculum OSCE to evaluate impact of curriculum; and Phase IV) developed and implemented two “Unannounced” Standardized Patient (USP) cases.

In completing all four phases of the EMPACT Project, we learned a lot about our residents, how to improve our OSCEs, and how to implement another USP project in the future. Residents agreed that the curriculum helped them to improve on the strengths and weaknesses identified by the OSCE. Our comparison of the residents’ pre- and post-OSCE performances has shown significant improvement in overall Communication, Relationship Development, and Patient Education Skills. Also, through our USP pilot, we learned that we will need a better understanding of the system in which we practice before embarking on such an endeavor and more USP cases to better gauge how residents perform in reality.

Even having taught communication skills in other disciplines, teaching the same skills in EM provided rich learning opportunities for us as curriculum innovators, evaluators, and administrators. It is clear that learners need and appreciate curricula that are interactive and role model key patient centered skills. Performance based assessment, OSCE and Unannounced Patients though time intensive are meaningful assessment tools for both learners and programs.


B. INTRODUCTION (BACKGROUND)

Since the 1960’s, Emergency Medicine (EM) researchers’ efforts have worked to demonstrate the importance of patient-centered doctor-patient communication, only acknowledging decades later that advancing such patient-centered care will require increased and effective provider education. 12 Having completed the Macy Initiative in Health Communication, a controlled study of the impact of comprehensive, integrated clinical communication skills curriculum on student patient-centered skills,3 the Section of Primary Care (PC) faculty at New York University School of Medicine’s (NYUSOM) were prepared and eager to continue such work with the EM faculty on this very important topic. Drs. Linda Regan, Jeffrey Manko, and Eric Legome, directors of the NYUSOM-Bellevue Residency in EM, an integrated four-year residency dedicated to training highly competent emergency physicians, shared this enthusiasm and began to plan for such an initiative.

Our program, entitled Emergency Medicine Professionalism and Communication Training (EMPACT), expands on previous work by assessing and improving EM resident competency in communication and professionalism through the development, implementation, and evaluation of new curriculum and assessment measures. To ensure clinical competency of EM graduates in delivering patient-centered care, we incorporated both ACGME core competency requirements and several of the Picker Institute’s Dimensions of Patient-Centered Care into our program/research design. Our objectives were to:

1. Design, implement and evaluate patient-centered healthcare curriculum for all 60 EM residents;

2. Evaluate predictive validity of Objective Structured Clinical Examinations (OSCEs) by assessing correlation of OSCE performance with actual resident performance in emergent care setting for a cohort of PGY2 residents (n=15); and

3. Disseminate this Patient-Centered Care educational program to EM programs nationally.

1 Korsch BM, Negrete VF. Doctor-patient communication. Sci Am. 1972 Aug; 227(2):66-74. 2 Rhodes KV, Vieth T, He T, Miller A, Howes DS, Bailey O, Walter J, Frankel R, Levinson W. Resuscitating the physician-patient relationship: emergency department communication in an academic medical center. Ann Emerg Med. 2004 Sep; 44(3):262-7. 3 Kalet A, Pugnaire MP, Cole-Kelly K, Janicik R, Ferrara E, Schwartz MD, Lipkin M Jr., Lazare A. Teaching communication in clinical clerkships: a model from the Macy Initiative in Health Communications. Acad Med. 2004; 79(6):511-20.


C. METHODS (PROJECT IMPLEMENTATION AND ADMINISTRATION) To achieve our objectives, we conducted EMPACT in four phases. (See Figure 1. Project

Timeline) In Phase I, we established a baseline competency of EM interns using a 5-station OSCE. Phase II, we developed an interactive skills-based series of five workshops focusing on interpersonal and professionalism skills and integrated them into required monthly EM seminar series. In Phase III, we conducted a post-curriculum OSCE to evaluate impact of curriculum. In Phase IV, we developed and implemented two cases for the “unannounced” standardized patient (USP) project.4

Figure 1. Project Timeline

3/2007 4/2007 5/2007 6/2007 7/2007 8/2007 9/2007 10/2007 11/2007 12/2008 1/2008 2/2008

Curriculum Curriculum Development Curriculum Implementation Curriculum Packaging

Evaluation OSCE Development (Case

development, SP Recruitment & Training)

Pre-OSCE Data Analysis Report Card Generation Post-

OSCE

Individual Remediation of

Poor Performers

“Unannounced” SP Program

Program Development (Logistics of Implementation) Case Development

Generation of “Patient” in

computer record

Program Implementation in

ER

Data Analysis

Project Dissemination

Mid-year

Report Production of manuscripts, abstract

submissions, final summary reports, etc.

Phase I - Establish baseline competency of EM interns using a 5-station OSCE In order to determine effectiveness of our curriculum, we chose to evaluate a subset of

resident performance in a pre- and post-OSCE. We wrote five cases and developed checklists that assessed communication skills in scenarios commonly encountered by EM residents (See Table 1. OSCE Cases). The checklists used to evaluate residents’ performance included items that assessed overall communication skills (information gathering, relationship development, and patient education), case-specific skills, and whether patients would recommend seeing the resident as their physician.

Table 1. OSCE Cases OSCE Case Picker Dimension Communication Skills Informed Consent Via an Interpreter

Access; Respect for patient’s values, preferences, and expressed needs; Information, communication and education

Obtaining Informed Consent; Patient Education; Dealing with Challenging Patient

Disclosing a Medical Error

Respect for patient’s values, preferences, and expressed needs; Emotional support and alleviation of fear and anxiety

Rapport Building; Emotion Handling

Delivering Unexpected Bad News

Emotional support and alleviation of fear and anxiety; Information, communication and education

Emotion Handling; Patient Education

Transferring Care to Another Service

Coordination and integration of care; Transition and continuity

Interdisciplinary Communication; Telephone Skills

Using the Emergency Room for Primary Care

Access; Respect for patient’s values, preferences, and expressed needs; Emotional support and alleviation of fear and anxiety; Information, communication and education

Dealing with Challenging Patient; Emotion Handling; Patient Education

4 Kravitz RL, Epstein RM, Feldman MD, Franz CE, Azari R, Wilkes MS, Hinton L, Franks P. Influence of Patients’ Requests for Direct-to-Consumer Advertised Antidepressants: A randomized controlled trial. JAMA 2005;293:1995-2002.


The preparation for the pre-OSCE included multiple preparatory steps. We trained five standardized patients (SPs) to reliably and repeatedly portray their roles for the OSCE. SP training sessions allowed the SPs to ask questions about their character, develop the improvisational range that should be portrayed in their role, and practice how to consistently respond to participant reactions. Prior to the pre-OSCE, we piloted the five cases and videotaped them to fine tune the content of the cases and the checklists. Five EM chief residents, junior faculty, and medical students were assessed as the participants. After reviewing the videos of their performances, examining the data from checklists completed by the SPs, and hearing feedback from the participants in a debriefing session, we adjusted the OSCE and checklist for clarity, timing, and realism. After making the appropriate adjustments to the five cases, we were ready to launch the OSCE.

We conducted the pre-OSCE in three sessions. At each session, five residents went through all five stations. All 15 PGY2 EM residents completed the OSCE. We chose to test the PGY2 because we believe, developmentally, the intervention will have the most impact at this stage of learner. 90% of the OSCEs were audio and videotaped for the purposes of assessing inter-rater reliability afterwards.

Colleen Gillespie, PhD, our evaluation researcher, compiled the feedback from faculty observers and checklist data from SPs and summarized them as both a presentation for EM faculty and report cards for each individual resident (See Attachments – Sample Report Card). The report card noted each resident’s performance in five core areas: 1) communication, 2) overall recommendation, 3) ratings of ability to apply expertise, 4) specific skills across cases, and 5) overall case-specific skill scores. One case was not reliably scored (Delivering Bad News) and so scores associated with that case should be interpreted with caution (details of how these scores were calculated are included in the sample report card provided in the Attachments).

Overall, we noted there was room for improvement for all the residents in their Data Gathering, Relationship Building, and Patient Education Skills. Residents performed best at Data Gathering, less well at Relationship Building, and worst at Patient Education. As a group they also scored low on Emotion Handling. Such information was also included in the report cards, which demonstrated how the individual performed in comparison to the rest of the participants. This data guided us in our focus and approach to key topics covered in the curriculum. Residents told their program director that they found the OSCEs enjoyable and educational.

Phase II - Integrate an interactive skills-based series of five workshops —focusing on

interpersonal and professionalism skills—into monthly EM seminar series We developed curricula based on the Macy model and other literature that taught five key

patient-care tasks, including: 1) relationship development and maintenance, 2) patient assessment, 3) education and counseling, 4) negotiation and shared decision making, and 5) organization and time management of EM. Our curriculum was composed of five one-hour interactive sessions that addressed each of the core skills during the OSCE using different teaching modalities. (See Table 2. EMPACT Course Schedule) We clearly delineated cognitive, skills, and affective objectives for each session and highlighted them at the beginning of each session. We also created pocket cards that included take-home points and a bibliography of relevant literature for each session. (See Attachment X for the Session Objectives) Approximately 40 residents attended each of the session, with ~10 PGY2 residents at each.


Table 2. EMPACT Course Schedule Session Title Date Picker Dimension Communication

Skills Teaching Method

1. Making Every Session Count: Effective Communication Skills in the Emergency Room

08/01/2007 Respect for patient’s values, preferences, and expressed needs; Information, communication and education

Patient Education, Rapport Building

Videotape Reenactment and Debriefing, Mini Lecture

2. Interdisciplinary Communication and Respect

09/12/2007 Coordination and integration of care; Transition and continuity

Conflict Negotiation; Telephone Skills

Audiotape Trigger, Role Play

3. Delivering Bad News in the Emergency Department

10/03/2007 Emotional support and alleviation of fear and anxiety; Information, communication and education

Emotion Handling Videotape Trigger from Medical TV Show, Rolling Role Play between Attending and SP

4. Dealing with Culturally Diverse Populations in the Emergency Department

11/07/2007 Access; Respect for patient’s values, preferences, and expressed needs; Information, communication and education

Effective use of an interpreter, Elements of informed consent

Rolling Role Play between Residents and SP, Mini Lecture

5. Discussing Medical Errors in the Emergency Department

12/05/2007 Respect for patient’s values, preferences, and expressed needs; Emotional support and alleviation of fear and anxiety

Emotion Handling; Patient Education; Dealing with Challenging Patient

Videotape Trigger from Medical TV Show, Role Play with Small Groups

The first session, entitled “Making Every Session Count: Effective Communication Skills in

the Emergency Room,” aimed to provide residents with tools to maximize the effectiveness of their communication with patients and their families. The session began with a videotaped reenactment of OSCE case as a trigger for discussion. The session also included a PowerPoint presentation of how residents performed in the OSCE overall and how they can improve their professionalism skills. Residents’ feedback on this first session was very positive. They noted, “I feel the hurried atmosphere of the ER causes the communication skills to atrophy. I think this was a useful reminder of that and an effective tool relevant to ER situations.”

Our second session, entitled “Interdisciplinary Communication and Respect,” aimed to teach residents to effectively work with the professionals around them to optimize patient care. This session proceeded with a general discussion of how interdisciplinary communication can be both positive and negative. Then, we played a re-enacted audiotape of the “Transferring Care to Another Service” case they experienced in the OSCE, which we used as the trigger for discussion on how interdisciplinary communication can be made better. A short lecture outlined the key steps and skills to successful conflict negotiation and effective phone skills. Residents then participated in a role play to practice these skills. We debriefed the role play as a large group to help residents identify what personal traits or attitudes are barriers for successful interdisciplinary communication. We handed out a pocket card summarizing an approach to conflict negotiation and telephone skills. A number of residents stated that this was the first time these issues were ever addressed as part of their curriculum. In particular, they said, “Good suggestions on how to approach multidisciplinary communication. Short handout with key points helpful. Tape [was] very pertinent and important.”


The third session, entitled “Breaking Bad News in the Emergency Department,” aimed to improve residents’ effectiveness in their delivery of bad news and provide residents with facts about post-death procedures. The session began with the viewing of a trigger video clip from the Fox television series, “House,” where a patient is abruptly given an AIDS diagnosis by the maverick, Dr. Gregory House. This led to a conversation about what contributes to the sensitivities and difficulties of delivering bad news, regardless of how the residents may perceive the severity of the news to be (e.g. broken limb, new diagnosis of disease, or death of a loved one). Then, the residents directed a rolling role play between an SP and Dr. Regan, who had to break the news of a positive HIV diagnosis. The roll play was stopped a few times midstream to allow for a discussion of possible strategies to better manage the situation. The session concluded with the key take-home points, including protocol on how to follow-up on death notification, which residents took with them on pocket cards. The residents notes that this topic "...can be fairly dry, has been done so much in med school, BUT this was a very strong revisiting of this hard issue.” In particular, they said the session was “excellent because it was DYNAMIC… well prepared, very interactive. The role play was very well done."

The fourth session, entitled “Dealing with Culturally Diverse Populations in the Emergency Department,” aimed to improve residents interactions with culturally diverse patients and understand appropriate use of interpreters in the ED. The session began with a discussion of the challenges of providing cross-cultural care, including how different health beliefs affect patient and provider behavior and how language can act as the most apparent barrier. The conversation turned to the challenge of working with various kinds of interpreters and strategies to overcome common errors. During this session, a pair of Bengali-speaking SPs participated in a role play with Dr. Regan, who demonstrated a bad version. Residents were asked to strategize on how to improve the interaction and asked to come up and interact with the sp in front of the group. We used a Rolling Role Play as the educational strategy for this session. We concluded the session with a summary on how to use interpreters better. Residents again took home pocket cards that reviewed the key skills. They enjoyed the use of small group role play and said it was "a refreshing approach to this topic."

The fifth session, entitled “Medical Errors in the Emergency Department,” aimed to improve resident’s effectiveness in their disclosure of medical errors. This session began with a viewing of a videoclip from the NBC television series, “Scrubs,” where a resident debates whether or not to expose a potential medical error he believes was committed by his friend and colleague. While comical, this clip helped the residents to begin broaching the difficult topic. Then, the session continued with a discussion of frequent barriers to the disclosure of medical errors in general, as well as specific to the ED. Residents were then given a checklist of items to follow which represented common good practice for this sensitive topic. After explicitly discussing the 5Ws (Who, What, Where Why, and When), the session proceeded with a skills practice. Each group of three to five residents were given a scenario where one resident played the patient and another played the resident who had to deliver the news about one of three medical error scenarios. Each group was facilitated by a faculty member. The rest of the group observed and scored the scenario with a checklist, similar to that which the SP's would use during the OSCE. Each small group reported larger group the key learning points from their scenario. The session ended with the viewing of a final clip from “Scrubs,” where everyone is relieved to find out an error did not occur and a re-emphasis on the take-home points for the session.

Phase III - Conduct a post-curriculum OSCE to evaluate impact of curriculum. Two months following the final EMPACT session, we held the post-OSCE. For comparison

purposes, we used the same five cases as the pre-OSCE. Due to the availability of the SPs, however, we needed to train new SPs for four of the five cases. However, we purposefully chose SPs whom


we have worked with in the past and found to be reliable raters. Consequently, we believe the overall integrity of the OSCE remains the same.

The post-OSCE was held in three sessions, with approximately five residents attending each session. All 15 EM PGY2s participated in the post-OSCE and completed all five stations. Again, for interrater reliability purposes, each station was videotaped, with the exception of the Transfer Case, which was audio taped.

Colleen Gillespie and Tavinder Ark, MSc, our research associate, collected feedback from faculty observers, checklist data from SPs, and resident satisfaction data relating to both the EMPACT OSCE and curriculum. They summarized all data into report cards for each individual resident, this time with a comparison of how their performance differed in the two OSCEs. The report card reported each resident’s comparative performance in five core areas: 1) communication, 2) overall recommendation, 3) ratings of ability to apply expertise, 4) specific skills across cases, and 5) overall case-specific skill scores. The comparative data of the pre- and post-OSCE are described later in the Results section.

Phase IV - Develop and implement two cases for the “unannounced” standardized

patient (USP) project. The USP portion of EMPACT, was both exciting and educational. To our knowledge, based

on an extensive literature search in PubMed and Medline, the use of USPs in emergency clinical settings had not been done prior to our attempt. Despite posing us with many labor-intensive challenges, with full prior consent of residents, support of department and hospital leadership, and approval from our IRB, we launched the USP program in December 2007 and assessed 12 residents through 17 successful USP encounters in the ER.

For comparison purposes and to protect our SPs, we chose to use the Medical Error and Repeat Visitor cases for the USP visits, as they required non-invasive interventions by the residents.

Having obtained verbal confirmation from Medical Records, Registration, EM Nurses, EM Attendings, and the radiologists, we were poised to begin this aspect of the project. As the USPs in both the cases were supposed to have visited the Bellevue ER before, both cases required the entry of previous medical notes, x-rays, MRIs, and labs in the medical record system. We obtained specified Medical Record Numbers for the USPs. However, the challenges of this effort soon became apparent.

The rate limiting step in setting up the Medical Error case was the time frame allowed by MISYS, the medical records system, to enter prior visits into the record history. Because the USP was supposed to have visited the ER two days prior to the actual USP visit, we needed a visit to be opened two days prior in real time. The system would not allow us to enter future visits. This meant that the Bellevue Hospital EM Admitting needed to be ready to open the visit when we asked two days prior to the actual USP visit. This also meant that the PACS team, the group that handled all radiology related issues, had to be ready to upload the X-ray images and reports onto the system once the prior visit was opened. Because this was a voluntary effort on the part of the Admitting and PACS, it took a few tries to come up with an efficient system for getting all the required information adequately noted in the USPs fictitious medical records prior to the actual USP visit.

The main challenge of the Repeat Visitor case was the manipulation of the MRI images. Based on the original version of our case, the USP was supposed to have visited the Bellevue ER twice in the past and have taken MRI images here. In order to have the MRI images reflect the case details of each visit (e.g. dates, patient name, etc.), we needed to edit more than 50 images per visit. We consulted Sectra, the company that services our PACS system, who offered to write us a program that would quickly do so for $12,000. Since this was not possible given our financial situation, we ended up editing the USP case. In the new version, the USP visited another ER in New


York City two times and got an MRI at another location. The USP then brought the MRI report to the actual USP visit at Bellevue.

The third most prominent challenge of the USP project resulted from our need to limit the number of informed people in the ER, the unpredictability of the ER, and the assignment of the residents to the USP case on a given day. We tried to limit the number of people in the ER who knew that a USP was present to avoid detection. Although we tried our best to have the USP triaged exactly to where the targeted resident was supposed to be working on the given day, our efforts were often thwarted by eager medical students, rotating orthopedic residents, or unexpected schedule changes. During a few of our scheduled visits, the USPs were mistakenly examined by another care provider while the target resident was called away to see a more acutely ill patient. The attending may have known about the USP, but at times was engaged in the care of another patient when non-targeted personnel elected to see the USP.

After 29 attempts, we successfully evaluated 17 of the 30 planned visits (five residents were visited by both types of USPs, which accounted for ten of the visits). We audio taped ~71% of the encounters (12/17), which we will use to establish intra- and inter-rater reliability. Following each visit, we videotaped the USPs as they debriefed the entire experience and completed the checklists. As the last USP visit was just completed on April 8, 2008, a comprehensive comparison of the USP and OSCE performances is still pending.

D. RESULTS The OSCEs assess residents’ clinical skills in two major areas: 1) Communication Skills and

2) Case-Specific Skills. The Communication Skills describe residents’ ability in information gathering, relationship development and patient education skills. The Case-Specific Skills describe the residents’ ability to perform skills specific to each case. They are divided into five broad categories: 1) managing a difficult case, 2) accountability, 3) delivering bad news, 4) patient education and 5) treatment plan and management.

For the EMPACT OSCE and USP visits, Communication and Case-Specific Skills questions are rated by the SP on a 3-point scale of “not done” (resident did not perform the task at all), “partially done” (the resident attempted the task, but did not do it entirely correctly), or “well done” (the resident performed the task and did it correctly). In addition, residents’ were rated by the SPs on the degree to which they would recommend this doctor to a friend based on their interpersonal skills and expertise on a 4-point scale (1= Not recommend and 4= Highly Recommend). Residents’ Communication and Case-Specific Skills are calculated as the percent of items rated as “well done” across all cases. The overall recommendation rating was based on interpersonal skills and expertise was calculated across all cases as a mean average on a 4-point scale. These score was calculated across all 5 cases. A pre and post comparison was conducted. For the USP visits, this score was computed only across the repeat visitor case and broken wrist (medical error) and compared to the pre and post of only these two cases.

D1. Resident Experience of EMPACT Data on residents’ exposure to actual clinical situations similar to the OSCE cases highlight

the importance of having an opportunity to practice low frequency clinical situations: only 29% reported encountering a situation involving giving bad news since the pre-curriculum OSCE and slightly less than half (43%) reported exposure to a clinical situation involving a medical mistake. Despite evidence reported below that residents made substantial improvements from pre- to post-curriculum in some core clinical areas, from more than a third to close to half of residents reported


that their performance on the post OSCE was “about the same” as their performance on the pre OSCE (depending on the case, % ranged from 36% to 50%). Most agreed that the OSCE helped them identify their strengths and weaknesses (60%) and provided a good cross-section of cases (74%). However, some skepticism of the value of OSCEs was also apparent as just over half did not think that the OSCEs taught them something new (54%) or was a fair evaluation of their skills (60%). When asked in an open-ended manner to describe what was most helpful about EMPACT most focused on the OSCE (perhaps reinforced by having just completed the post OSCE!), focusing on practice (“repeated exposure to clinical scenarios”) and on being able to assess and reflect on one’s skills (“recognizing my triggers for what is a problem for me;” “self reflection about my weaknesses,” “the situations are a good reflection of what we see in the ED and they highlight some of the weaknesses we have in dealing with difficult situations. I know I tend to make the same mistakes over and over again.”). Several residents simply said that the EMPACT “curriculum” was the most helpful aspect of EMPACT overall.

D2. Impact of the Curriculum: Pre- vs. Post-Curriculum OSCE Results

Comparison of the pre- and post-curriculum OSCEs showed significant improvement in residents’ overall Communication Skills (pre=53.4% SD 14.9% vs. post=65.5% SD 11.5%; p=0.003). In particular, they improved on overall Relationship Development skills (pre=49.2% SD 21.5% vs. post=59.8% SD 17.8%; p=0.025) and especially in their overall Patient Education skills (pre=31.6% SD 15.1% vs. post=57.0% SD 15.2%, p<.001).

In terms of residents’ case-specific skills, significant improvement from pre- to post-curriculum was seen in the Repeat Visitor case (pre=38.7% SD 18.1% vs. post=73.3% SD 16.7%, p<.001) and close to significant improvement in the Bad News case (pre=54.0% SD 15.5% vs. post=66.9% SD 22.1%; p=.066).

SPs rated residents more highly in terms of the degree to which they would recommend them (using a 4-point scale) for their interpersonal skills (pre=2.84 SD .58 vs. post=3.09 SD .41; p=.066) and for their medical expertise (pre=2.90 SD .48 vs. post=3.19 SD .29; p=.014).


53%

70%

49%

32%

65%

74%

60%57%

0%

10%

20%

30%

40%

50%

60%

70%

80%

OVERALL COMMUNICATION

Information Gathering Relationship Development

Patient Education

% W

ell D

one

Impact of EMPACT: Pre-Curriculum vs. Post-Curriculum OSCE Communication Scores (n=15)

Pre Postp<.01

p<.05p<.001


54% 54%

44%

39%

54%

67%

53% 53%

73%

59%

0%

10%

20%

30%

40%

50%

60%

70%

80%

Bad News Interpreter Broken Wrist (Medical Error)

Repeat Visitor Transfer

% W

ell D

one

Impact of EMPACT:Pre-Curriculum vs. Post-Curriculum OSCE Case Specific Scores (n=15)

Pre Postp<.10

p<.001


D2. Comparison of OSCE and USP Scores A major goal of this project was to begin to explore how residents’ performance in an OSCE

relates to their actual clinical performance, at least as assessed by an USP. Given that the pre-OSCE took place in July, the post in March, and the USP visits anytime between mid-January and early April, scores generated from the USP visits were compared with both pre- and post-curriculum OSCE scores. Although, we expected USP scores to be more highly correlated with post-OSCE scores since they generally occurred closer in time. Twelve residents had at least one USP visit and 5 residents were visited by both USPs (Repeat Visitor and Medical Error). We report correlations for both sets of data in order to maximize our sample size (including all 12 residents by reporting whatever USP data is available for each resident be it one or two visits) and maximize our sample of actual clinical performance (including only those 5 residents from whom we have two samples of performance data, i.e., two USP visits).

Correlations between OSCE and USP Scores

At least 1 USP Visit (n=12) 2 USP Visits (n=5) USP Scores Pre OSCE Post OSCE Pre OSCE Post OSCE

Overall Communication

Skills

.70 (p=.011)

.17 (p=.600)

.83 (p=.088)

.53 (p=.379)

Overall Case Specific Skills

.63 (p=.029)

.17 (p=.598)

.64 (p=.249)

.85 (p=.066)

2.84 2.90

3.093.19

1

2

3

4

Recommendation - Interpersonal Skills Recommendation - Applic of Expertise

Impact of EMPACT:Pre-Curriculum vs. Post-Curriculum Recommendation Ratings (n=15)

Pre Post

p<.10 p<.01

Highly Recommend

Recommend

Recommendw Reservations

Not Recommend


Results suggest that the USP scores are strongly correlated with the pre-OSCE scores for both residents with one or more visits and for only those residents with an adequate sample of clinical performance (both Repeat Visitor and Medical Error USP visits). However, it is only among those with both USP visits that we see strong correlations with post OSCE scores. It may be that residents’ performance on the pre-curriculum OSCE best represents how they are in actual clinical practice while their performance on the post-curriculum OSCE was more reflective of how they perform when being evaluated on the basis of clear criteria (as shared through the 5-session curriculum). These exploratory results also demonstrate the importance of including multiple samples of performance – one USP visit is probably not sufficient to obtain a true and accurate picture of physician skills.

We assessed two additional dimensions of clinical performance: patient-centeredness (e.g., fully explored my experience of the problem, took a personal interest in me, earned my trust, acknowledge impact of situation on my life) and the degree to which the resident “activated the patient” (e.g., helped me to understand the nature and causes of my condition, helped me find out about the different medical treatment options available, made me feel confident I can figure out new solutions if my situation changes) (Hibbard ref). There is increasing evidence that these skills, along with core communication and case-specific skills, are associated with important patient outcomes. Therefore, we examined correlations between average scores residents received from USPs on these items and their OSCE scores and found, as above, that both pre and post OSCE communication and case-specific skills were strongly (albeit not significantly) and positively correlated with patient centeredness and patient activation.

2 USP Visits (n=5)

Overall Communication Skills Overall Case Specific Skills

USP Scores

Pre OSCE Post OSCE Pre OSCE Post OSCE Patient

Centeredness.56

(p=.326) .78

(p=.120) .79

(p=.112) .84

(p=.078) Patient

Activation.68

(p=.202) .60

(p=.282) .85

(p=.070) .84

(p=.078)

E. DISCUSSION

There are many things we can learn from the development and implementation of a new

curriculum designed to help residents with their communication skills. Even having taught communication skills in other disciplines, teaching the same skills in EM provided rich learning opportunities for us as curriculum innovators, evaluators, and administrators

First, residents portray an outward confidence about their communication skills, which lacked grounding in their assessment levels. Despite their relaxed attitude about the OSCE cases, the data showed that they had difficulty with some of the scenarios. This came as a great surprise to some, though the majority already knew there was some deficiency when questioned. Resident reported they learned that: 1) without listening to what patients have to say about their condition, it is difficult to hear what the patient is actually trying to convey, without appropriately providing patient education, quality of care may be compromised, 2) without communicating effectively with other disciplines, it will be difficulty to coordinate care, and 3) without demonstrating empathy, kindness, patient satisfaction is hard to achieve. Having the opportunity to step back from the flurry


of activities in the EM, residents were able to acknowledge their respective shortcomings in communication skills and commit to improving them for their patients.

Second, residents received their feedback in a much more affirmative manner than we had hoped. We are struck by their positive feedback for the “much needed” education on “basic skills” that are essential for success as EM physicians. Their enthusiasm for this education is surprising and gladly received. They have been instructive in helping us to design our curriculum so that they can get the most out of the experience for their practical day-to-day use.

Third, as measured by a reliable and valid OSCE, the EMPACT project shows that a focused curriculum, with five one-hour group interactive sessions on communications and professional curriculum, can significantly improve residents’ rapport building and patient education skills. These skills were tested months after the curriculum. Our curriculum is unique, not for its topics, but because of the variety of educational methods we incorporated (i.e. role play, modeling with standardized patients, discussion triggered by “TV medical clip” and reenactments of real residents’ performances). This approach is highly acceptable and engaging to residents, as evidenced by their feedback.

Fourth, through the USP aspect of this project, a novel endeavor, we have shown that this methodology is feasible and acceptable to residents, program directors, and faculty and hospital administrations. As noted by the program director, this project has already brought added value to the resident learning and patient care. By informing the residents that USPs would be visiting them in the ED, the residents seemed to perform at a higher level, not knowing which patients might be evaluating their performance and what measures were being evaluated. One resident commented that when he thought a patient was a USP, he washed his hands more frequently, thinking that hand washing was the metric we were evaluating. A faculty member noted that when one resident thought he had identified a USP, he seemed more empathic and professional when discussing the discharge plan and follow-up care. Clearly, the patients also benefited from the study, as higher professional standards, including stricter adherence to Joint Commission Safety Initiatives were being executed by the residents to more patients, not only the USPs.

We must further analyze our USP results, debriefing tapes, and audio tapes to understand what additional information we can learn about our residents’ skills using this innovative methodology. The fact that our post-OSCE results did not fully match the residents’ USP encounters further supports the need to perform larger USP studies with multiple cases in order to better understand the degree to which OSCEs reflect real world skills. It is our hope that we can in what ways OSCEs can predict real life performance in order to enable us as educators to use them as efficient and effective tools to help learners become expert physicians.

With the ACGME recently placing greater importance on evaluation of patient outcomes and its linkage to medical education, we believe that our project is representative of a new way to assess real-time resident physician performance. As program evaluators working toward enhancement of curricula that better meet patient needs, this project has contributed much to our larger efforts. The data collected from these OSCEs have been incorporated into Database for Research on Education Academic Medicine (DREAM), an initiative of our Research on Medical Education Outcomes Unit (ROMEO), which enables long-term, longitudinal assessments of participant performance both in residency and beyond. Further comparison of OSCE evaluations with USP encounters will enable educators to determine whether or not these commonly used evaluation tools actually mimic real practice. The current OSCE data will be assessed in conjunction with future evaluations and patient outcomes. We eagerly await results of a larger trial.

Lastly, this collaboration between NYUSOM Primary Care and Emergency Medicine has enabled us to further heighten the overall abilities of NYUSOM faculty to teach and communicate with each other and to our residents. Additionally, we believe this curriculum also provided an added


value as a faculty development opportunity. Faculty members in the Emergency Department have gained a standardized approach to teaching and assessing communications skills after participating or playing facilitative roles in the curriculum.

F. DISSEMINATION

We have already begun to share our methods with other departments and institutions.

Owing to the success of the EMPACT OSCE, the Gastroenterology fellowship used our cases for their OSCE held on October 6, 2007. Their use of our communication skills checklist will enable us to compare performance across disciplines and levels of training. They are planning a second OSCE for additional fellows in May 2008. Additionally, current plans are under way within the Department of Emergency Medicine at Johns Hopkins to apply for funding to support the use of USPs in evaluation of curriculum focusing on disaster education.

In terms of publication, the Arnold P. Gold Foundation, which promotes and affirms more compassionate medical care and caregivers, accepted our abstract (“A Curriculum in Patient-Centeredness for Surgery and Emergency Medicine Residents: Establishing the Baseline.” M. Hochberg, S. Zabar, L. Regan, R. Laponis, R. Richter, A.L. Kalet), for presentation at the Gold Foundation Symposium, How Are We Teaching Humanism in Medicine and What is Working?, which was held on September 27-29, 2007, Chicago, IL. Future plans include submission to Academic Emergency Medicine, the journal of the Society of Academic Emergency Medicine as well as to the national Council of Residency Directors (CORD) meeting for Emergency Medicine which is held annually. G. FINANCIAL REPORT The Financial Report will be provided by the NYUSOM Sponsored Programs Administration under separate cover.


H. ATTACHMENTS a. Sample Case and Checklist b. Sample Report Card c. Session Objectives d. Sample Pocket Card e. Sample Feedback f. Dissemination

i. Gold Foundation Abstract

EM RESIDENT TRAINING IN INTERPERSONAL AND PROFESSIONALISM SKILLS (Progress Report) Page 18 PI: Sondra Zabar, MD NYU School of Medicine

Attachment – Sample Case and Checklist (Medical Error)

STATION OVERVIEW

OBJECTIVES To test the resident’s ability to:

1. Admit an error has been made 2. Be empathic 3. Address patient concerns surrounding an error

LOGISTICS Personnel:

Standardized patient, male, 32 y.o., dressed in regular clothing, sitting in chair.

Station Materials:

• Resident instructions • SP Instructions • SP evaluation forms • Faculty evaluation forms

Room Arrangement:

• Station signs • Chair (2) • Exam table


RESIDENT INSTRUCTIONS

Name: John McCoy

Age: 32

PATIENT

INFORMATION

REASON FOR ENCOUNTER

• John McCoy came to the ER 2 days ago complaining of right wrist pain after falling while rollerblading near Washington Square Park.

• At that time, his hand x-ray was MISREAD by a resident as normal and he was sent home with an Ace bandage and some ibuprofen.

• The Radiology Attending re-read the x-ray and found a non-displaced, non-intra-articular right distal radius fracture.

• He presents today to the ER after having been called back.

YOUR ROLE ER Resident

YOUR TASKS

1) See the patient, explain what has occurred, and develop a plan.


STANDARDIZED PATIENT INSTRUCTIONS

THE SCENARIO Your name is John McCoy and you are 32 years old. 2 days ago you were rollerblading in Washington Square Park prior to when your shift started for work at a restaurant (you work as a waiter at the Union Square Cafe). You fell and hit your outstretched right hand on the pavement. Your right wrist hurt a lot and you were afraid that it might have been broken. This was particularly concerning as you work as a jazz pianist occasionally. You went to the Emergency Room and after waiting for 4 hours, finally saw a doctor. They took some x-rays and told you it was just a sprain. You got some pain drugs (ibuprofen) and a bandage to wrap your wrist. You were told to rest your wrist, use ice, and keep it wrapped and raised as much as possible. Because of the wait at the ER, you had to have someone cover for you at work. Because you don’t get sick pay, you decided to work yesterday even though you were in pain. This morning, you got a call from a nurse instructing you to return to the ER as the doctors had some information about your wrist. You again got someone to cover for you (although you still won’t get paid) in order to go back to the ER today. Today, the pain in your right wrist is about 5/10 (10 being the worst pain in your life) and it only gets worse when you bend it back or press on it. The swelling has gone down from 2 days ago and it seems like it is slowly getting better, despite having used it yesterday at work. Objective: • To understand what has occurred and know when

you can return to work Obstacles: • You are upset about missing work as you are

having a tough time making ends meet.

CHARACTER DESCIRPTION

Tactics: You are initially somewhat agitated as you are missing work again When you hear the news of the mistake you become further agitated If the resident is empathic, apologizes, and is helpful, you calm down a little. If, however, the resident is at all defensive, argumentative or unhelpful, then your agitation continues to increase.


SINCE YOU

LEFT THE ER

Since you left the ER 2 days ago, you have been trying to do what the doctor told you to do: rest it, use ice, compress it with the bandage and keep it elevated. You did, however, go to work yesterday after taking a few ibuprofen (Advil) tablets and a strong gin and tonic in order to minimize the pain. You got thru your shift without too much trouble and were able to compensate using your left hand more often than usual. Today, you still have some pain, but the ibuprofen is helping.

PERSONALITY You tend to be a little dramatic. When you are happy, you border on gushy and when you are upset, you can get angry. This is partly due to the fact that your financial situation is slightly unstable and it can put you on edge at times.

CURRENT LIFE SITUATION

You live with a roommate in the East Village. You have no children. You work as a waiter at the Union Square Cafe and play jazz piano intermittently with various local groups. You are still hoping to make it as a pianist, but it hasn’t worked out that well so far.

PAST MEDICAL AND SURGICAL

HISTORY

None. You are otherwise very healthy and active.

FAMILY

HISTORY

Your mother and father are both living in Ohio. They are healthy as far as you know. You have one brother who is healthy and married living in Ohio as well.

SOCIAL

HISTORY

You smoke ½ pack a day for the past 10 years. You drink alcohol at least 3 times per week, usually having 2-3 drinks each time. You do not use recreational drugs. You are sexually active with a girlfriend you have had for the past 6 months. You use condoms for protection. You are eating and sleeping well and staying active by rollerblading and going to the gym occasionally.

MEDICATIONS

ALLERGIES

Ibuprofen (Advil) – 2 tablets every 4 hours for pain None

THE

ENCOUNTER

When the Resident knocks and enters the room, you are sitting in a chair in the exam room talking with a colleague trying to get someone to cover for you as you are missing work. You are upset interrupting


the person on the other end of the phone line and end the conversation about 20-25 seconds after the resident enters the room. When you hang up, you are still upset having had to miss work for the second time this week. You show this by making eye contact with the resident, occasionally breathing deeply and audibly, and have aggravated tone to your voice. You are testy and confrontational the entire interview and occasionally interrupt the resident to voice your frustration. If asked in an open-ended way why you are here, state: “You guys called me. I was here a couple days ago about my wrist, so I assume it’s about that.” With respect to your wrist- Any pain? – “A little, but the Advil helps.” How bad is the pain? – “About 5 out of 10” Any pain with movement? – “Only when I bend it back” Any swelling? – “It’s gotten a lot better.” Any tingling or loss of sensation? – “No” Any redness? – “No” Any tenderness? – “It hurts a little when I push on it.” In general currently: How have you been? – “Fine, I guess. My wrist hurt a bit during work yesterday, but I got through it. But I’ve missed two days because of this stupid thing.” If/when you are told a mistake was made (i.e. someone read the x-ray of your wrist incorrectly and you actually have a bone fracture) regardless of where it occurs in the interview, take a moment to let it set in and then at first become upset. Raise your voice, but do not shout, look the Resident straight in the eye, and impatiently tap your finger on the desk or table to underline your frustration. State: “So my wrist is broken?” “This is so annoying.” “I mean, what’s going on here? I had to miss two days of work because of this.” If then the Resident acknowledges the mistake, states that he/she is sorry that it happened/empathizes, you still remain angry and state in a slightly aggressive tone: “Oh man. I knew it. I knew it was something bad. This


always happens to me. Well, will there be any long-term damage?” When you realize the long term damage will be nil or minimal, you are only a little relieved. State in a somewhat frustrated way: “Why did this happen? What if this was something really serious? I mean, my God, does this happen all the time?” Whatever the resident’s response is state: “Well, don’t you think this is a bad system here?” If the Resident remains apologetic and non-confrontational, you calm down a little and ask: “Well, when can I go back to work?” If the Resident acknowledges that a mistake was made, but then becomes defensive, does not empathize or say he/she is sorry, or makes up a bizarre story -> get more upset: “I mean, me missing work today would have been totally unnecessary right? If you guys actually did your job, I wouldn’t have had to come down here.” “I knew I shouldn’t have come to his ER.” If the resident asks if they can write you a note, state sarcastically: “A note? What I am I going to do with a note?” Whenever the Resident changes course and becomes more apologetic/empathic, react accordingly. Adequately challenge the resident. You are upset for a multitude of reasons: losing work pay, being in pain, losing faith in your health care provider, and not being able to play piano. If you feel the resident is making a genuine effort to address your concerns, is empathic and non-confrontational, become less angry, but maintain a baseline of annoyance and frustration. If the resident ever becomes dismissive/confrontational or you don’t feel supported, become more upset. Towards the end of the interview, regardless of the Resident’s reactions, become calm. Your motivation for doing this is as follows: If the Resident has admitted the mistake and acted appropriately, you are satisfied. If the Resident has done poorly by not admitting the mistake or making fabrications you become withdrawn contemplating a lawsuit: (Please note: Do not mention lawsuit, litigation, suing, or anything relating to malpractice unless the Resident brings it up - this is purely an internal cue for you to help you act out the character). If the latter is the case – partially cross


your arms, rest your head on one hand, and avoid eye contact. Once you have calmed down a little, state: “Well, I came all the way down here. Now what?”

CHALLENGES FOR THE

RESIDENT

• Admit that an error was made

• Regain patient trust

Non-verbal 1 At the beginning of the interview, eye contact with occasional audible breathing.

CUES FOR THE

RESIDENT

Verbal 2: State: Why exactly was I called back? -> Resident to verbally acknowledge your concern and explain reason

Verbal-Non-Verbal 3:

Express anger (state that you are upset, raise your voice, look at the Resident in angry and accusatory fashion, underline your verbal comment with tapping your fingers on the table) -> Resident to verbally acknowledge your anger/being upset and label it as understandable

Verbal-Non-Verbal 4:

Calm down in last part of encounter; if Resident acted appropriately: calm down (e.g., appear more relaxed in your posture and voice); if Resident acted inappropriately: withdraw (e.g., cross arms, speak in short sentences, etc). State: “Well, I’m here. What do we do now?”

Initially: You are already a little upset.

TIMING

Ongoing: If the Resident is empathic/truthful/straightforward, become more and more calm. If the Resident is defensive/evasive/making up bizarre stories, become more and more upset.

2 minute warning: Begin to calm down because the Resident is acting appropriately or withdraw because the Resident is acting inappropriately. State: “What do we do now?”


Evaluator’s Checklist COMMUNICATION Not Done Partially Done Well Done

Information Gathering

Elicited your responses using appropriate questions:

No leading questions Only one question at a time

Impeded story by asking leading/judgmental questions

AND more than one question at a time

Used leading/judgmental questions OR asked more than

one question at a time

Asked questions one at a time without leading patient in their

responses

Clarified information by repeating to make sure he/she understood you on an ongoing basis

Did not clarify (did not repeat back to you the information you

provided)

Repeated information you provided but did not give you a chance to indicate if accurate

Repeated information and directly invited you to indicate

whether accurate

Allowed you to talk without interrupting Interrupted Did not interrupt directly BUT cut

responses short by not giving enough time

Did not interrupt AND allowed time to express thoughts fully

Relationship Development Communicated concern or intention to help

Did not communicate intention to help/concern via words or

actions Words OR actions conveyed

intention to help/concern Actions AND words conveyed

intention to help/concern

Non-verbal behavior enriched communication (e.g., eye contact, posture)

Non-verbal behavior was negative OR interfered with

communication Non-verbal behavior

demonstrated attentiveness Non-verbal behavior facilitated

effective communication

Acknowledged emotions/feelings appropriately

DID NOT acknowledge emotions/feelings Acknowledged emotions/feelings

Acknowledged & responded to emotions/feelings in ways that

made you feel better

Was accepting/non-judgmental Made judgmental comments OR facial expressions

Did not express judgment but did not demonstrate respect

Made comments and expressions that demonstrated

respect

Used words you understood and/or explained jargon

Consistently used jargon WITHOUT further explanation

Sometimes used jargon AND did not explain it

Explained jargon when used, OR avoided jargon completely

Education and Counseling Asked questions to see what you understood

Did not check for understanding Asked if patient had any

questions BUT did not check for understanding

Assessed understanding by checking in throughout the

encounter

Provided clear explanations/information Gave confusing OR no

explanations which made it impossible to understand

information

Information was somewhat clear BUT still led to some difficulty in

understanding

Provided small bits of information at a time AND summarized to

ensure understanding

Collaborated with you in identifying possible next steps/plan

Told patient next steps/plan Told patient next steps THEN asked patient’s views

Told patient options, THEN mutually developed a plan of

action


ADDRESSING MEDICAL ERROR

Accountability

Disclosed error • Direct (used the words “error” or

“mistake”) • Prompt disclosure

Did not directly disclose the error (there was a “problem”) NOR was the explanation upfront

Did not directly disclose the error (there was a “problem”) OR directly disclosed late in the

interview Directly disclosed the error upfront

Personally apologized for the error (“I am sorry that this happened)

Did not apologize for error NOR for the inconvenience it caused

you Apologized for the error OR for

the inconvenience it caused you Apologized for the error AND for the inconvenience it caused you

Shared the cause of the error (i.e., Explained issues with system)

Did not acknowledge issues with system

Acknowledged issue with system BUT was dismissive/

condescending Acknowledged issue with system AND was genuine in addressing it

Took responsibility for situation Took no personal responsibility for your present situation (e.g., assigns your problem to other

person/department)

Took a general responsibility as part of the department for your

present situation Took a personal responsibility for

your situation (“I will…)

Identified future preventative strategies to prevent situation from happening again

Did not address how situation would be prevented in future

Made general suggestion for improvement (e.g., “We’ll look into it,” “I’ll make a note of it to

my Attending”)

Offered specific strategies for potential improvement of system

Managing a Difficult Situation

Avoided assigning blame Became defensive/

argumentative AND assigned blame to a person/department

Became defensive/ argumentative OR assigned

blame to a person/department Remained calm AND did not mention blame someone else

Maintained professionalism by controlling emotions

Unable to control emotions, became dismissive and

condescending

Attempted to control emotions (e.g. was somewhat dismissive

or condescending)

Maintained a high level of professionalism in handling your specific situation, did not show

anger or frustration

Delivering Bad News

Prepared you to receive the news: • Entered room prepared to deliver news • Ensured sufficient time and privacy

Entered room in a manner unfitting the news AND

physically situated him/herself far from you

Entered room in a manner unfitting the news OR physically situated him/herself far from you

Entered room in a manner befitting the news AND physically situated

him/herself close to you

Assessed your readiness to receive news: • Gave warning shot (e.g., “I have

some good and bad news for you…”) No warning shot

Attempted to deliver warning shot, BUT inappropriately (does not pause for your assent OR

warning shot too long)

Gave you a well-timed warning shot

Gave you opportunity to emotionally respond: • Remained sensitive to your venting of

shock/anger/disbelief/accusations • Attended to emotions before moving on

Responded inappropriately to your emotional reaction (no

opportunity to vent, cut you off, became defensive)

Allowed you to emotionally respond (vent) BUT did not

address/acknowledge response before moving on

Allowed you to express your feelings, fully giving you the feeling you were being listened to before

moving on

Directly asked what you are feeling: “What are you thinking/feeling?”

Did not ask specifically “What are you thinking/feeling?”

Acknowledged your feelings (e.g., “I see that you are upset…”) BUT did not

specifically ask you to name your emotions

Specifically asked you “What are you thinking/feeling?”

Provided appropriate “next steps” • Orthopedics for immediate care • What to expect long-term

Did not offer next steps AND evaded response as to what will

happen long-term

Offered only general next steps (e.g., I’ll be calling Ortho) OR

promised to “ask the attending” for next steps

Offered specific next steps (e.g. Orthopedics is going to fit you for a

cast) AND informed you of long term care needs (e.g., unable to

use arm for 6 weeks)


Would you recommend this doctor to a friend for his/her interpersonal skills?

Not Recommend Recommend with

Reservation Recommend Highly Recommend

Would you recommend this doctor to a friend for his/her medical competence?

Not Recommend Non -exemplary Physician:

superficial, artificial demeanor applied knowledge base inadequate to my

situation

Recommend with Reservation Unexceptional

Physician: awkward, knowledge base only

somewhat apparent in application to my situation

Recommend Satisfactory Physician:

appropriate knowledge base applied adequately to my specific situation

Highly Recommend Model Physician:

sophisticated, wise, thoughtful, applied profound knowledge base specifically to

my situation

COMMENTS:


Attachment – Sample Report Card

EMPACT OSCE Report of Results – July 2007

Clinical skills were assessed in 5 cases. Your scores in 5 core areas – communication scores, overall recommendation scores, ratings of ability to apply expertise, specific skills across cases, and overall case-specific skill scores -- are reported in the charts that follow. For case-specific skills and recommendation ratings, results for each case are included as well. One case was not reliably scored (Unexpected Death) and so scores associated with that case should be interpreted with caution. Overall communication score: Calculated across all cases as the % of behaviorally-anchored

communication items (8-14 items per case) for which you were rated as having performed well (“done well”). Sub-domains include: Information gathering, relationship development, and patient education.

Overall recommendation rating: Calculated across all cases on the basis of rating of degree to which “would recommend physician to a friend based on his/her communication skills” with the following response options: Not Recommend – Recommend with Reservations – Recommend – Highly Recommend.

Overall rating of application of expertise: Calculated across all cases on the basis of rating of degree to which applied expertise effectively, using a 4-pt scale: Insufficient Application, Slight Application, Sufficient Application, Exceptional Application of Expertise.

Selected skills across cases: Calculated as the % of items rated as well done for specific skills measured across at least several cases including: delivering bad news, managing difficult situations, accountability, handling emotions.

Overall case-specific skills: Calculated across all cases as the % of items rated as well done for core knowledge and skill items specific to each case.


Communication Scores for Sample Student

51% 52%

61%

33%

64%

50%56%

27%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Your Scores Class Mean

% W

ell D

one

OVERALL COMMUNICATION SCORE

Communication - Information Gathering

Communication - Relationship Development

Communication - Patient Education

Error Bars: +/- 1 Std Dev


Overall Recommendation Rating for Sample Student

2.75

3.35

Une

xpec

ted

Dea

th*

Info

rmed

Con

sent

Rep

eat V

isit

Tran

sfer

of C

are

X-R

ay R

ecal

l

1

2

3

4


Recommendation Ratings for Each Case

Error Bars: +/- 1 Std DevHighly Recommend

Recommend

Recommend with

Reservation

Not Recommend OVERALL

RECOMMENDATION *Unreliable Case - Interpret w/ Caution


Overall Rating of Application of Expertise for Sample Student

2.00

2.84 Une

xpec

ted

Dea

th*

Info

rmed

Con

sent

Info

rmed

Con

sent

Rep

eat V

isit

Rep

eat V

isit

Tran

sfer

of C

are

Tran

sfer

of C

are

X-R

ay R

ecal

lX-

Ray

Rec

all

1

2

3

4


Ratings for Each Case


Exceptional Application of Expertise

Sufficient Application

Slight Application of Expertise

Insufficient Application OVERALL RATING

APPLICATION OF EXPERTISE*Unreliable Case - Interpret w/ Caution


Case-Specific Skills for Sample Student

49%48%

Une

xpec

ted

Dea

th 6

4%*

Info

rmed

Con

sent

70%

Rep

eat V

isit

50%

Tran

sfer

of C

are

86%

X-R

ay R

ecal

l 67%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%


OVERALL CASE-SPECIFIC KNOWLEDGE SKILLS

Rating of Knowledge Skillsfor Each Case

Error Bars: +/- 1 Std

*Unreliable Case - Interpret w/ Caution


Skills Across Cases for Sample Student

47.1%

66.5%

27.5%

46.3%

38%

55%50%

57%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%


% W

ell D

one

Delivering Bad NewsManaging Difficult

SituationsAccountability Handling Emotion



Attachment – Session Objectives Session Goal Cognitive Objectives Skills Objectives Affective Objectives 1. Making Every Session Count: Effective Communication Skills in the Emergency Room

Provide residents with tools to maximize the effectiveness of their communication with patients and their families

Appraise and interpret data from the EMPACT performance assessment of the professionalism skills of emergency medicine residents.

Identify the major communication challenges of emergency medicine practice (e.g. explaining frightening bio-technical procedures, talking about risks and consequences, counseling acutely anxious individuals).

Describe an evidence-based approach to efficient therapeutic relationship building.

Critique a videotaped interview of a Emergency Medicine resident talking with a patient who makes frequent trips to the ER for back pain.

Support a patient centered communication model which requires the physician to elicit the patient’s story, accurately name and appropriately respond to patient’s emotions and ensure effective patient education.

Commit to developing expertise through a process of “deliberative practice” in the realm of professionalism which requires honest assessment of communication skills.

2. Interdisciplinary Communication and Respect

Teach residents to effectively work with the professionals around them to optimize patient care.

Name important stages of conflict management

Explain the role of interdisciplinary respect in the quality of patient care.

Demonstrate effective interdisciplinary communication skills

Demonstrate effective telephone communication skills

Identify your own personality traits which may add to or detract from these interactions.

3. Delivering Bad News in the Emergency Department

Improve residents’ effectiveness in their delivery of bad news and provide residents with facts about post-death procedures.

Know the basic skills of delivering bad news

Demonstrate the ability to deliver bad news

Demonstrate recognition and appropriate responses to patients’ emotional reactions to bad news

Acknowledge patients’ strong emotional reactions as appropriate and healthy responses

Understand that delivering bad news is difficult for both the patient and doctor and may influences the doctor patient relationship.

4. Dealing with Culturally Diverse Patients in the Emergency Department

Understand appropriate use of interpreters in the ED and improve residents’ interactions with culturally diverse patients in the ED

Know the different types of interpreters

Know the basic skills for using an interpreter

Demonstrate the ability to use an interpreter

Recognize the difference between physician and patient perceived need for using an interpreter

1. Develop self awareness about one’s owns cultural influences and biases

5. Medical Errors in the Emergency Department

Improve resident’s effectiveness in their disclosure of medical errors

Know the basic skills of disclosing medical errors

Understand barriers to disclosure of errors in the ED.

Demonstrate the ability to disclose a medical error

Demonstrate recognizing and responding to patients emotional reactions

Recognize that a patients strong emotional reaction is an appropriate and healthy response


Attachment – Sample Pocket Card INTERDISCIPLINARY COMMUNICATION AND RESPECT

Working with fellow professionals to optimize patient care

Beginning a Dialogue: • Seek to achieve understanding and clarify meaning • Use Brainstorming to solicit ideas for problem-solving • Understand the other point of view

Enter into a Discussion: • To make a decision • To reach closure • Discussion is NOT a debate • Discussion is NOT about winning • Focuses on issues, not people

Negotiating Agreement: • Focus on INTERESTS rather than POSITIONS • Ask • Discuss

Conflict between disciplines results from ignorance, distrust, lack of confidence, and poor communication. Fisher and Ury. Getting to Yes: Negotiating Agreement Without Giving In. New York: Penguin Books, 1991.

Talking to other services: Telephone communication is difficult due to the following factors: • Limited education • Limited feedback • Loss of body language • Background noise • Timing

Important Telephone skills: • Identify your name and role • Clarify who you are talking to • Clarify information and your understanding • Listen for tone of conversation • Clearly acknowledge emotions • Summarize next steps

Communication of concern and understanding: • Use Reflective Statements

(i.e. it sounds like…) • Express Partnership

(i.e. we need to…..) • Communicate Respect

(i.e. you have really done everything…) • Behaviors that instill confidence: • Speak with a firm ,clear and calm voice • Ask and respond to concerns and questions • Summarize essence of problem


Attachment – Gold Foundation Abstract Abstract Title: A CURRICULUM IN PATIENT-CENTEREDNESS FOR SURGERY AND EMERGENCY MEDICINE RESIDENTS: ESTABLISHING THE BASELINE Author: Hochberg M, Zabar S, Regan L, Laponis R, Richter R, Kalet A Goal/problem statement: Surgical and acute care settings are increasingly challenging environments to provide humanistic care. Both logistic and cultural factors at the resident level of training generate significant barriers to assessing, developing and maintaining patient-centered attitudes and skills. Typically, curricula dedicated to teaching and enhancing these skills are generic, rare and not endorsed by leadership. This project was designed to develop patient-centered attitudes and skills among Surgery and Emergency Medicine residents using a multi-modal, interdisciplinary, skills-based curriculum and to assess skill acquisition using an Objective Structured Clinical Exam (OSCE) and unannounced standardized patients. Description of project or intervention: 27 Surgery Residents and 15 Emergency Medicine Residents at an academic medical center will complete 6 faculty-led one hour seminars designed to enhance attitudes and skills in the following competencies: informed consent, delivering bad news, disclosing errors, interdisciplinary respect during transfer of care, cultural sensitivity and self-reflection. Residents will be provided with a course manual including specific goals and objectives for each session, select specialty-specific literature on the seminar topic, self-reflective exercises and quick reference pocket cards. Seminar teaching modalities will include videotape review, standardized patient (SP) interviews, role-playing, group discussion, and expert didactics. Emphasis will be on understanding a conceptual model and developing specific skills applicable to interactions with colleagues, patients and patients’ families. Evidence of effectiveness: Residents attitudes will be assessed by a focused, literature-based survey. Skill level in the above competencies will be measured by a multi-station OSCE. Behaviorally-anchored checklists are designed to assess skills across 5 domains: information gathering, relationship building, shared decision making, accountability, and respect. The scenarios will assess resident capability in performing complicated professional tasks such as obtaining informed consent through an interpreter for a patient with a non-Western health belief model, negotiating end of life care for a patient who’s next of kin’s wishes are at odds with that of the patient’s, disclosing a medical error, recognizing colleague impairment, and delivering news of an unexpected death to the deceased’s spouse. The same OSCE will be administered at the end of the curriculum and individual performance data will be paired for pre and post curriculum comparison. Transfer of these skills to the bedside will be assessed via unannounced SPs who will be placed in actual clinical settings in scenarios similar to that of the OSCE and linked to individual performance data to assess OSCE score validity and resident capability in real clinical encounters. This approach is the next step in linking resident behaviors to patient outcomes. Discussion and conclusions: Although our project will not yet be ongoing for a year by the end of September, we will present detail rich data from our baseline attitudes survey and the pre-curriculum 6 station OSCE. We will also share our curriculum manual, seminar materials and checklists from the OSCE and unannounced SP cases.

zabar final report cg

Documents