Conducting Your Program’s Annual Review

Esther Tucci, CTAGMEPinnacle HealthHarrisburg, PA

About Pinnacle Health GME Internal Medicine Residency – 35 General Surgery Residency

MD – 13 DO – 6 Toxicology Fellowship - 1 Bariactric Surgery Fellowship – 1 Osteopathic Programs

FP – 3 IM – 3 Orthopedics – 12 – Intern - 12

What is an Annual Review?

RRC of Surgery Program Requirements VI.C.1.2.3 – Program Evaluation

“The educational effectiveness of a program must be evaluated at least annually in a systematic manner”

Annual Review vs. Internal Review Annual Program/Common

Requirements Done Annually Conducted by program Program

representatives Plan of action approved

by faculty and documented in minutes

Internal Institutional Requirements Done midpoint between

ACGME accreditation surveys

Conducted by GMEC External representatives A written report by the

GMEC and required to give to ACGME site visitor for Institutional Reviews

Definition of Annual Review Internal program personnel, i.e. PD,

Faculty and at least one resident Review program goals and objectives

and the effectiveness of the program Must have documentation of the

meeting at least annually Must evaluate written comments from

the faculty, most recent GMEC report, residents confidential written evaluations

Definition cont’d…. Use resident performance and outcome

assessment in its evaluation of the educational effectiveness of the residency program

Program should have a process in place to use these assessment results to improve the residency program

If deficiencies are found the group should prepare a plan of action approved by faculty and documented in the minutes

WHAT WE DID………..

Formed Our “group”

PD All paid faculty Two residents Invite all volunteer faculty – those

that responded attended Administrative/DIO (VP of Medical

Affairs)

Location and Time

Did survey on preference of date and time

Off campus ½ day time commitment –

(breakfast/lunch) 730 am to 1200 noon

Close to the hospital

Evaluation Sources Used in the Review Faculty Eval of Prog Resident Eval of Prog Resident Eval of

Faculty Resident Eval of

Services ACGME Resident

Survey Summary

Duty Hour Reports Medical Student

Clerkship Evaluations M&M Conference Evals Last GMEC Internal

Review Last RRC Site Visit

Results Last Institutional

Review

Evaluation Sources Used in the Review

Adequacy of clinical experience: 5-year summary of ABS Exam Results Resident ABSITE Score Results Graduate Surveys Resident SOL data Resident bedside procedures

Evaluation Sources Used in the Review

ACGME Competency Assessment Surgery Competency Charts

Pre-meeting Work Assignments

Each member of the “group” received the following pre-meeting work assignments: Copies of all evaluation sources for

their review S.W.O.T Form Faculty Development Survey Items of Discussion List

S.W.O.T Form

Please list what you feel are the residency program’s…. “S”trengths “W”eaknesses “O”pportunities “T”hreats

Needs Assessment for Faculty Development

Each member was asked to list their interest in learning more about the following topics: Teaching Evaluation Technology Research Career Development Rate his/her preference for session times

Items of Discussion List Need to develop this list to fit your own

program needs Examples of some items for Pinnacle:

Incorporating competencies into goals/objectives Improving standards for our Selection Policy and

criteria Enforcing higher standards for Promotion and

Resident Evaluation Policy Recruitment of FT Faculty On-line curriculum for ABSITE review New Academic Curriculum 3-chief year July 06 – June 07

Working Binder Each member of the group received a

binder ahead of time that included the following: Pre-work assignments Evaluation sources Copies of Program Requirements Current Rotation Goals & Objectives Program Policies/GMEC Policies Definition of Core Faculty

Each member was instructed to bring this binder with them to the meeting

At the Meeting

Agenda S.W.O.T form results Review items off Discussion List Documentation and Action Plan

Based on the deficiencies or outcomes from first two items

New Initiatives

Meeting Documentation

DISCUSSION And RRC REQUIREMENT (**)

ACTION PLAN/GOAL

Full Time Faculty numbersFaculty manpower needs, funding for faculty

** In addition to the PD for each approved chief residency position, there is a least one additional geographic full time teaching staff member whose major function is to support the residency program.

Pursue Critical Care Surgeon for full time faculty. Will no longer pursue a Trauma Surgeon. Need to know from Administration the funds available for recruiting and salary. Find out what the program’s reimbursement amount is and how the money is used. Need support from Administration for aggressive recruitment to meet this need.

Meeting Documentation

DISCUSSION And RRC REQUIREMENT (**)

ACTION PLAN/GOAL

Lack of Trauma Protocol for HHLack of trauma alerts coming to HH ERSome senior and all junior residents will notmeet their numbers in trauma due to the changeof no trauma alerts coming to HH ER

** Residents are required to have 10 operative trauma cases and 20 non-operative trauma cases.

Will begin to send residents to Shock Trauma for one-month at a PGY 2 level and one month at a PGY 4 level. May need to add an additional monthFor more trauma and/or surgical critical care

Approval of Plan of Action

We then submitted our “Plan of Action” at the next faculty meeting for discussion and approval which was documented in the minutes

A summary of the Annual Review outcome was then forward to the GMEC for review and to address any concerns or issues.

Timeline

You want to do this early enough so that changes or deficiencies can be addressed for the upcoming academic year starting in July

Follow Up

We planned a follow up meeting 3-6 months after the meeting to give updates or failed attempts to our action plan

Will continue to hold an annual review meeting each year ~ March/April

Tips or Variations for your Own Annual Review

First, you must know what your RRC requires in regard to your review of the program

Set up a template to meet your own program needs and/or deficiencies and if you have no deficiencies noted then use it to plan new initiatives

Tips or Variations for your Own Annual Review

Use the template as your meeting agenda to make sure all areas are discussed and use it to write your minutes for documentation

Distribute the minutes to all faculty and residents even those not in attendance at the meeting

Tips or Variations for your Own Annual Review Keep a copy of the meeting minutes

handy for your next RRC accreditation site visit

Make this a “living document” and plan follow-up meetings in 3-6 months Evaluate what has happened to any plans What have been the successes What have been the challenges

Tips or Variations for your Own Annual Review

Most of us already have multiple opportunities to review our residency programs…Faculty Meetings, Internal Curriculum Committee Meeting, schedule Retreat/Planning Meetings.

Once you identify the strategy that works best for your program, remember to thoroughly document the review.

Questions

Special Thanks to:

Michael Holman, MDRaymond Kostin, MD