The Application of Entrustable Professional Activities (EPAs) in UME and GMEBronwyn H. Bryant, MD and Elise N. Everett, MDUniversity of Vermont College of MedicineTeaching Academy Med Ed Grand Rounds, October 30, 2020

Conflicts of Interest

• Nothing to disclose

BB

Purple Book-AAMC

Objectives

• Define competency based education, Milestones, and EPAs.

• Review the evolution of EPAs and its current role in competency based education

• Learn how to write an EPA for a learner in UME or GME.

BB

Definition of Competencies

• Medical education is moving towards competency-based training.– Medical school/UME: Domains of competence (AAMC)– Residency/GME: General Competencies (ACGME)

• Describe learner’s abilities• Competencies remain theoretical if not grounded in

practice– Knowledge– Professional attitudes/behaviors– Communication skills

• Milestones describe the steps a learner takes towards competencies

BB

Define Entrustable Professional Activity

Entrusted to a Trainee

Define the Practice of a

Specialty

Units of Professional

Practice

“A unit of professional practice, defined as tasks or responsibilities to be entrusted to the unsupervised execution by a trainee once he or she has attained sufficient specific competence.”

Specific tasks a trainee can be entrusted to execute, unsupervised, once competency is reached.

EPAs align the assessment process with learner action and faculty observation (nothing is implied)

BB

EPAs and Competencies

• Entrustable Professional Activities (EPAs) focus on day-to-day activities in the medical profession.

• EPAs link competencies to actual work• Faculty observe trainees doing EPAs through

a lens of competencies• Assessment is grounded in TRUST, and its

implications for level of supervision (and graduated responsibility).

BB

EPAs, Milestones, and Competencies

MilestonesStages (or levels) of

development to reach specific competencies, a continuum

EPAsDescriptors of physician workSpecific tasks

Map to ≥ 2 competencies

CompetenciesSkills, knowledge, and attitudes that

describe a physician, learner’s abilities

BB

UME vs GME Terminology

UME (AAMC)• Domains of

competence (8)• Competencies (many)• Milestones (2 levels)

– Novice, pre-entrustable learner

– Entrustable learner– ? Intermediate level-

advanced beginner (Can perform EPA with minimal intervention from a supervisor)

GME (ACGME)• General competencies (6)• Sub-competencies (many)• Milestones (5 levels)– Level 1 – Novice– Level 2 – Early learner– Level 3 – Proficient (mid-training)– Level 4 – Competent, ready for

unsupervised practice– Level 5 – Expert, aspirational

EE

EPAs – New kid on the block

• Gap between PD expectations and new resident performance

• In 2013, AAMC defined 13 core EPAs that every graduating medical student, intern should be able to do without DIRECT supervision on day one of residency

• ACGME milestones, while required for accreditation, can be abstract and hard to assess.

• Residency programs have implemented EPAs in various specialties (surgery, ortho, peds).

EE

13 Core Entrustable Professional Activities for Entering Residency1. Gather a History and perform a PE2. Prioritize a DDX3. Recommend and interpret common diagnostic and screening

tests4. Enter and Discuss orders/Rxs5. Document a clinical encounter6. Provide an oral presentation of a clinical encounter7. Form Clinical Questions8. Give or receive a handoff9. Collaborate as a member of an inter-professional team10. Recognize a patient requiring urgent or emergent care and

initiate evaluation and management11. Obtain informed consent for tests and/or procedures12. Perform general procedures of a physician13. Identify system failures and contribute to a culture of

safety and improvementEE

Guiding Principles for EPA Development

1. Patient Safety– Clinical skills of interns are variable– Interns perform many patient care activities

without supervision2. Develop a core of common skills3. Learner assessment is critical to implementation of

core requirements– Formative assessment– Entrustment decisions (Summative)

4. Implementation must be coupled with robust resources for faculty development

EE

Definition and Characteristics of EPAs

• Tasks or responsibilities that faculty entrust a trainee to do unsupervised once competence obtained– Units of work that focus on care delivery (WRVUs)

• Executable within a timeframe (start and end)• Observable• Measurable• Units of work:

– Obstetric care of an uncomplicated pregnancy patient– Work-up of an adverse reaction to transfused blood products.

EE

Develop an EPA for your learning environment

• EPAs– Hx and PE– DDX– W/U, tests– Orders– Note– Oral presentation

• EPAs– Form ?, EBM– Handoff– Team– Recognize sick pt– Informed consent– Procedures– Culture of Safety

EE

What is a task in your clinical space that would be suited to an EPA?

Examples: obstetric history, ddx for vaginal bleeding, pap smear, delivery note, place a foley in a female patient

Example EPA Worksheet

Steps to Designing an EPA

1. Title2. Map EPA to the Domains of Competency framework3. Map to the Competency framework

• Required knowledge, skills, attitudes, and behaviors

4. Identify information sources to assess progress (student, peers, faculty, SP)

5. Identify method to arrive at entrustment decision• Narrative descriptions of expected behaviors• Clinical vignettes for both novice and entrustable learners

6. Link to Residency Milestones Day One7. Write description(A well-written EPA is a mini-curriculum for learner)

EE

A UME exampleEE

Title

Step 1: Title

Title ObGyn Specific EPA#1

Gather a personal and family history of gynecologic cancers or related cancers

EE

Step 2 Map to DOC, Step 3 Map to Competency

DOC and Competencies

Step 2: Map the EPA to Domains of Competency (DOC)

1. * Patient Care (DoC1) 2. * Medical Knowledge (DoC2) 3. Practice-Based Learning and Improvement (DoC3)4. * Interpersonal and Communication Skills (DoC5) 5. Professionalism (DoC4)6. Systems-Based Practice (DoC6)7. Inter-professional Collaboration (DoC7)8. Personal and Professional Development (DoC8)

A UME example EE

EPADomains of Competency Mapping

EPA PCDOC1

MK DOC2

PBLDOC3

IPCSDOC4

ProfDOC5

SBPDOC6

IPEDOC7

PPDDOC8

Gather a PH, FH of Gyncancer x x xPerform a vaginal delivery x x xConsent patient for a BTL x x x x

A UME example

Domains of Competency (DOC)

EE

PC Competencies

Step 3: Map the EPA to Competencies (K, S, A, B)

• PC1: Perform all medical, diagnostic, and surgical procedures essential for area of practice (Comp 1.1)

• *PC2: Gather essential and accurate information with H and P, labs, imaging (Comp 1.2)

• PC3: Organize and prioritize responsibilities to provide safe care (Comp 1.3)

• PC4: Interpret labs, imaging, and other tests (Comp 1.4)

• PC5: Make informed decisions about diagnostic and therapeutic interventions using EBM (Comp 1.5)

• PC6: Develop and carry out management plans (Comp 1.6)

A UME example EE

Medical Knowledge Competencies

IPCS Competencies

Step 4: Identify Information Sources

• Self Assessment• Peer Assessment• Patient Assessment• Faculty Assessment• Standardized Patients• Formative/Summative• How many, cut points

Step 5: Making entrustment decisions

MILESTONE LEVELNovice (Pre-entrustable)Emerging

Proficient (Entrustable)

Competent/Expert

UM

EENTRUSTMENT LEVELObserving the activityActing with direct supervision in the roomActing with supervision available within minutesActing unsupervised (i.e. Clinical Oversight)Independent practice /supervision others

EE

Relationships of EPAs, DoCs, Cs, and Ms

EPA

DoC1* PC

C1.2M1

Novice NarrativePre-entrustable

M2

C1.4M1

M2

DoC2 MK

C2.1M1

M2

C2.5M1

M2

DoC4IPCS

C4.1M1

M2

C4.7M1

M2

Competent NarrativeEntrustable

DoC1=Patient CareDoC2=Medical Knowledge DoC4=Interpersonal and Communication Skills A UME example EE

Patient Care Behaviors

MK and IPCS Behaviors

IPCS Behaviors

Entrustable Narrative-Checklist

Step 6: Link to Residency Milestone 1, Level 1 and 2

General EPA 1: Gather a history Developer’s Guide/Gather a History

-Obtain a complete and accurate history in an organized fashion-Demonstrate patient-centered interview skills-Identify pertinent history elements -Obtain focused, pertinent histories in urgent, emergent, consultative settings-Consider cultural and other factors that may influence patient’s care-Identify alternate sources of info-Demonstrate clinical reasoning-Demonstrate cultural awareness and humility and awareness of bias

ObGyn Specific EPA 1: Gather a personal and family history of gynecologic cancers or related cancers

Milestone 18: Healthcare Maintenance and Disease Prevention—Medical KnowledgeLevel 1:

-Demonstrates knowledge of the characteristics of a good screening test-Demonstrates knowledge of indications and limitations of commonly used screening tests

Level 2:-Demonstrates knowledge of evidence-based, age-appropriate guidelines for women’s health maintenance and disease prevention (breast screening, cervical cancer screening)-Recommends age-and risk-appropriate vaccinations

Level 3:-Interprets age- and risk-appropriate tests (DEXA, MMG, lipids, thyroid)-Develops patient-centered management plans to maintain health and prevent disease

A UME example EE

OB/GYN Example

Domains of Competency: Patient Care (PC), Medical Knowledge (MK) for Practice (KP), Interpersonal and Communication Skills (IPCS)Patient Care Specific Competencies:

-PC1.2: Gather essential and accurate information through history-taking Medical Knowledge for Practice Specific Competencies:

-MK2.1: Apply principles of epidemiological sciences to the identification of risk factors, disease prevention/health promotion efforts for patients

-MK2.5: Apply principles of social-behavioral sciences to provision of patient care, including the assessment of the impact of psychosocial-cultural influences on health, disease, care-seeking, care-compliance, and barriers to and attitudes toward careIPCS Specific Competencies:

-ICS4.1: Communicate effectively with patients, families

Assessment: Observation of student by supervising physician with checklist to ensure all aspects are covered.

Number of entrustable-level observations required for entrustability: at least one cancer history from each of 3 assessment faculty

Level of situational complexity: short, focused history, ambulatory clinic, caring for a panel of 3 patients in a session. Low level of complexity.

A UME example EE

Step 7: Write your description, vignette, mini curriculumOur description/vignette of an entrustable learner for the ObGyn Specific EPA 1 of gathering a personal and family history of gynecologic cancers or related cancers might be:Asks patient about personal history of vulvar, vaginal, cervical, uterine, FT/ovary, or breast, or colon cancer. Asks patient about family history of vulvar, vaginal, cervical, uterine, FT/ovary, or breast cancer.Ask age of diagnosis for family members with cancer.Asks about relationship of family member with cancer to patient, i.e. first degree relative.Asks about risk factors for HPV related cancer: age at first intercourse, number of sexual partners, history of STDs, HPV vaccination, history of abnormal pap smears, colposcopy, and cervical procedures.Asks about risk factors for uterine cancer, i.e. early menarche, late menopause, infertility, PCOS, obesity, unopposed estrogen exposure, family history of uterine/colon cancerAsks about risk factors for FT/ovary cancer, i.e. early menarche, late menopause, infertility, endometriosis, family history of breast/ovarian cancer, symptoms of ovarian cancer (ovarian cancer symptom index)Asks if patient is up to date on screening pap, mammogram, colonoscopy according to age group recommendations.Asks about any abnormal results, endometrial biopsy, pelvic ultrasound with a thickened stripe, abnormal ovary, abnormal mammogram or colonoscopy or additional tests such as breast MRI, breast US, breast biopsy.Demonstrates clinical reasoning and recommends appropriate screening based on age, risk factors, i.e. pap/HPV testing, HPV vaccine, MMG, colonoscopy.Demonstrates clinical reasoning and identifies patients who may be at risk for a genetic syndrome, BRCA or Lynch and recommends genetic counseling. Obtain a complete and accurate history in an organized fashion using patient-centered interview skillsUtilizes other sources of information, operative reports, pathology reports, family members to obtain the most accurate and thorough history.

A UME example EE

A GME exampleBB

GME Example

• Title: Perform an intraoperative consultation/Frozen section

A GME example BB

Mapping EPAs to ACGME Milestones

1. Patient Care (PC)2. Medical Knowledge (MK)3. Practice-Based Learning and Improvement (PBLI)4. Professionalism (PROF)5. Interpersonal and Communication Skills (ICS)6. Systems-Based Practice (SBP)

A GME example

**

*

BB

Relationships of EPAs, Competencies, Subcompetencies, and Milestones

EPA

PC

PC4

Lv2

Lv3

Lv4

PC5

Lv2

Lv3

Lv4

MK

MK1

Lv2

Lv3

Lv4

MK2

Lv2

Lv3

Lv4

ICS ICS2

Lv2

Lv3

Lv4

A GME example

Direct supervision

Proficient

Competent

Patient care (PC)Medical Knowledge (MK)Interpersonal Communication Skills (ICS) BB

Level of Entrustment vs Milestone

MILESTONE LEVEL

Novice (Pre-entrustable)

Entrustable/Proficient

Competent/Expert

GME

ENTRUSTMENT LEVELObserving the activityActing with direct supervision in the roomActing with supervision available within minutesActing unsupervised (i.e. Clinical Oversight)Independent practice /supervision others

BB

Pathology Example: Frozen

1. Properly identify patient specimen before and throughout the frozen section/intraoperative consultation process (PC5)

2. Determine correct method of tissue preparation needed (e.g., frozen section, cytologic preparation, gross evaluation, etc.) (MK1, PC5)

3. Sample tissue appropriately for diagnosis and/or evaluation of margins (MK1, PC5)

4. Prepare (e.g., cut frozen sections on cryostat, perform cytologic touch preparations) and stain slides for microscopic evaluation (PC5)

5. Evaluate and interpret microscopic findings (MK1, MK2, PC4, PC5)6. Communicate diagnosis, including limitations of technique or

interpretation in a timely manner (ICS2)7. Apportion tissue for special studies if needed and prepare tissue in

appropriate fixative or media for transport, storage, or further processing (MK1, PC5)

A GME example BB

Incorporating EPAs across training(the big picture)

EE

Learner Portfolio Report Card

Portfolioof:Learner Jones

MS1 MS2 MS3 MS4 PGY1 PGY2 PGY3

EPA A 1 2 3 3 3 3 3 3 4 4 4 5 5 5

EPA B 1 1 2 3 3 3 3 3 3 4 4 4 5 5

EPA C1 1 1 2 2 3 3 3 3 3 4 4 4 5

EPA D 1 1 1 1 1 2 2 2 3 3 3 4 4 5

Manageable # of EPAs for 4 year program=20-301. Observing the activity2. Acting with direct supervision in the room3. Acting with supervision available within minutes4. Acting unsupervised (i.e. Clinical Oversight)=Proficient5. Independent practice / Supervising others EE

Summary/Next Steps

• General Advantages– Align assessment with learner action/faculty observation (nothing is implied)– Describe specific expectations

• Promotes SMART (formative) feedback – EPAs introduce TRUST and its implication for level of supervision

• Entrustment rating• Summative feedback

• Learner Advantages– A well-written EPA is a mini-curriculum, outline K/S/A/B to complete an activity– Clear expectations of “competent”– Entrustment assessments make it clear when learner can advance to more graduated

responsibility

• Evaluator Advantages– Promotes direct observation of learner– Checklist of K/S/A/B to observe, allows for SMART feedback– Overall “Entrustment Score”

• Multiple evaluations can inform when learner can advance to more graduated responsibilities or decreased supervision

EE

Questions

EE

References

• Englander R, Flynn T, Call S, Carraccio C, Cleary L, Fulton T, etc. Toward Defining the Foundation of the MD Degree: Core EntrustableProfessional Activities for Entering Residency. Academic Medicine. October 2016;91, No. 10:1352-1358

• Englander R, Carraccio C. From Theory to Practice: Making Entrustable Professional Activities Come to Life in the Context of Milestones. Academic Medicine. October 2014;89, No 10:1321-1323

• Cate O. AM Last Page: What Entrustable Activities Add to a Competency-Based Curriculum. Academic Medicine. April 2014;89, No. 4:691

• Englander R, Cameron T, Ballard A, Dodge J, Bull J, Aschenbrener C. Toward a Common Taxonomy of Competency Domains for the Health Professions and Competencies for Physicians. Academic Medicine. August 2013;88, No. 8:1088-1094

• AAMC Core Entrustable Professional Activities for Entering Residency, Faculty and Learner’s Guide