milestones: 1.0, 2.0, and oh my! - ahme

10/2/2020

1

Milestones: 1.0, 2.0, and Oh My!

Laura Edgar, EdDVice President, Milestones Development

©2020 ACGME

Disclosure

• Full-time employee of ACGME

2

©2020 ACGME

Milestones

• A milestone is a significant point in development

• Milestones follow an individual’s developmental trajectory across a range of knowledge, skills, and attitudes

3

1

2

3

10/2/2020

2

©2020 ACGME

4

©2020 ACGME

Milestones were designed to be formativeA repository for other assessments

Not every Milestone can or should be evaluated on every rotation Not everything that should be evaluated is included in the Milestones

Milestones as Assessments

Milestone Evaluation

ITE

DOMSF

5

©2020 ACGME

Differences

Patient Care and Medical Knowledge have two options outside of the levels:• Not yet completed Level 1/Critical deficiencies • Not yet rotated/Not yet assessable

6

4

5

6

10/2/2020

3

©2020 ACGME

What changed?

7

©2020 ACGME

Supplemental Guide

Examples for Levels 1-5Assessment methodsResources

8

©2020 ACGME

Supplemental Guide

9

7

8

9

10/2/2020

4

©2020 ACGME

Supplemental Guide

Review the Milestones and Supplemental guide with your CCC

• Program’s expectations at each level• Assessment tool(s) you will use• Rotations each will be assessed

Your CCC should do a shared mental model exercise

10

©2020 ACGME

Supplemental Guide - Map

11

©2020 ACGME

Milestones 2.0

12

10

11

12

10/2/2020

5

©2020 ACGME

Identify and address obstacles a) Be open and listen to concernsb) Answer what you can – Call ACGME if notc) For content – remind them that their specialty

community created them

Implementing Changes

13

©2020 ACGME

Identify programs who are doing the change wella) Let your champions demonstrateb) Offer additional faculty developmentc) Begin development of new shared mental model


14

©2020 ACGME

Share success stories and document what strategies worked best

a) Each specialty should have its own championb) Share experiences at GME meetingsc) Share experiences at specialty meetings


15

13

14

15

10/2/2020

6

©2020 ACGME


16

©2020 ACGME

Using Milestone Data

17

©2020 ACGME

Self-Assessment Leads to Learning Plans• Milestones are an ideal way for residents and fellows

to monitor their professional progress• Faculty can also benefit from self-assessment with

the Milestones• Requires Purposeful and Deliberate self-review and

practice to continue growth

18

16

17

18

10/2/2020

7

©2020 ACGME

Family Medicine 2019

Using Milestones Data: Boxplots

19

©2020 ACGME

Milestones Trajectories

n = 270(20%)

n = 1066(80%)

Jul ‘13 Entrance

1st Assessment Graduation

N=1336

How much risk at this point in training?

20

©2020 ACGME

1st year 2nd year 3rd yearCut-offs: Levels Mid-Year 1 Year-End 1 Mid-Year 2 Year-End 2 Mid-Year 3

% Not Achieving Level 4 At Graduation <= Lev5.0<= Lev4.5<= Lev4.0<= Lev3.5 17.5 18.3 26.9<= Lev3.0 17.3 17.4 18.8 24.7 52.4<= Lev2.5 17.4 18.4 26.2 51.5 81.0<= Lev2.0 18.5 22.8 39.5 62.6 88.1<= Lev1.5 22.4 32.9 48.5 90.9<= Lev1.0 27.8 38.1= Lev0.0 25.2

PPV Matrix: Example

21

19

20

21

10/2/2020

8

©2020 ACGME

Dermatology - PC01

22

©2020 ACGME

Surgery PPV – MK01

23

©2020 ACGME

Simply ask “why?”

• Is further assessment or clarification needed?• Or is this just simply an expected pattern in your program?

• Could such a realization help guide an individualized learning plan or need for a formal remediation plan?

The purpose of Milestones analytics is to facilitate professional growth and improvement

If You See These Patterns…

24

22

23

24

10/2/2020

9

©2020 ACGME

• ACGME did not require submission of Milestones in spring 2020

• More than 80% of programs did submit• National Report will show mid-year data

Impacts of COVID

25

©2020 ACGME

Where do I find...?

26

©2020 ACGME

• Are you aware of the Milestones resources available from ACGME?

• Resident-Fellow Guidebook for Milestones?• Annual Milestones National Report?

Resources

27

25

26

27

milestones: 1.0, 2.0, and oh my! - ahme

Documents