© 2019 American Board of Medical Specialties

Richard E. Hawkins, MDPresident and Chief Executive OfficerAmerican Board of Medical Specialties

UPDATE ON ABMS CONTINUING CERTIFICATION

November 16, 2019

- Salaried employee of the American Board of Medical Specialties- Co-editor of a textbook on Evaluation of Clinical Competence for

which I receive royalties from Elsevier- Co-editor of a textbook on Health Systems Science for which the

AMA receives royalties from Elsevier

2

DISCLOSURES

- Describe the new paradigm of continuing board certification (CBC), which has replaced Maintenance of Certification (MOC)

- List the advantages of participation in CBC- Explain the current position of the AMA and its contributions to

improvements in MOC/CBC, based on Council on Medical Education reports and AMA policy

3

LEARNING OBJECTIVES

- Background on ABMS continuing certification- Supporting evidence

- Drivers for change- Physicians’ concerns and discontent- Emerging evidence regarding learning and assessment

- ABMS response - Innovation in continuing certification programs - Focus on new formats in knowledge assessment

- Moving forward in enhancing the value of continuing certification- Optimizing formative and summative goals

4

PRESENTATION OUTLINE

BACKGROUND ON ABMS CONTINUING CERTIFICATION

ABMS MEMBER BOARDS

24 Independent Member Boards | 40 Specialties | 87 Subspecialties

6

As of June 30, 2019

More than

900,000 Board Certified

Physicians

Approximately

590,000Participating in

Continuing Certification

7

- Medical science advances at an astounding pace, how can we keep up?- Bastian, et al. (2010, PLoS Med), Glynn, et al. (2010, PLoS One)

- Knowledge and skills decline over time- Choudhry, et al. (2005, Ann Intern Med), Durning, et al. (2010, J Cont

Educ Health Prof), Hawkins, et al. (2016, J Cont Educ Health Prof)

- Physicians cannot accurately assess their own performance- Davis, et al. (2006, JAMA)

- The least skilled are least able to self-assess accurately- Kruger, et al. (1999, J Pers Soc Psychol)

- Patients and credentialers expect it- Brennan, et al. (2004, JAMA), Freed, et al. (2013, J Hosp Med)

8

THE NEED FOR CONTINUING CERTIFICATION

- Examination performance and level of participation associated with disciplinary action against medical license

- Examination performance correlates with quality and cost of care and patient outcomes

- Participation in Board approved quality improvement activities associated with quality of care and patient outcomes

9

EVIDENCE SUPPORTING CONTINUING CERTIFICATION

Reviews: Nichols, Pediatrics, 2017 & Price, Academic Medicine, 2018

DRIVERS FOR CHANGE

WHATWE’VEHEARD Not meeting learning and

improvement needs Not integrated into practice

Not relevant to practice High Stakes Exam Difficult to Navigate

11

- Failure to participate in continuing certification- Public criticism: academic literature, lay publications, social media- Legislation to restrict use of the certificate- Litigation opposing the requirement to participate

- Distraction from responding to legitimate needs and concerns- Potential for increased costs to physicians

- Most significant concern: erosion of confidence in the profession

12

CONSEQUENCES OF PHYSICIAN DISCONTENT

- Suggest alternative forms of knowledge assessment superior to point-in-time, high-stakes examination in ensuring physicians remain current in their specialty

- Leverage emerging evidence and methods to ensure more durable learning (test-enhance learning, spacing effect, mastery learning)

- Evolution in thinking about how lower stakes, assessment for learning outcomes could be aggregated to support defensible assessments of learning (van der Vleuten & Schuwirth)

- Perspective – it is wasteful to not take advantage of high-quality assessment outcomes to support physician learning and improvement (Medical Education Assessment Advisory Committee, Medical Council of Canada, 2013)13

ADVANCES IN LEARNING THEORY AND EVIDENCE

CONTINUING BOARD CERTIFICATION: VISION FOR THE FUTURE COMMISSION

- Collaborative effort that brought together multiple stakeholders to envision the future of continuing board certification.

- Independent body of 27 individuals representing diverse stakeholders

- Charge: Make recommendations regarding principles, frameworks and program models for the continuing board certification system

- The Commission gathered and considered stakeholder input to produce the set of recommendations in the Final Report.

- Series of meetings February 2018 – January 2019 (Final Report February 2019)

15

ABOUT THE COMMISSION

- Core themes:- Need to bring value to physicians to support their learning and

improvement needs- Need to bring value to the profession (and other stakeholders) by

offering a meaningful credential- Meaningful self-regulation requires a system of engaged stakeholders –

the solution is a collaborative one

- Recommendations:- Foundational Recommendation- Short-term and Intermediate Recommendations (10)- Aspirational Recommendations (3)

16

COMMISSION REPORT

- Integrated program including professionalism, assessment, life-long learning & practice improvement

- Alternatives to high-stakes, point-in-time assessments- Responsibility to take action if standards are not met- Remediation pathways before/after certificate revocation- Enhanced consistency across programs- Enhance engagement and communication with physicians- Increase transparency regarding physician participation- Independent research regarding program effectiveness

17

COMMISSION REPORT: KEY RECOMMENDATIONS

- Enhance environment supporting physician engagement in meaningful practice improvement

- Develop methods for assessing professional behaviors- Enhance information and data sharing with partners in professional

self-regulation

18

COMMISSION REPORT: ASPIRATIONAL RECOMMENDATIONS

ABMS RESPONSE: CURRENTAND FUTURE IMPLICATIONS

- Endorsed all 14 Commission recommendations- Agreed to alternatives to high-stakes, point-in-time assessments- Appointed:

- Oversight Committee to ensure timely implementation of recommendations

- Standards Task Force to develop new continuing certification standards by end of 2020

- 4 “Collaborative” Task Forces:- Assessment of professionalism- Advancing practice- Information and data sharing- Remediation pathways

20

ABMS RESPONSE

- Bringing value to the profession and the public- Supporting physician CPD and ensuring a meaningful credential for

patients- Balancing formative and summative approaches

- Collaborative solution within the profession- Task forces – collaborative in composition and process- Better engagement of physicians and other stakeholders

- Specialty and state medical societies- Academic health centers- CME/CPD organizations- Regulatory organizations

21

FOUNDATIONAL CHANGE IN HOW ABMS ACHIEVES ITS MISSION

Engaging Physicians• Surveys & focus groups• Committee appointments

Expanding Resources• Easier access to CME• Registries, Portfolio Program

Reducing burden• Remote deliver of assessments• More formative approaches• More user-friendly websites

Increasing Practice Relevance• Credit for practice-based activities

(“wide door”)• Modular testing formats

Continuing Certification Enhancements

ABMS MEMBER BOARDS ARE RESPONDING

22

- Transition from 10-year, high-stakes examination to more frequent lower-stakes formats

- Performance data aggregated over time to support summative decisions

- New approaches across the 24 Member Boards include:- Longitudinal formats (MOCA Minute, MOCA Peds)- Modular examinations- Two-year “knowledge check-ins” with multiple attempts to pass- Article based assessments

23

INNOVATIONS IN KNOWLEDGE ASSESSMENT

Frequent, short assessments with spaced repetition to verify learning

LONGITUDINAL ASSESSMENT AS A SOLUTION

• Real time inclusion of current/emerging content• Results accumulated over time to inform

summative decisions

Emphasis on targeted feedback to help close knowledge gaps

• Supports learning & improvement• Immediate and detailed critiques/rationales• Dashboard displays areas of strength and weakness

Practice relevance• Individualized user customization• Ratings of confidence and relevance• Reflect how knowledge is applied in practice

Web and mobile administration

• Flexible anytime, anywhere access• Users complete assessments at a time and

place of their choosing

24

0

10

20

30

40

50

60

70

80

90

100

Category 10

10

20

30

40

50

60

70

80

90

100

Category 10

10

20

30

40

50

60

70

80

90

100

Category 10

10

20

30

40

50

60

70

80

90

100

Category 195% indicated a reduction in test

anxiety

98% prefer CertLink and longitudinal

assessment over the every 10-year exam

99.6% feel CertLink is a feasible method to keep up-to-date with

developments

88% feel CertLink is an adequate assessment of fundamental knowledge

used in everyday practice

CERTLINK®: USER SURVEY RESULTS

25

VOICES OFDIPLOMATES WHOVALUE MOC

“(I like) The online annual certification from a predetermined selection of journal articles, which gives us the choice to select topics which may be of greater interest or relevance to our practice setting.”

- OB-GYN Diplomate on the Board’s online annual certification

“MOCA Minute is a forward-thinking, engaging form of assessment which supports learning in an ongoing way, not just infrequent testing that ultimately actually teaches nothing.”

- Anesthesiology Diplomate on the Board’s MOCA Minute

“Reviewing material in my subspecialty especially through CARSEP and SESAP are most valuable to me.”

- Surgery Diplomate on the Society’s test review materials

26

27

THANK YOU