the next accreditation system and sponsor site visit...

Accreditation Council for Graduate Medical Education

The Next Accreditation System and Sponsor Site Visit Program

The Clinical Learning Environment Review (CLER)

Thomas J. Nasca, MD MACP Chief Executive Officer

© 2012 Accreditation Council for Graduate Medical Education (ACGME)

Disclosure

• Professor of Medicine, Jefferson Medical College (volunteer)

• Internist Nephrologist

• Full Time Salaried by ACGME

• No conflicts of interest to report

Clinical Learning Environment Review (CLER)

• History and important elements

• Evolution of the concept • Key Elements of CLER

• How it “fits” in the “Next Accreditation System”


Clinical Learning Environment Review History

• 2008 Institute of Medicine (IOM) Report calls for ACGME: • “Strengthen monitoring processes,” • to visit all accredited programs “unannounced,” on a yearly

basis • to oversee compliance with duty hours standards

• February 2009 ACGME Board of Directors approves

“annual sponsor site visit program” • oversee institutional oversight of duty hour standards adherence • implementation when planned revised duty hour standards

implemented


Clinical Learning Environment Review History 2009-2010 ACGME Duty Hours Task Force

• Revised standards, based on level of training and expertise

• Links duty hours to progressive and conditional independence through requirements for levels of supervision

• Prepares the field for linkage of competency based Milestone performance to duty hour standard progression


2009-2010 ACGME “Duty Hours Task Force” Task Force for Quality Care and Professionalism

• Links duty hours standard adherence and integrity in reporting to professional responsibilities to patients for safety and quality

• Establishes importance of institutional Patient Safety and Quality Improvement programs in education of residents

• Places onus of responsibility at the level of the institution for oversight of : • integration of residents into Patient Safety programs of the

institution • integration of residents into Quality Improvement programs of

the institution • establishment and implementation of Supervision policies • transitions in care • duty hours standards implementation


2010-2011 National Advisory Committee Advise ACGME on establishment and goals of the Sponsor Site Visit Program

Carolyn Clancy, MD Director, Agency for Healthcare Research and Quality James Bagian, MD PE Director, Veterans Administration National Patient Safety Center Kevin Weiss, MD MPH President, American Board of Medical Specialties Joann Conroy, MD Chief Health Care Officer, Association of American Medical

Colleges John Combes, MD Senior Vice President, American Hospital Association Paul Schyve, MD Senior Vice President, The Joint Commission Donald Goldmann, MD Senior Vice President, Institute for Health Care Improvement David Nash, MD MBA Dean, School of Population Health, Thomas Jefferson University Carl Patow, MD MPH Vice President, Health Partners Institute for Medical Education Robert Wachter, MD Associate Chairman, Department of Medicine, Univ. of

California, San Francisco Timothy Flynn, MD Senior Associate Dean for Clinical Affairs, Chief Medical Officer,

Univ. of Florida and Shands Medical Center Baretta Casey, MD MPH Director for the Univ. of Kentucky Center for Excellence

in Rural Health John Duval, MBA Chief Executive Officer, Medical College of Virginia Hospitals Timothy Goldfarb Chief Executive Officer, Shands Medical Center Paige Amidon, MBA Vice President, Health Programs. Consumers Union Carmen Hooker Odom, MRP President, Milbank Memorial Fund Thomas Nasca MD ACGME Convener

National Advisory Committee Recommendations

• Linked to accreditation, but not an “accreditation site visit”

• Full time and volunteer peer site visitors

• Report drafted by site visit team

• Report finalized by an “Evaluation Committee”

• Report provided to institution as a quality improvement tool, and to Institutional Review Committee (IRC) as “continuous data” element

• First Visit and Report solely for establishment of baseline, and for learning (for all) – not to be used in accreditation


• require no specific format or “standardized” data submission

• use currently acquired institutional data

• rely on records maintained current in key areas

• announced with short warning (2-3 weeks)

• involve senior leadership of sponsor (and principle teaching hospital)

• evolve to involve all major teaching sites of sponsored programs

National Advisory Committee Recommendations Visit Should:


• involve assessment of effectiveness of institution in assurance of : • integration of residents into Patient Safety programs of the

institution, and demonstration of impact

• integration of residents into Quality Improvement programs of the institution, and demonstration of impact

• establishment and implementation of Supervision policies

• oversight of transitions in care

• oversight of duty hours standards implementation

National Advisory Committee Recommendations CLER Visit Should:


Where are we in Implementation of CLER?

• First cycle of visits begin September 2012

• First visit will be solely for feedback, learning, and establishment of baselines on behalf of sponsors, evaluation committee, and Institutional Review Committee

• Only theoretical exception would be identification of potential egregious violations involving threats to patient safety or resident safety/well being

• First cycle should result in dissemination of salutary practices by Evaluation Committee



• ACGME Board of Directors approved policies and budget for CLER program, including: • Recruitment of Senior Vice President

• Recruitment of Site Visitors, staff

• Establishment of Evaluation Committee

• Renamed SSVP to CLER

• Structured CLER consistent with National Advisory Committee

Recommendations



• Senior Vice President for Institutional Accreditation • Co-Chair of Evaluation Committee • Kevin Weiss, MD MPH • President, ABMS • Co-Director, Graduate Programs in Healthcare

Quality & Patient Safety, Northwestern University • March 12, 2012

• Co-Chair of Evaluation Committee • James Bagain, MD, PE • Former Director, VA National Patient Safety Center • Director of the Center for Health Engineering,

Department of Anesthesia, University of Michigan


Emphasis of CLER Assessment of Effectiveness of Sponsor in:

• integration of residents into Patient Safety programs of the institution, and demonstration of impact

• integration of residents into Quality Improvement programs of the institution, efforts to reduce Disparities in Health Care Delivery, and demonstration of impact

• establishment and implementation of Supervision policies

• oversight of transitions in care

• oversight of duty hours standards implementation

• Emphasis on Professionalism throughout


Why is this Important?

Integration of CLER into The Next Accreditation System

• Provides frequent onsite sampling of the learning environment • Meaningful information for IRC • Permits lengthening of interval for visits of individual programs if

other parameters of program performance are at expected levels

• Emphasizes elements of “new” competencies demanded by the public

• Provides the opportunity for GME Sponsors to demonstrate leadership in Patient Safety, Quality Improvement, and Reduction in Disparities

• Ultimately hope to move from “duty hours” to Quality and Safety of Patient Care


Temporal Trends in Rates of Patient Harm Resulting from Medical Care, 2002-2007 Landrigan, C.P., et.al. NEJM 2010; 363:2124-34

The North Carolina Experience

• No significant change in: •All Harms

•Preventable Harms •High-Severity Harms

• …whether evaluated by external or internal

reviewers

Evaluating Residency Programs Using Patient Outcomes n= 4,906,169 deliveries in Florida and New York, 1992-2007

4124 physician program graduates of 107 residency programs

0

2

4

6

8

10

12

14

Q5 Q4 Q3 Q2 Q1 Q1-Q5

Residency Program of Origin, Ranked (Quintile) by Program Complication Rate

Rate of Major Obstetric Complications by Graduates (%)

10.1-10.5

11.3-11.4 11.9-12.0

12.3-12.5

13.6-14.0

2.8 – 3.8

∆

Difference remains after correction for

USMLE performance

Excess Risk ∆ 33% Q1 vs Q5

JAMA 2009;302(12):1277-1283. Asch, DA, et.al., Table 4

“All systems are perfectly designed to get the results they are getting.”

Various Attributions:

Paul Batalden MD Donald Berwick MD W. Edwards Deming

http://www.brainyquote.com/quotes/quotes/a/aristotle379604.html

http://www.brainyquote.com/quotes/quotes/a/aristotle379604.html

We want America’s Teaching Hospitals to lead the way to Safety and Quality!

1 Nasca, T.J., Philibert, I., Brigham, T.P., Flynn, T.C. The Next GME Accreditation System: Rationale and Benefits.

New England Journal of Medicine. Published Electronically, February 22, 2012. In Print, March 15, 2012. DOI:10.1056/nejmsr1200117 www.nejm.org .

NEJM. 2012.366;11:1051-1056.

Forerunner of the Educational Innovations Project (EIP)

A New Model for Accreditation of Residency Programs in Internal Medicine Allan H. Goroll, MD; Carl Sirio, MD; F. Daniel Duffy, MD; Richard F. LeBlond, MD;

Patrick Alguire, MD; Thomas A. Blackwell, MD; William E. Rodak, PhD; and Thomas Nasca, MD, for the Residency Review Committee for Internal Medicine

Ann Intern Med. 2004;140:902-909.

Outline

• Expansion of elements contained in recent New England Journal of Medicine article1

• Background for Next Accreditation System (NAS),

including Milestones

1 Nasca, T.J., Philibert, I., Brigham, T.P., Flynn, T.C. The Next GME Accreditation System: Rationale and Benefits.

New England Journal of Medicine. Published Electronically, February 22, 2012. In Print, March 15, 2012. DOI:10.1056/nejmsr1200117 www.nejm.org .

NEJM. 2012.366;11:1051-1056.

The 2005 ACGME Strategic Plan1: Emergence of “The New Accreditation Model”

“At its November 2005 retreat, the ACGME Executive Committee endorsed four strategic priorities designed to enable emergence of the new accreditation model:

• Foster innovation and improvement in the learning environment • Increase the accreditation emphasis on educational outcomes • Increase efficiency and reduce burden in accreditation • Improve communication and collaboration with key internal and

external stakeholders “

1 ACGME 2005 Strategic Plan. (Emphasis Added, TJN) © 2012 Accreditation Council for Graduate Medical Education (ACGME)

Brief Reminder of Competencies/Milestones Mid-late this past decade

• Competency evaluation stalls at individual programmatic definitions

• MedPac, IOM, and others question • the process of accreditation • preparation of graduates for the “future” health care delivery

system

• House of Representatives codifies “New Physician Competencies”

• MedPac recommends modulation of IME payments based on competency outcomes

• Macy issues 2 reports (2011) • IOM 2012-2013


Suffice it to say that there is sufficient motivation,

internal and external to the ACGME, and all involved in GME,

for change.

There are no easy solutions or shortcuts!

“I have a microwave fireplace. You can lay down in front of the fire

all night in eight minutes.”

Steven Wright

Background on Milestones

The Continuum of Clinical Professional Development Authority and Decision Making versus Supervision

Authority and Decision Making Low High

Supe

rvis

ion

Low

High Physical Diagnosis

Internship

Residency

Fellowship

Sub-Internship

Attending

Clerkship “Graded or Progressive

Responsibility”


The Continuum of Professional Development The Three Roles of the Physician1

Dev

elop

men

t

Low

High

Physical Diagnosis

Clerkship Internship Residency Fellowship Sub-Internship Attending

Clinician Teacher

Manager of Resources

1 As conceptualized and described by Gonnella, J.S., et. al. Assessment Measures in Medical Education, Residency and Practice. 155-173.

Springer, New York, NY. 1993, and in 1998 Paper commissioned by ABMS. Descriptively graphed by Nasca, T.J. © 2012 Accreditation Council for

Graduate Medical Education (ACGME)

The Six Competencies, and the Continuum of Clinical Medical Education – Dreyfus (modified) Conceptual Model1

• Medical Knowledge • Patient Care and

Procedural Skills 2 • Interpersonal and

Communication Skills • Professionalism • Practice Based Learning

and Improvement • Systems Based Practice

1 as presented by Leach, D., modified by Nasca, T.J. American Board of Internal Medicine Summer Retreat, August, 1999.

2 Patient Care Competency modified 9/2010 by ACGME and ABMS

• Novice • Advanced Beginner • Competent • Proficient • Expert • Master

• Undergraduate • Graduate • Continuing


The Goal of the Continuum of Clinical Professional Development

Master

Expert

Proficient

Competent

Advanced Beginner

Novice

Undergraduate Graduate Medical Clinical Medical Education Education Practice


Master

Expert

Proficient

Competent

Advanced Beginner

Novice

PGY 1 PGY 2 PGY 3 PGY 4 MOC

Anesthesia Related Technical Skills System Based Practice, OR Team Skills Patient Care, Non-Procedural

Increase the Accreditation Emphasis on Educational Outcomes

The Goal of the Continuum of Professional Development in the 4 year preparation of the Anesthesiologist


Milestones

• Observable developmental steps moving from Novice to Expert/Master

• “Intuitively” known by experienced medical educators in each specialty

• Organized under the rubric of the six domains of clinical competency • Describe a trajectory of progress from neophyte towards

independent practice • Articulate shared understanding of expectations • Set aspirational goals of excellence • Provide a framework and language for discussions across the

continuum


ACGME Goal for Milestones - Permits fruition of the promise of “Outcomes Based Accreditation”

• Tracks what is important - Outcomes

• Begins using existing tools and observations of the faculty

• Clinical Competency Committee triangulates progress of each resident • ABMS Board has the opportunity to track the identified individual

• ACGME Review Committee tracks unidentified individuals’ trajectories


ACGME Goal for Milestones - Permits fruition of the promise of “Outcomes Based Accreditation”

• Specialty specific normative data and common expectations for progress of individual residents

• Less prescriptive ACGME program requirements, lengthened program site visit cycles, less frequent standards revision • Promote curricular innovation

• Enhance curricular and rotation design flexibility

• Development of specialty specific evaluation tools and techniques


ACGME Goals for Milestones

• Able to provide accountability for effectiveness of educational program in producing outcomes

• ACGME can work with AAMC, LCME to focus graduation level preparation

• ACGME can work with ABMS, AHA, ACCME, others to identify areas for milestone improvement at graduation from residency/fellowship

Milestones

The “Envelope of Expectations” AKA - Milestones

Aspirational Goal

Graduating Resident

Intermediate Level Resident

Finishing PGY 1

Entering PGY 1

Expert

Proficient

Competent

Advanced Beginner

Novice

PGY 1 PGY 2 PGY 3 PGY 4 PGY 5 MOC


Move from Numbers to Narratives1

• Numerical systems produce range restriction (ABIM, others)

• Narratives easily discerned by faculty • Narratives shown to produce data without range

restriction (Hodges)

1 Most recent reference: Regehr, Glen, Ginsburg, S., Herold, J., Hatala, R., Eva, K., Oulanova, O.

Using “Standardized Narratives” to Expolre New Ways to Represent Faculty Opinions of Resident Performance. Academic Medicine. 2012. 87(4); 419-427.


The “Envelope of Expectations” Professionalism:

Accepts responsibility and follows through on tasks

Medical PGY 1 PGY 2 PGY 3 PGY 4 PGY 5 MOC School

Expert

Proficient

Competent

Advanced Beginner

Novice

Resident completes many assigned tasks on time but needs extensive guidance on local practice and/or

policy for patient care.

Resident routinely completes most assigned tasks in a timely manner in accordance with local practice and/or policy, but still requires guidance in

unfamiliar circumstances.

Resident frequently prioritizes multiple competing demands and completes the vast majority of his/her responsibilities in

a timely manner. Self identifies circumstances and actively seeks

guidance in unfamiliar circumstances.

Resident always prioritizes and willingly works on multiple competing complex

and routine cases in a timely manner by directly providing patient care or by

overseeing it. In difficult circumstances appropriately seeks guidance. Is regularly sought out by peers and

subordinates to provide them guidance.

Resident effectively manages multiple competing tasks, and effortlessly manages complex

circumstances. Is clearly identified by peers and subordinates as

source of guidance and support in difficult or unfamiliar circumstances.


Professionalism

Unprofessional

Resident seeks out opportunities to demonstrate compassion and

empathy in the care of all patients; and demonstrates respect and is

sensitive to the needs and concerns of all patients, family members, and members of the

health care team.

Resident demonstrates compassion and empathy in care

of some patients, but lacks the skills to apply them in more

complex clinical situations or settings. Occasionally requires

guidance in how to show respect for patients, family members, or

other members of the health care team.

Resident frequently fails to recognize or actively avoids opportunities for compassion

or empathy. On occasion demonstrates lack of

respect, or overt disrespect for patients, family members,

or other members of the health care team


Expert

Proficient

Competent

Advanced Beginner

Novice

Increase the Accreditation Emphasis on Educational Outcomes © 2012 Accreditation Council for Graduate Medical Education (ACGME)

Expert

Proficient

Competent

Advanced Beginner

Novice


Singapore End of PGY-1, Mid PGY-2 Year Evaluation, Overall Rating of Six Competencies across All Specialties

1

2

3

4

5

6

7

8

9

End PGY 1 Mid PGY 2

Professionalism

Communications

Medical Knowledge

Patient Care

Practice Based Learning and Improvement Systems Based Practice

n=122 paired observations


Expert

Proficient

Competent

Advanced Beginner

Novice


Singapore End of PGY-1, Mid PGY-2 Year Evaluation, Overall Rating of Professionalism across All Specialties

1.00

2.00

3.00

4.00

5.00

6.00

7.00

8.00

9.00

Y1 Professionalism Y2 Professionalism

n=122 paired observations (100%)


Expert

Proficient

Competent

Advanced Beginner

Novice


Singapore End of PGY-1, Mid PGY-2 Year Evaluation, Overall Rating of Patient Care and Technical Skills

across All Specialties

1.00

2.00

3.00

4.00

5.00

6.00

7.00

8.00

9.00

Y1 Patient Care and Technical Skills

Y2 Patient Care and Technical Skills

n=122 paired observations (100%)


Singapore Milestone Data, End of PGY 1 to Mid Year PGY 2 All Specialties (n=122, 100%)

1

2

3

4

5

6

7

8

9

Y1 Professionalism

Y2 Professionalism

1

2

3

4

5

6

7

8

9

Y1 Communication

Skills

Y2 Communication

Skills

1

2

3

4

5

6

7

8

9

Y1 Medical Knowledge

Y2 Medical Knowledge

1

2

3

4

5

6

7

8

9

Y1 Patient Care and Technical

Skills

Y2 Patient Care and Technical

Skills

1

2

3

4

5

6

7

8

9

Y1 Practice Based Learning

Y2 Practice Based Learning

1

2

3

4

5

6

7

8

9

Y1 Systems-Based Practice

Y2 Systems-Based Practice

Professionalism Communications Med Knowl Pt Care/Tech Sk PBLI SBP


The “Next Accreditation System”

Who Are We, and Why Are We Doing

What We Do?

Patient

“Client”

“Customer”

Traditions Contributing to the Moral and Ethical Practice of Medicine

Justice Based Equitable Distribution

of the “Good” of Health Care in

Society

John Rawls Paul Ricoeur

Powers and Fadden

Social Justice “System”

Medicine as a Moral Enterprise

“Physician as Moral Agent”

Aristotle Aquinas

Maimonides Pellegrino

Thomasma

Virtue Based Ethics as the Basis of

Medical Practice

Hippocratic Tradition

Evolution from Guild to Profession

Percival Gregory

Bacon Hume

The Virtuous Physician-

Character Based Driven by Principles: •Beneficence •Autonomy •Justice •Non-Malificence

Guiding Virtue: Effacement of Self Interest

Social

Contracts

Social Justice

Devolution to The Guild

Physician as “Service Provider”

Commercial Contract

Veatch

“Professional Behavior,”

not Character

Governed By

Rules and Regulations, Normative Behaviors

Professionalism Commitment to:

•Competency •Altruism •Public Trust •Self-Replicate

Voluntary Oath To Society

Percival, T. Medical Ethics, or, a Code of Institutes and Precepts: Adapted to the Professional Conduct of Physicians and Surgeons.

Manchester, UK: S. Russell;1803.

Nasca, T.J., Day, S.H., Amis, E.S., for the ACGME Duty Hour Task Force. Sounding Board: The New Recommendations on Duty Hours from the ACGME Task Force.

New England Journal of Medicine. 362 (25): e3(1-6). 2010. June 23, 2010.

The actions of the ACGME must fulfill the social contract, and must cause sponsors to maintain an educational environment that assures:

• the safety and quality of care of the patients under the care of residents today

• the safety and quality of care of the patients under the care of our graduates in their future practice

• the provision of a humanistic educational environment where residents are taught to manifest professionalism and effacement of self interest to meet the needs of their patients


Goals of The “Next Accreditation System”

• To begin the realization of the promise of Outcomes

• To free good programs to innovate

• To assist poor programs to improve

• To reduce the burden of accreditation

• To provide accountability for outcomes to the Public


2012-2020

We can’t afford to close any programs or positions

24.5% Reduction In Positions Estimated By DIO’s in 2011 Study

Nasca, T.J., Miller, R.S., Holt, K.D. The Potential Impact of Reduction in Federal Funding in the United States:

A Study of the Estimates of Designated Institutional Officials. Accreditation Council for Graduate Medical Education. www.acgme.org/acWebsite/home/Impact ReductionFederal GMEFundingTJN.pdf

JGME. 2011. 3(4):585-590.

Iglehart J.K. The Uncertain Future of Medicare and Graduate Medical Education. N Engl J Med 2011; 365:1340 – 1345

The “Next Accreditation System” in a Nutshell

• Continuous Accreditation Model – annually updated • Based on annual data submitted, other data requested, and

program trends

• Scheduled Site Visits replaced by 10 year Self Study Visit

• Standards revised every 10 years • Standards Organized by

• Structure • Resources • Core Processes • Detailed Processes • Outcomes


The Conceptual Change From…

The Current Accreditation System

Rules

Corresponding Questions

“Correct or Incorrect” Answer

Citations and

Accreditation Decision

Rules

Corresponding Questions

“Correct or Incorrect” Answer

Citation and

Accreditation Decision


The Conceptual Change To…

“Continuous” Observations

Assure that the Program Number of Potential

Fixed the Problem Related “Rules” Problems Promote

Innovation

Diagnose the Problem

If there is one!



Conceptual Model of Standards Implementation Across the Continuum of Programs in a Specialty

STANDARDS

Structure Resources

Core Process Detailed Process

Outcomes

Initial Accreditation

New

Programs

Structure Resources


Outcomes Withhold Accreditation

Withdrawal of Accreditation

2-4% 15% 75%

6-8%

Accreditation with Warning

New Programs,

Accredited Programs with Major Concerns

Probationary Accreditation

Structure Resources


Outcomes

Maintenance of Accreditation

Accredited Programs without

Major Concerns

Maintenance of Accreditation with

Commendation

Structure Core Process

Resources Detailed Process

Outcomes




Institutional Review Committee

Institution Oversight

Program

Program

Program

Program

Program

Program

Program

Program

Program

Program

Review Committee

Review Committee

Review Committee

Review Committee

Review Committee

Review Committee

Review Committee Review Committee

Review Committee

Review Committee

Institutional Visit Program – Patient Safety, Quality Improvement Supervision, Transitions in Care, Duty Hours

Trended Performance Indicators “6.5 of 8” Already in Place

Annual ADS Update Program Attrition – Changes in PD/Core Faculty/Residents Program Characteristics – Structure and Resources Scholarly Activity

Board Pass Rate – Rolling Rates Resident Survey – Common and Specialty Elements Clinical Experience – Case Logs or other Faculty Survey – Core Faculty Semi-Annual Resident Evaluation and Feedback

Milestones

Annual Sponsor Site Visit (CLER)


Challenges/Opportunities

• Culture Change and Faculty Development

• Program Directors, Designated Institutional Officials

• Faculty

• Review Committee Members

• “Retooling” of ACGME Infrastructure and Personnel

• The “Community of Educators” in each specialty has come together and agree on:

• core elements of the competencies

• levels of performance

• core methods of assessment


“Faced with the choice between changing one's mind and

proving that there is no need to do so,

almost everybody gets busy on the proof.”

John Kenneth Galbraith American Economist

“Somebody has to do something,

Jerry Garcia The Grateful Dead edits, TJ Nasca Gratefully Not Dead

and it’s just incredibly pathetic fantastic that it has gets to be us.”

Accreditation Council for Graduate Medical Education

Thank You!

the next accreditation system and sponsor site visit...

Documents