joseph s. green, phd

1

JOSEPH S. GREEN, PHDMASSACHUSETTS MEDICAL SOCIETY

MAY, 2016

“When we commit to a vision to do something that has never been done before, there is no way to know how to get there. We simply have to build the bridge as we walk on it.”

Robert E. Quinn

USING ADULT LEARNING PRINCIPLES AND BACKWARDS PLANNING TO DESIGN CPD ACTIVITIES:

Improving Physician Performance and/or Patient Outcomes

2

“

”

PURPOSE:

TO PROVIDE COURSE CHAIRS, FACULTY AND HEALTH EDUCATORS THE INFORMATION NECESSARY TO MAKE ENHANCEMENTS TO THE LEARNING EXPERIENCES FOR CURRENT AND FUTURE CLINICIANS…

BECAUSE…the more effective the learning experience, the greater the likelihood that physicians will be able to use what they learn to provide better care to their patients

SECTION #1 [BEFORE]:‘FUNDAMENTAL LINKAGE AND BACKWARDS PLANNING’

Link learning experiences to healthcare quality measures and assure that planning starts with the end in mind

3

BEFORE PLANNING CPD ACTIVITY

Theoretical Foundations

1. Gaps in competence/performance

2. Learning needs

3. Motivation for learning

Program Planning-Practical Applications

A. Link to healthcare outcomes

A. Define learning goals and objectives

B. Develop measurement strategies

THEORETICAL FOUNDATIONS

4

ADULT LEARNING

Pedagogy: Teacher-centered learning for children

Andragogy: Self-directed learning for adults

Fluid intelligence: making new neural connections without any base (children)

Crystallized intelligence: new learning grows like crystals on existing knowledge

8

HOW PHYSICIANS LEARN

A physician will typically go through several stages in response to a “problem” in practice.

Recognizes that there is an issue with performance

Takes ownership, accepts opportunity for improvement, searches for resources

Commits to learning and learns

Tries out what was learned

Incorporates what was learned where appropriate

5

MULTI-STAGE PHYSICIAN CHANGE PROCESSROGERS, 1983

Physician learns about innovation

Is persuaded to think about it

Decision to try the change is made

Confirms the change was appropriate

Continues to use it

10

HELPING PHYSICIANS LEARN

What is going on in the mind of the physician who is going through the stages just described?

He or she is dealing with “cognitive dissonance”

Recognizing the gap between “what is” and “what should be”

Feeling discomfort with the “gap”

Searching for resources to reduce or eliminate the “gap”

The CME planner needs to help the physician who is going through these stages to learn more effectively and efficiently

6

ORGANIZING LEARNING—PHYSICIAN READINESS TO LEARN

Instructional Framework

Predisposing activities

Enabling activities

Reinforcing activities

Using these activities was associated with effective CME

THE PLANNING PROCESSOLD AND NEW

7

Decide on topic

Location

Select faculty

Faculty select content

Put content into lectures and panels

Assess success

#’s, $$, happiness

PREVIOUS PLANNING MODEL

14

EFFECTIVE CE/CPD

Effective (dictionary definition):

having an intended or expected effect

designed to produce the desired outcome or result.

Effective CE/CPD is accomplished by planning for outcomes or results that are intended or desired.

8

Identify gaps in physician performance

Measure self-perceived gaps in learner competence

Delineate desirable outcomes for learning intervention (objectives) based on gaps

Create content needed to satisfy objectives

Pick most effective methods to meet objectives

Select best expert faculty to provide content

Determine the success of the activity in relation to desirable outcomes

NEW PLANNING MODEL

BACKWARDS PLANNING FOR OUTCOMES

Problems/Opportunities in healthcare

Health professions involved

Desired outcome measures of an intervention

Community Health

Physician Performance

Physician Competence

Outcomes based objectives

Knowledge

Skills

Attitudes

9

LEVELS OF PHYSICIAN LEARNING

17Miller, 1991

LEVELS OF OUTCOMES FOR CPD: TARGET OUTCOMES AT LEVELS 4-6!

Participation (1)

Satisfaction (2)

Learning Knows (3A)

Knows how (3B)

Shows how (4)

Performance (5) Patient Health (6) Community health (7)

10

UNDERSTANDING THE PROBLEM/OPPORTUNITY GAPS

IT IS ALL ABOUT THE “GAP”!

• The difference between What is and…

• What ought to be

• What could be

• What is desired

• What peers are doing

• As it relates to…

• What a learner knows (knowledge)

• What a learner is capable of performing (competence)

• What a learner actually does in their practice (performance)

11

What is

NEEDS & GAPS

What ought to be

DESCRIBING PROBLEM/OPPORTUNITY GAPS

• Problem—physicians not performing according to professional standards

• Opportunity—physicians performing at standards, but could be performing at higher level because of new developments

Problem/opportunity caused by lack of:

* Knowledge

* Skills

* Attitudes

“How are physicians performing against standards of care and do we have a problem or opportunity?”

12

DETERMINING CAUSES OF GAPS

• Use literature review, surveys or focus groups to understand why gap exists

• Does gap exist at least partially because physicians don’t know or understand something and can it be defined in terms of knowledge, skills or attitudes?

• Is gap caused primarily by other issues such as systems problems, lack of resources, cultural differences, reimbursement issues?

“How do we know that the gap will lend itself to an educational solution?”

IDENTIFYING BARRIERS AND STRATEGIES

• Gather data on why physicians are not practicing at highest possible level

• Clearly describe barriers to performance

• Find examples of successful strategies to get around barriers

• Use surveys or focus groups to understand dynamics of practice setting

“How do I understand why physicians aren’t performing at an optimal level?”

13

UNDERSTANDING THEIR OWN “GAPS” AS A LEARNERS

• Lets learners know what they don’t know

• Lets learners know how their clinical performance

compares to guidelines and to how their peers

are performing

• Provides them with the motivation to learn

(closing the gap)

IDENTIFYING THE TARGET AUDIENCE(S) AND ENHANCINGLEARNER MOTIVATION

14

LEARNER PROFILES:“WHO ARE THE LEARNERS?”

Identify important demographics such as age, sex, years since residency, specialty

Determine primary motivator for attending (e.g. content, location, credit, faculty, sponsor)

Assess primary learning needs before and after activity

Remain flexible—adapt to identified needs

• Physicians are most motivated to learn when they compared to standards or peers

• Importance of and performance gaps and

selecting/ designing appropriate learning experiences

• Increased use of

15

PROGRAM PLANNING--PRACTICAL APPLICATIONS

30

WHAT SHOULD CE PLANNERS DO?

Examples of Predisposing CE Activities Providing feedback from audit and performance

improvement

Providing information about practice guidelines

Providing information about clinical trials

16

IDENTIFYING STANDARDS OF CARE

Specialty Societies

Governmental agencies (AHCPR)

Institutional guidelines

Regional agencies

Literature searches

“Are there appropriate and relevant standards of care that guide physician clinical behavior?”

NEEDS ASSESSMENT: METHODS

Focus groups

Telephone/personal interviews

Questionnaires

Practice pattern analysis

QI/QA data analysis

Activity evaluations

Literature reviews

Analysis of new developments in dx & rx

Question cards

ARS polling

17

PRE-COURSE

What are the healthcare issues that need to be solved through learning activities?

Who has competency/performance issues and learning needs?

What are the learner expectations based on the identified learning needs?

What are the demographics of the target audience?

Do the learners know what they don’t know?

PRE-POST ATTITUDE INVENTORY:“WHAT DO THE LEARNERS FEEL?”

Identify attitudes critical to learning material or skills

Use 5-point Likert scales

Provide feedback to learners of typical (average) response

Show changes at completion of activity to reinforce learning

18

PRE-POST TEST OF KNOWLEDGE:“WHAT DO THE ACTUAL LEARNERS KNOW ?”

Select most important concepts to be learnedProvide immediate feedback to learnerAssure results are known only by learnerAllow learners to compare results with peersTest for application of knowledge in real world settingUse same test items for post-test (or pick from same pool of

questions)Use multiple choice questions to assure learner can make

fine discriminations

IMPLICATIONS FOR FACULTY/COURSE CHAIRS/STAFF

Learners need to know what they don’t know

Show learners comparative data of colleagues or standards of care

Base planning of learning experience on solving competency/performance issues of learners

Start the planning with the end in mind

19

SECTION # 1 :SMALL GROUP TASKDiscuss Program Planning Process

A.How would you assess healthcare or performance ‘gaps’?

B. How would you tie these gaps to health professional competence or performance needs?

C.What would you need to ‘Define outcomes, goals, objectives’ and target audience?

D.What are your thoughts about how to ‘Develop measurement strategies’?

SECTION #2 [DURING]:‘CONSTRUCTIVE ALIGNMENT’

Design and implement effective/measureable learning activities and modify, if necessary, to meet learner needs and to solve healthcare problems

20

DURING ACTIVITY


1. Learner engagement

2. Relevance and translation to practice setting

3. Role of feedback

4. Verifiable outcomes through constructive alignment

Program Planning– Practical Applications

A. Identify teaching methods

B. Define delivery formats

C. Develop content

D. Establish Measurement & Evaluation tools

E. Implement formative assessments


21

IMPACT OF FORMAL CMEDave Davis, MD, et al JAMA, 1999

Traditional, formal CME (lectures) failed to success in changing performance or health care outcomes

Those using interactive techniques (case discussion, role-playing, hands-on practice sessions) were more effective

WHAT IS THE DESIRED OUTCOME OF MOST OF THE LEARNING EXPERIENCES YOU MIGHT HELP CREATE? WHAT SHOULD IT BE?

• Dissemination of informationOR

• Impacting knowledge, competence

or performance of your colleagues

22

PEPF

Presentation

Examples

Practice

Feedback

D.Merrill

WHAT OBJECTIVES ARE

:

A description of what the faculty is going to discuss

A division of the content of the talk or course

23

WHAT OBJECTIVES SHOULD BE:

Focus on learner perspective

Emphasize measurable outcomes in relation to knowledge, skills and/or outcomes

WHY OBJECTIVES ARE IMPORTANT

• Objectives need to follow understanding of learner needs and precede selection of content, faculty and methods…

• They help planners articulate what they are trying to accomplish and enhance the chances of succeeding

• They identify gaps in knowledge, skills and attitudes that need to be filled

24

ALIGNING PLANNING AND PHYSICIAN LEARNING

47

IS EDUCATION ALWAYS ENOUGH TO CHANGE PHYSICIAN BEHAVIOR?

Education is not always adequate to facilitate physician behavior change

Other non-educational strategies:Patient education to facilitate compliance

Reminder systems for drug-drug interactions

Use of specific treatment algorithms

Patient feedback of effectiveness of physician

communication

Methods to study practice patterns

25

HOW SHOULD BARRIERS TO LEARNING BE ADDRESSED?

Must first assess what the barriers are that prevent physicians from using the information provided in CME activities

Identify strategies to overcome barriers

Include possible barriers and strategies in content to supplement clinical content

Discuss what it will take to use the clinical information provided in CME activities

across multiple formats needed to ensure appropriateness for different learners

Provide of learning by using different formats over time

available to facilitate learning

Match content to and learning style preferences

26

INTERACTION WITH WHOM OR WHAT?

Content

Faculty

Peer Learners


27

53


Examples of Enabling CE activities

Sequence

Lecture

Demonstration

Practice

Feedback

Recall

Case Discussion

INNOVATIVE FORMATS AND METHODS FOR CE/CPD ACTIVITIES

Distance Learning

Quality/physician performance

Academic detailing

Patient simulators

Simulated patients

Web-based, e-learning

Blended learning

ARS and case studies

Hands-on skill building

Just-in-time training

Practice-based learning

Point-of-care learning

Chart stimulated recall

28

Small Group Discussion

LIVE PROGRAM LECTURES

29

PANEL DISCUSSIONS

HANDS-ON SIMULATION

30

CRITERIA FOR SELECTING THE MOST APPROPRIATE FORMATS AND METHODS

Nature of educational need

Outcomes-based learning objectives

Content

Learning style preference of learners

Practice-based realities of target audience

Cost

Potential for impact

DURING THE COURSE--FORMATIVE EVALUATION

HOW ARE WE DOING IN MEETING OUR COURSE OBJECTIVES? MODIFICATIONS NECESSARY?

• Paper and pencil

• Focus groups

• External evaluators

• ARS

• Faculty feedback

31

FOCUS CONTENT ON CLINICAL PROBLEMS

Summarizing information contained in recent research publications

Comparing personal performance with peers

Information about personal outcomes with patients and comparing to standards of care

Seeking colleague resources

Tools to help integrate into practice

RELATIONSHIP OF CONTENT COMPONENTS

C: Continuity ─ Continuity provides learners the opportunity to revisit knowledge and skills in more depth as they progress through an educational activity.

S: Sequence ─ The sequence in an educational plan focuses on the order in which things occur: from simple to complex learning, from known to unknown.

I: Integration ─ Integration is concerned with the linkages of information to enable learners to develop a holistic overview.

32

CONCLUSION OF COURSE—SUMMATIVE EVALUATION

Overall course evaluation

Knowledge gain/enhanced competence

Expectations of learners to use information to improve practice

Learner commitment to change

Suggestions for improvement

VERIFIABLE OUTCOMES THROUGH CONSTRUCTIVE ALIGNMENT

Alignment of

Healthcare problems and patient outcomes

Appropriate target audiences

Health professional gaps in competence/performance

Outcomes based learning objectives

Decisions on:

content

formats

methods

technologies

faculty

evaluation methodologies

Provide follow-up reflection and learning reinforcement

Implement measurement and outcome studies

Share results with all involved audiences

33

SECTION # 2 :SMALL GROUP TASK

Discuss Program Planning ProcessA. On what basis wouId you ‘identify best

teaching methods’?

B. How would you ‘Define most effective delivery formats’?

C. Who and when should you ‘Develop content’?

D. Who has the skills in your organization to ‘Establish M&E tools’?

E. Why do you need to ‘Implement formative assessments’?

SECTION #3 [AFTER]:‘APPLICATION OF LEARNING TO PRACTICE—VERIFIABLE OUTCOMES’

Provide adequate time for learner reflection and evaluate whether objectives were met and desired outcomes were achieved

34

AT CONCLUSION OF AND AFTER THE ACTIVITY


1. Continuous assessment

2. Opportunities for reflection

3. Feedback

4. Reinforcement

Program Planning—Practical Applications

A. Implement measurement and evaluation tools

B. Allow for reflection

C. Provide summative feedback to all audiences

D. Commitment to change


35

FEEDBACK

Seven principles of good feedback practice:

It clarifies what good performance is (goals, criteria, expected standards)

It facilitates the development of self-assessment in learning

It provides high quality information to students about their learning

It encourages teacher and peer dialogue around learning

It encourages positive motivational beliefs and self-esteem

It provides opportunities to close the gap between current and desired performance

It provides information to teachers that can be used to help shape teaching.

CONTINUOUS ASSESSMENT

Needs assessment

Formative assessment

Summative assessment

36

COMMITMENT TO CHANGE

Measurements of physicians’ intent to change taken at time of CE activity and after time has passed in the practice setting

P.Mazmanian

POST-COURSE FOLLOW-UP—SUMMATIVE ASSESSMENT

Improvements made in practice

Barriers to using information

Additional learning needs

Other suggestions for enhancements to course

37


74


Examples of Reinforcing Activities

Reminders

Commitment to Change

Practice Portfolios

Feedback

Post-course Materials

Peers and Opinion Leaders

Communities of Practice

38

END-OF-COURSE EVALUATION QUESTION

Course Objective #1: Understand and apply the concepts of effective treatment of traumatic injuries to the foot and ankle.

A) I did not learn anything new.

B) I confirmed that what I was doing was appropriate.

C) I learned something new, but I do not want to use it in my practice.

D) I learned something new that I want to use, but I have not yet been able to do so.

E) I learned something new and have used it in my practice.

76

DESCRIBING OUTCOMES/RESULTS

Participation: how many attended?

Satisfaction: did they like it?

Knowledge: did anybody learn?

Performance: did behavior change?

Patient health: did it improve?

Population health: did it improve?

adapted from Kirkpatrick,1998; Walsh,1984; Dixon,1977; Moore, 1998

39

SOURCES OF DATA—LEVELS OF OUTCOMES

• Participation

• Satisfaction

• Learning

• Performance

• Patient Health

• Community Health

• Attendance records

• Questionnaires

• Pre/post tests• Expert Observation• Self Report

• Patient charts/observation• Self Report

• Patient charts• Self Report

• Epidemiology data

SELECTION OF MOST APPROPRIATE ASSESSMENT METHODS

Mix and match methods to fit the information needs of course:

Paper and pencil questionnaire

ARS

Focus groups

External expert evaluators

E-mail surveys

Opinions/cases/cognitive test items

“If you are not going to use it, don’t ask it”

40

ASSESSING ACCOMPLISHMENT OF DESIRED RESULTS THROUGHOUT THE LEARNING EXPERIENCE

Assessing Physician Performance: screening evaluation detection diagnosis prevention management plan prescribing follow-up

Did the clinical performance of physician participants improve?

Self report questionnaires to physician or patient Observation on learning environment Case scenarios (ARS) Small group work Standardized patients

ASSESSING ACCOMPLISHMENT OF DESIRED RESULTS THROUGHOUT THE LEARNING EXPERIENCE

Patient Health Status: morbidity and mortality physiologic measures clinical events

Did the health status of the physician’s patients change as a result of participation in the CME event?

patient health record administrative data self-report questionnaires from chart reviews

41

OPPORTUNITIES FOR PRACTICE AND FEEDBACK

Goal not retention of facts, but application of what was learned into practice setting

Planners need to create authentic settings that engage learner in complex, realistic and “messy” clinical problems

Actively involve learners in own learning

Provide opportunities to interact with colleagues

Provide learners with feedback on performance

Moore DE, Green JS and Gallis HA, Achieving Desired Results and Improved Outcomes: Integrating Planning and Assessment Throughout Learning Activities, Journal of Continuing Education in the Health Professions, 29(1):1-14, 2009.

Fox R. and Miner C., Motivation and the Facilitation of Change, Learning, and Participation in Educational Programs for Health Professionals. Journal of Continuing Education in the Health Professions Volume 19, Number 3, Summer 1999

Fox, R. D., "Discrepancy Analysis in Continuing Medical Education: A Conceptual Model." Mobius, Vol. 3, No. 3, 1983, pp. 37-44.

De Boer, P.G. and Green, J.S.(editors), AO Principles of Teaching and Learning, AO Publishing, Thieme, Switzerland, December, 2004.

Moore DE, Jr. Needs assessment in the new health care environment: combining discrepancy analysis and outcomes to create more effective CME. J Cont Educ Health Prof. 1998; 18.3:133-141.

REFERENCES

42

SECTION # 3 :SMALL GROUP TASK

Discuss Program Planning ProcessA. What are the most important aspects of

‘Implementing measurement and evaluation tools’?

B. How would you ‘Allow for reflection’?

C. To whom would you ‘Provide summative feedback’?

joseph s. green, phd

Documents