st annual education day - virginia tech

1st Annual Education Day

October 28, 2015

1

The mission of TEACH is to promote learning excellence at Carilion Clinic, Virginia Tech Carilion School of Medicine, Virginia Tech Carilion Research Institute, and Jefferson College by creating a community of educators and fostering their development as

teachers, learners, and education researchers.

2

Join TEACH now!

3

medicine.vtc.vt.edu/teach

4

Sarah Bean, MD

Looking in the Mirror: self-reflection, self-assessment, and direct observation

Sarah M. Bean, M.D.Associate Professor, Department of PathologyCytopathology Fellowship Program DirectorSurgical Pathology Fellowship Program DirectorClinical Competency Committee Chair

Learning Objectives

1. Define self-regulated learning

2. Discuss the challenges of self-assessment

3. Identify barriers to direct observation

4. Understand the importance of self-reflection

Self-Regulated Learning

• Epistemology

• Learning styles

• Strategies

• Principles/methods

• Self-assessment

• External feedback

• Goal setting

• Motivation

• Self-reflection

• Attribution

Adjustment Planning

LearningAssessment

Zimmerman BJ. Self-regulation of learning and performance : issues and educational applications. Hillsdale, NJ: Earlbaum, 1991:3-21.

Competency-Based Education

Competency-Based Education

Gruppen LD et al. Human Resources for Health. 10:43, 2012.


• Epistemology

• Learning styles

• Strategies


• Self-assessment


• Goal setting

• Motivation

• Self-reflection

• Attribution

Adjustment Planning

LearningAssessment



• Metacognition

• Strategic action

• Motivation to learn

Learner Differences

Learner Differences

When compared to high performers, low performing learners have:

Task valueSelf-efficacy beliefs

Course-related anxietyFrustrationBoredom

Artino AR et al. Acad Med. 86(10):S35-S38, 2011.


• Epistemology

• Learning styles

• Strategies


• Self-assessment


• Goal setting

• Motivation

• Self-reflection

• Attribution

Adjustment Planning

LearningAssessment


Planning: Goal Setting

Self-efficacy, an individual’s belief in his or her ability to achieve specific goals, is critical to setting and achieving challenging yet attainable goals.

Planning: Motivation and Self-Determination Theory

Ryan and Deci. Am Pscychol. 55(1):68-78, 2000.


• Epistemology

• Learning styles

• Strategies


• Self-assessment


• Goal setting

• Motivation

• Self-reflection

• Attribution

Adjustment Planning

LearningAssessment


Learning

• Epistemology (beliefs about knowledge and knowing)

• Learning styles (innate characteristics possessed by learners that underlie and influence their learning)

• Learning strategies

• Principles and methods


• Epistemology

• Learning styles

• Strategies


• Self-assessment


• Goal setting

• Motivation

• Self-reflection

• Attribution

Adjustment Planning

LearningAssessment


Assessment: Internal and External Feedback

What is Self-Assessment?

self-as·sess·ment

• assessment or evaluation of oneself or one's actions and attitudes, in particular, of one's performance at a job or learning task considered in relation to an objective standard.

www.google.com

Self-Assessment

• Identify learner needs

• Influence learning activity

• Change clinical practice

• Improve patient outcomes

• Be accurate

Colthart I et al. Med Teach. 30:124-145, 2008.

Kruger and Dunning: Is it funny?

Kruger J & Dunning D. J of Personal and Soc Psychol. 77(6):1121-1134, 1999.

Kruger and Dunning: Logical Reasoning

Kruger J & Dunning D. J of Personal and Soc Psychol. 77(6):1121-1134, 1999.

Self-Assessment

• Physicians have limited ability to accurately self-assess.• Davis et al. JAMA 296(9):1094-1102, 2006.

• Colthart et al. Med Teach. 30:124-145, 2008.

• Regehr G et al. Acad Med. 71(10):S52-54.

• The most confident are the weakest self-assessors.• Eva KW and Regehr G. Self-assessment in the health professions: reformulation and research agenda. Acad Med. 80(10):S46-54.

Contributing factors

• Motivation

• Metacognition

• Selective recall of past behavior

• Tendency to ignore proficiency of others

• Purpose of the self-assessment task (how is it used?)

Improving Self-Assessment

• Provide explicit assessment criteria

• Benchmarking

• External feedback (verbal, video)

Ward et al. Am J Surg. 185: 521-524, 2003.


• Epistemology

• Learning styles

• Strategies


• Self-assessment


• Goal setting

• Motivation

• Self-reflection

• Attribution

Adjustment Planning

LearningAssessment


Importance of Feedback

“Without feedback, mistakes go uncorrected, good performance is not reinforced, and clinical competence is achieved empirically or not at all.”

- J. Ende, 1983

Ende J. Feedback in Clinical Medical Education. JAMA. 250(6):777-781, 1983.

Guidelines for Giving Feedback

• Teacher and learner are allies

• Timely, expected

• Based on direct observation

• Focused on specific behaviors

• Includes positive behaviors

• Limited to behaviors amenable to change

• Descriptive, constructive

• Quantity is regulated

Direct Observation

• Strongly associated with the perception of accurate knowledge of a learner’s performance

• Perception of accurate knowledge of a learner’s performance is associated with increased feedback

Mazor KM et al. Acad Med 86(10):S63-S68, 2011.

Importance of Direct Observation

Miller GE. Acad Med. 65:s63-s67, 1990.

What is Direct Observation?

Direct Observation

Kogan JR et al. JAMA. 302(12):1316-1326, 2009.

Direct Observation: Does it happen?

• NBME survey reported students were observed more often by a resident than by a faculty member

• 20% indicated that had been observed by a faculty 0-2 times

Direct Observation: does it happen?

Rotation History-Taking Focused Physical Complete Physical

Internal Medicine 59% 51% 74%

Surgery 74% 71% 91%

Pediatrics 49% 47% 77%

Psychiatry 27% 71% 87%

Family Medicine 26% 25% 70%

Ob-GYN 68% 61% 88%

Howley L & Wilson W. Acad Med. 79(3):276-280, 2004.

What are the Barriers to Direct Observation?

Direct Observation

• Develop policies that encourage and reward direct observation

• Develop education champions

• Evaluate the process

Holmboe ES. Acad Med. 79(1):16-22, 2004.

Direct Observation

• Define objectives to guide DO

• Formative or summative?

• Use an existing tool

• Culture change

• Faculty development

• Give feedback

Hauer KE et al. Med Teach. 33:27-33, 2011.

• Make a learner action plan

• Orient learners

• Use the tool multiple times per trainee

• Measure outcomes

Hauer KE et al. Med Teach. 33:27-33, 2011.

Direct Observation


• Epistemology

• Learning styles

• Strategies


• Self-assessment


• Goal setting

• Motivation

• Self-reflection

• Attribution

Adjustment Planning

LearningAssessment


Self-Reflection

• Analyzing, questioning, and reframing an experience for the purposes of learning

– Aronson L. Med Teach. 33:200-205,2011.

• ‘recapture their experience, think about it, mull it over and evaluate it’

– Boud, Keogh and Walker (1985)

Self-Reflection

• Talking

• “How did it go?”

• Root cause analysis

• Journal entry/writing

• Story telling

• Self-reflection exercises

• Be creative (art, twitter, dreams)

Kolb’s Experiential Learning Cycle

Self-Reflection

• Adds meaning to complex situations

• Enables experiential learning

Mann K et al. Adv in Health Sci Educ. 14:595-621,2009.

Encouraging Self-Reflection

• Role model the behavior

• Establish an educational alliance with learner

• Facilitate reflection

Branch & Paranjape. Acad Med. 77(12):1185-1188, 2002.

Setting the Stage

Link past, present, and future

Integrate cognitive and emotional

Consider multiple perspectives

Reframe

Describe lessons learned

Plan for future learning

Aronson L. Med Teach 33(3):200-205,2011.

Aronson L. Med Teach 33(3):200-205,2011.


• Epistemology

• Learning styles

• Strategies


• Self-assessment


• Goal setting

• Motivation

• Self-reflection

• Attribution

Adjustment Planning

LearningAssessment


Conclusions

• Self-regulated learning is the theoretical foundation of competency based education.

• Physicians do NOT accurately self-assess.

• External feedback with direct observation is the check and balance of SRL.

• Self-reflection enables experiential learning.

Discussion

st annual education day - virginia tech

Documents