Teaching Medical Students to Reflect “Deeper” Ilho Kang MD, Amy Hayton M.D, Raymond Wong MD, and Lawrence Loo MD Department of Medicine, Loma Linda University School of Medicine, Loma Linda, CA

Discussion

Conclusions

References

Introduction

While many studies have examined the importance of reflective writing in

medical education[1,2], there is a paucity of research evidence for any particular

intervention to improve the quality or “depth” of reflection among medical

students. To facilitate the development of critical reflection, a new curriculum

enhancement was introduced during the third year of medical school.

1. Plack MM, Greenberg L. The reflective practitioner: reaching for excellence in

practice. Pediatrics. 2005 Dec;116(6):1546-52

2. Stern DT, Papadakis M. The developing physician--becoming a professional. N

Engl J Med. 2006 Oct 26;355(17):1794-9.

3. Aronson L. Twelve tips for teaching reflection at all levels of medical education.

Med Teach. 2011;33(3):200-5

4. Mann K, Gordon J, MacLeod A. Reflection and reflective practice in health

preofessions education: a systemic review. Adv Health Sci Educ 2009;14:

695-621.

5. Wald HS, Borkan JM, Taylor JS, et al.: Fostering and evaluating reflective

capacity in medical education: developing the REFLECT rubric for assessing

reflective writing. Acad Med 2012;87:41-50.

6. Landis, J.R., Koch, G.G. (1977). The measurement of observer agreement for

categorical data. Biometrics. 33, 159–174.

In the initial analysis, our 90 minute educational intervention

successfully improved the depth of reflection by increasing the number

of students who wrote level four “critical reflection” papers. We believe

this short curriculum enhancement could readily be adapted by others

seeking to enhance student reflection and foster lifelong learning.

To our knowledge this is the first study to demonstrate that an

educational intervention to teach critical reflection improves the depth of

reflection by medical students.

•Strengths: The number of faculty evaluators and their successful

blinding that minimized bias in grading. The percentage agreement in

distinguishing “old” vs. “new” is similar to what one might expect by

chance (i.e. 50%) suggesting the authors were not able to distinguish

“old” versus “new” papers.

•Limitations: Low inter-rater reliability in using the REFLECT rubric.

Kappa statistic was used in our analysis and our value of 0.37 indicates

only “fair agreement”. This may limit the applicability of our results to

other institutions.

•Further Study: Development of a rubric that is more easily adaptable

and reliable may be valuable in future studies on reflection. We are

exploring the impact of an open-ended versus more directed prompt

on the depth of critical reflection. We are also considering whether

reflection level of students changes over the course of the academic

year.

In June 2012, a 90-minute interactive teaching session[3,4] was introduced to

all junior medical students rotating through the Internal Medicine clerkship at

Loma Linda University. Key components included:

Methods

The primary analysis showed that the number of students writing “critical

reflection” papers increased after the educational intervention from 19% (16/83)

to 43% (39/90) p = 0.0002 (Figure 1), using the Mann-Whitney U test. Inter-rater

reliability among the four faculty graders demonstrated only “fair” agreement, as

measured by a kappa statistic = 0.37[6].

Figure 1. “Critical reflection” pre vs. post new curriculum

Results

Table 1. REFLECT RUBRIC[5] Criteria used to judge reflection papers

Reflection Level Non-Reflective

1

Thoughtful Action

2

Reflection

3

Critical Reflection

4

Writing

Spectrum

Superficial descriptive

writing approach (fact

reporting, vague

impressions)

Elaborated descriptive

writing approach and

impressions without

reflection

Attempting to

understand, question,

or analyze the event

Exploration and critique

of assumptions, values,

beliefs, and/or biases

and the consequences

of action

Sense of Writer

Presence Writer partially present Writer partially present

Writer largely or fully

present

Writer largely or fully

present

Description of Conflict,

Disorienting Dilemma,

Challenge or Issue of

Concern

No description Absent or weak

description Description

Full description that

includes multiple

perspectives, exploring

alternative explanations,

and challenging

assumptions

Attending to

Emotions

Little or no

recognition or

attention to

Recognition but no

exploration or

attention to

Recognition,

exploration, and

attention to

Recognition, exploration,

attention to and gain of

emotional insight

Analysis & Meaning

Making None Little or unclear Some Comprehensive

Reflection level Non-reflective

1

Thoughtful

Action

2

Reflection

3

Critical

Reflection

4

Summary

Pre-

Intervention Number (%)

2

(2.4%)

30

(36.1%)

35

(42.6%)

16

(19.3%)

83

(100%)

Post-

Intervention Number (%)

2

(2.2%)

12

(13.3%)

37

(41.1%)

39

(43.3%)

90

(100%)

Column Total Number (%)

4

(2.3%)

42

(24.3%)

72

(41.6%)

55

(31.8%)

173

(100%)

Table 2. Summary of total written reflection papers (N=173)

The grades for the reflection papers prior to and after the introduction of the

curriculum enhancement are summarized in Table 2 & Figure 1.

The four faculty were asked to judged whether each reflection paper was “old”

(written prior to educational intervention) or “new” (after the intervention). The

proportion agreement = 44% and p > 0.05. The poor kappa statistic of 0.18

among the 4 faculty graders also suggested adequate blinding of this process.

Eighty three reflection papers written prior to the intervention were compared to

ninty reflection papers after the intervention. To minimize bias and blind the four

faculty graders, names and dates were removed from each typed reflection

paper. Differences in grading were discussed and resolved by consensus.

Four minute

video from the popular TV show

Scrubs, highlighting a

young doctor’s reflection

experience of a patient facing

death.

Four minute

video from the popular TV show

Scrubs, highlighting a

young doctor’s reflection

experience of a patient facing

death.

Large group discussion of the key concepts

and core components of

critical reflection.

Large group discussion of the key concepts

and core components of

critical reflection.

Small group interactive

exercise where students

compared and contrasted three

essays portraying different levels of

reflection.

Small group interactive

exercise where students

compared and contrasted three

essays portraying different levels of

reflection.

A faculty presenter shared

a personal reflection

critical to her own professional

development.

A faculty presenter shared

a personal reflection

critical to her own professional

development.

Students were then asked to write two reflection papers

during the 10-week rotation. A previously developed and

validated tool, the REFLECT rubric (Table 1)[5], was used to

grade the written reflection papers on four levels.

p = 0.0002

N

u

m

b

e

r

Level of Reflection5

Source: https://www.mededportal.org/icollaborative/resource/352