Surgical Learning – It’s Not Just PowerPoint Anymore

orWhat I did with my Winter Vacation

Michael E. Shapiro, M.D.General Surgery Program Director

Rutgers – New Jersey Medical SchoolHarvard – Macy Scholar

“The ideal college is Mark Hopkins* on one end of a log and a student on the other”**

*Williams College President 1836-72**James A. Garfield, Williams ‘1856

What we (now) need to learn/teach

An Experiment:

At the end of this talk, there will be a quiz.

The Basics - 1

• As an educator, what is our prime role?– To promote learning

• What is learning?– Change (in knowledge, beliefs, behavior) that is

sustained• What is teaching?– The act of promoting/catalyzing change– (ideally, done intentionally and informed by best

practices)

The Basics - 2

• What is assessment/feedback?– Measuring change – • To promote further learning

– Identify strengths and weakness to the learner– Learner centered

• To demonstrate competence– Patient safety– Entrustable professional activities (EPAs)– Certification– System centered

– Need balance of both

The Basics - 3

• Who is the assessment/feedback we do really for (i.e., who are the stakeholders)?– Learner (e.g., student, resident)– Teacher – is our teaching of value?– Regulator

• NJMS– Dean– GMEC

• ACGME• ABS

• How do their interests relate to promoting learning or insuring competence?

The Basics - 4

• Deliberate teaching = defining planned change, the developing appropriate strategy to implement it, and assessment/feedback to promote it and demonstrate it.

• As an educator, asking what do I/we need/want to change, what is the best way to accomplish that, and how would I know if I did?

This case is a work of fiction. Any similarities to any person, living or dead, is purely coincidental and unintentional.

Case Discussion – the “surprised” resident • Bob (not real name) is a PGY-3 surgical resident in

the middle of the year. He graduated from a well-known medical school, and has had excellent evaluations his first two years. The CCC has just met to review the PGY-3’s. The evaluations for Bob this year were very different, and quite negative. They all noted his failure to progress to a stage where he can lead a team, make diagnoses and plans, and make decisions for the patient. All stated they had provided this feedback to Bob.

Case - 2

• You meet with Bob as Program Director. You begin by asking him how he thinks things are going? He responds that he thinks things are going pretty well, in fact, “great.” He is happy in the program, and feels that, although the transition to third year was initially challenging, he now “has everything under control.” At first gently, then more firmly, you raise the concerns of his attendings.

Case - 3• Bob claims to have never heard any of this negative

commentary before. You try to express your concerns to Bob, who refuses to accept those assessments, and states that he “respectfully disagrees” and the faculty concerns are “unfounded and incorrect.”

• Now what do you do?• What do you suspect happened?• How do you reconcile the different perspectives?• Does this suggest anything about the need to track

real-time learning?

Adult Learning Theory

• Adult learning is different from children’s learning.

• Children learn for the joy of learning.– Some of us never grow up

• Adults learn for a purpose– Further educational goals– Gain skills that can be applied– Pass a regulatory exam

ALT - 2

• Prior knowledge is the foundation for new learning

• Adult learners should activate and build upon prior knowledge

• Increasing the links to prior learning is critical• Adult learners should be actively involved in

constructing individual educational goals and meanings – they want to learn.

ALT-3

• Adult learners should own their learning (intrinsic motivation) as opposed to primarily responding to their teachers/evaluators (extrinsic motivation). Intrinsic motivation is associated with deeper learning.

• Teaching adult learners should promote both learner autonomy and growth, ultimately making the teacher unnecessary.

• Our job is, ultimately, to become superfluous.

ALT-4

• So, to maximize learning:– Adult learners need to be actively involved in the

learning process, including:• Initial self-appraisal• Setting goals• Developing mastery• Participating in the assessment/feedback process

– Adult learners need to be active partners, not empty vessels to be filled by the teacher.

A Simple Model of Competence

Prof

essio

nal

Auth

entic

ity

Behaviour Does

Shows howKnows how

Cognition Knows

Miller, 1990 17

Spacing effect

Blocking vs. Interleaving in learning and retention

Judged Performance

Blocked Same Interleaved0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

Proportion of participants

Actual Performance

Blocked Same Interleaved0

0.10.20.30.40.50.60.70.80.9

Proportion of participants

Wood, W.B. and Tanner, K.D., CBE - Life Sciences Education, 2012, 11:3-9

Characteristics and Behaviors of Expert Tutors

• Intelligent– Superior content and pedagogical content knowledge

• Nurturant– Establish and maintain personal rapport and empathy

• Socratic– Provide almost no facts, solutions or explanations, but

elicit these by questioning• Progressive– Move from easier to progressively more challenging

cycles of diagnosis, solutions and new problems

Wood, W.B. and Tanner, K.D., CBE - Life Sciences Education, 2012, 11:3-9

Expert Tutors, cont’d

• Indirect– Provide both negative and positive feedback by

implication. Praise the solution, not the student• Reflective– Ask students to articulate their thinking, explain

their reasoning, generalize to other contexts• Encouraging– Use strategies to motivate students and bolster

their confidence

Translating what effective tutors do to the lecture hall

• Figure out where your students are starting• “Flip” the classroom– Provide the information to the learners to review

before the class – readings, podcasts, handouts• Be Socratic– Replace “telling” with “asking”. Allow learners to work

through problems together.• Avoid direct criticism – Use students to provide feedback to each other– Rather than identifying an answer as wrong, find

someone with the correct answer, and praise that

“Lecture” (2)

• Include “testing” as part of the session– Anonymous audience feedback engages learner

and shows where they fit without embarrassment.– Teaching and testing gives better retention than

teaching and repeating.– Real-time testing provides teacher with immediate

feedback about learning and areas of weakness– Opportunities (digital media) to make testing of

teams, competition generates excitement. Technology also permits re-testing for incorrect answers, reinforcement, etc.

Lecture (3)

• Foster metacognitive awareness– Have learners reflect on their thought processes,

articulate concepts they find difficult or troubling– Allow learners to “know what they know”– Encourage learners to identify which learning

strategies are most adaptive for them• Be supportive – not always a surgical tradition– “Why didn’t you just take out a gun and shoot him?”– “Your patient just died…”

A humble proposal to help “Bob” and all our Surgical learners

• ALT has shown that learners need to be responsible for their own learning.

• Learning occurs at the “Zone of Proximal Development”, i.e., the leading edge of the learners knowledge

• Learners require continuous and immediate assessment and feedback

• Teachers need to communicate with each other to do appropriate learner “hand-offs.”

Educational KanbanTime Frame Traditional Evaluation EK

Beginning of Rotation No formal meetingReview general objectives

Self-appraisalReview EK to date with supervisorReview rotation objectivesSet specific goals with superv. Integrating past experiences

Every month None Interim self-appraisal

During rotation None Update EKMandated performance feedbackReview and set new goals

Summative EvaluationEnd of rotation

Summative supervisor formNo continuity to next rotation

No summative formCollaborative self-appraisal and formative feedbackSet future specific goals

Goldman, SJ, The Educational Kanban: Promoting Effective Self-Directed Adult

Learning in Medical Education. Academic Medicine, 2009, 84:927-934

Goldman, SJ, The Educational Kanban: Promoting Effective Self-Directed Adult Learning in Medical Education. Academic Medicine, 2009, 84:927-934

Educational Kanban-2Time Frame Traditional Evaluation EK

Twice-yearly review Summative supervisor forms Program director review

Informal meetingNot part of formal review

Mentor Meetings None/irregular Review EK quarterly external to rotationContinuity/collaboration

Ownership Training Program FilesPermanent Record

Resident OwnedNot “permanent record”

Humble proposal-2

• Applying Goldman’s EK model to our surgical residency would require:– Milestone-based individual goals for each resident,

each rotation,– Collaborative assessment/feedback at beginning and

end of each rotation between faculty and each resident,

– Communication between faculty from one rotation to the next,

– More frequent interaction between residents and mentors.

Humble proposal-3

• Mostly, it will require:– Interest on the part of the faculty– Dedication of time to teaching and assessment by

faculty on each rotation at each hospital– Clear expectations from faculty to residents– Honest, frequent, real-time feedback

• Last I checked, all three of our (non-VA) hospitals have “University” in their names –

• Time to own up to that!

How many ELEPHANTS were there?

• A – 2• B – 3• C – 4• D – 2 and a Northern White Rhino• E – there were elephants? I thought the

Williams mascot was a purple cow!

Williams College, Class of 2015