ratl module 4: evaluation and feedback module created by: charlene m. dewey, m.d., m.ed., facp...

Module 4:

Evaluation and Feedback

Module 4:

Module Created by:

Charlene M. Dewey, M.D., M.Ed., FACP Associate Professor of Medicine

Teri L. Turner, M.D., M.Ed., MPHAssociate Professor of Medicine

Joan Friedland, M.D., MPHAssociate Professor of Medicine

Web Page and Module Formatting by:

Maria Victoria Tejada-Simon, Ph.D.Assistant Professor of Molecular Physiology and BiophysicsR

Module 4:2 of 137

Created by Culberson, Dewey, Ismail, Friedland, Tejada-Simon & Turner. NIH Funded Relationship-Centered Transformation of Curricula, Baylor College of Medicine, 2006 .

Welcome

This is Module 4: Evaluation and Feedback. You should have completed a pretest for this module. Did you complete the “honesty pledge” question?

In that pledge you agreed to take the pretest first, then read the module and then take the post-test after reading the module. If you did not complete the pretest, please exit the module now and complete it; then return to the module. Your honesty is appreciated.

Click here to continue.Exit

Module 4:3 of 137

Welcome

By now you are familiar with the general format of the RATL modules. This module also consists of a pre-assessment, content of the topic and a post-assessment tool.

There are 136 total slides; many are games and interactive activities. Overall it should take about 45-60 minutes to complete this module.

Module 4:4 of 137

Welcome

However, at the completion of Module 4 we will ask you to complete the Teacher Identity Scale for the second time.

Please remember to provide feedback on the module content and format by completing a module 4 evaluation form found on the main RATL web page.

Module 4:5 of 137

Introduction

So what are you going to say about these teaching modules? First, you need to compare them to something that is known to be a good example or a gold standard. Then you must decide if they were as good as the gold standard and why. Then you will have to organize your thoughts to provide us with feedback that can help us improve the modules for the future. Lastly, you must convey those thoughts by writing an evaluation.

Module 4:6 of 137

Introduction

If you do all that and then never send us your comments, we will continue to think these modules are helpful in teaching residents how to teach. If they are not, and you don’t let us know, then this is an educational tragedy!

You do not want to spend a week, a month or a semester teaching learners and never let them know if they made the grade. That would truly be tragic!

Module 4:7 of 137

Introduction

What we are talking about is assessments, feedback and evaluations. All are key concepts in teaching and advancing a learners’ performance. It is part of the teacher identity and done correctly will elevate learners to new levels of knowledge, skill and attitudes.

Module 4:8 of 137

Introduction

Module 4: Evaluation and Feedback focuses on accurately assessing a learner's level of ability, providing effective feedback and writing appropriate evaluations.

Your role as a teacher will be enhanced by adapting appropriate assessment, evaluation and feedback principles.

Module 4:9 of 137

Our goals for Module 4 are that participants will review:

1. General concepts of learner assessment

2. ACGME core competencies

3. How to provide effective feedback

4. Documenting written evaluations of learners

Module 4:10 of 137

Objectives

By the time you complete this module, you should be able to:

1. Explain the difference between formative and summative feedback and evaluation.

2. List the 6 ACGME core competencies.

3. List at least three characteristics of effective feedback.

4. Listen to examples of feedback.

5. Discuss the importance of accurate evaluations.

Module 4:11 of 137

Agenda

In module 4 we will cover the following:

1. Definitions of evaluation and feedback

2. The ACGME 6 core competencies

3. Assessment of learners

4. Providing feedback to learners

5. Writing learner evaluations

Module 4:12 of 137

Imagine yourself as a chef in a top fancy restaurant and you are cooking for a food critic for the national newspaper in your area–a very important person.

As you are preparing your dish, you periodically assess the taste of your soup and make changes by adding more ingredients based on your assessment.

Module 4:13 of 137

You also ask a few colleagues in the kitchen to assess your soup and you make adjustments based on their objective opinion.

What has just happened? The first important step here is to gain an understanding of how the soup is progressing as compared to your standard soup – this is assessment.

Module 4:14 of 137

The second important step is seeking an assessment from others in order to improve the soup – this is asking for feedback.

After hearing the comments of other chefs, you contemplate for a while if and how to make your adjustments – this is reflection.

Now your soup is ready to be served and judged by the food critic.

Module 4:15 of 137

You then observe the critic eating your soup. You try to make assumptions on how they liked it based on their facial expressions as they tasted the soup. If you ask their opinion you are seeking feedback.

When they assess your soup based on all the soups they have tasted and you await their final judgment in tomorrow’s newspaper - this is their written evaluation.

Module 4:16 of 137

This process of assessment, providing feedback, reflection and evaluation are the key ingredients to evaluating and providing feedback to learners.

Feedback should be something sought after and used to enhance learner knowledge, attitudes and skills. It should be provided during and after every rotation.

Module 4:17 of 137

Evaluation: The process of making qualitative judgments or value statements about a person’s work or performance in comparison to a standard or expectations.

Feedback: A verbal summary of a learner’s performance that is based directly on an observation of the learner. It is given in an attempt to improve behavior.

Module 4:18 of 137

There are two (2) basic purposes of evaluation and feedback:

1) To assess learners.

2) To provide a means for changing behavior to prevent repeated errors and reinforce good behaviors.

Module 4:19 of 137

There are two (2) major types of evaluation and feedback:

1. Formative2. Summative

There are four (4) major foci of evaluation and feedback:

1. Knowledge2. Attitude3. Skill4. Overall behavior/performance -

professionalism

Module 4:20 of 137

Formative – means during the teaching encounter or “on the spot.”

Summative – means at the end of a teaching rotation or training period.

Both evaluation and feedback can be either formative or summative.

Module 4:21 of 137

Question

Explain the difference between feedback and evaluation.

(Click here for the answer)

Module 4:22 of 137

Answer

Evaluation: The process of making qualitative judgments or value statements about a person’s work or performance based on standards or expectations.

• Making a judgment on how learners do

Feedback: A verbal summary of a learner’s performance that is based directly on an observation of their performance. It is given in an attempt to improve behavior.

• Telling learners what they did well

• Telling learners how to improve

• Telling learners want not to do in the future

Module 4:23 of 137

First let’s focus on evaluation and feedback during day to day teaching activities – or formative evaluation and feedback.

We will then return to summative evaluation and feedback latter in the module.

Let’s discuss how that last

procedure just went.

Great! I need some

feedback on my suturing

skills.

Module 4:24 of 137

Recall the purpose of learner objectives. In module 2 you learned that students have learning objectives for each rotation. These are the skills, knowledge and attitudes that they should acquire by the end of each rotation. Thus, the expected outcomes.

Outcomes are the end result. Let’s now introduce the concept of competencies.

Module 4:25 of 137

Competencies

Competencies are the outcomes and measures the student should achieve. We use competencies to assess how well a student has done – how competent they are for a given area of knowledge, skill or attitude.

Both residents and students have competencies based on the ACGME topics.

Module 4:26 of 137

Competencies

The 6 ACGME core competencies are:1. Professionalism2. Medical knowledge3. Patient care4. Systems-based practice5. Interpersonal communication6. Practice based learning and improvement

Leadership was added as a 7th core competency graduation goal for students at the Baylor College of Medicine.

Module 4:27 of 137

Competencies

The student objectives and core competency graduation goals can be found on the RATL resource page.

We suggest you familiarize yourself with the competencies because these are the standards that your teachers are using to assess your performance and how the school assesses their graduating students.

Module 4:28 of 137

Competencies

Here is a brief glimpse into the ACGME core competencies document. You may also print them from the RATL resource page or click below to go to the ACGME web page for a shortened version.

Module 4:29 of 137

Competencies

As a reminder, the 6 core competencies are:

1. Professionalism (PRO)2. Medical knowledge (MK)3. Patient care (PC)4. Systems-based practice (SBP)5. Interpersonal communication (ICS)6. Practice based learning and improvement

(PBLI)

Use these abbreviations for the following matching game.

Module 4:30 of 137

QuestionMatch the examples of ACGME competencies with the correct core competency area: PC, PRO, SBP, PBLI,ICS, MK

Communicate effectively and demonstrate caring and respectful behaviors…

Counsel and educate patients and their families…

Know and apply the basic and clinically supportive science…

Facilitate the learning of students and other healthcare professionals…

Apply knowledge of study designs and statistical methods…

Use effective listening skills…

Work effectively as a member or leader of a health care team…

Demonstrate compassion, respect and integrity…

Practice cost-effective care and resource allocation…

Know how to partner with health care managers…

Know how types of medical practice and delivery systems differ…

Module 4:31 of 137

Answers

How did you do? The answers are below.Communicate effectively and demonstrate caring and respectful behaviors… PC

Counsel and educate patients and their families… PC

Know and apply the basic and clinically supportive science… MK

Facilitate the learning of students and other healthcare professionals… PBLI

Apply knowledge of study designs and statistical methods… PBLI

Use effective listening skills… ICS

Work effectively as a member or leader of a health care team… ICS

Demonstrate compassion, respect and integrity… PRO

Practice cost-effective care and resource allocation… SBP

Know how to partner with health care managers… SBP

Know how types of medical practice and delivery systems differ… SBP

Patient care (PC) Medical knowledge (MK) Professionalism (PRO)

Interpersonal communication skills (ICS) Practice-based learning & improvement (PBLI) Systems-based-practice (SBP)

Module 4:32 of 137

First let’s focus on assessing learners. Keep in mind, you can also use this process for patient performance or adherence to lifestyle modifications.

As the teacher, you will have to assess how your student or patient is doing and then give them the feedback to help them improve.

Assessment

Module 4:33 of 137

Assessment

Assessing students requires a basic understanding and familiarity with their course learning objectives and the descriptions of competencies.

Competency goals may be set by national or sub-specialty organizations.

– Hemoglobin A1C levels less than 6.5 mg/dl– 15 IV placements– 24 endotracheal intubations– Duty hours must be limited to 80 hours per week,

averaged over a four-week period, inclusive of all in-house call activities.

Module 4:34 of 137

Assessment

You can also use your student evaluation forms to compare your student’s performance with the given scale – helping to determine those with a high level of performance and those with a low level of performance.

For example, if the objective is to accurately measure the blood pressure on several different occasions, then you must observe the student taking the blood pressure.

Module 4:35 of 137

Assessment

Then you must compare their results to your standard measurements or an experts’ measurements.

How well they measured the blood pressure compared to the standard is their assessment or evaluation. Are they competent in this skill given the standards or expectations for their level?

Module 4:36 of 137

Assessment

If you have directly observed your student’s performance or knowledge and they need improvement, then you must provide them with the feedback to guide them toward improvement and future success.

Not competent =Needs

Feedback

Module 4:37 of 137

Assessment

In our example of the chef, they sought out other opinions in order to improve the soup. They must have felt comfortable seeking those opinions and they must have valued those individuals they asked for opinions.

As the teacher, you must also make learners feel comfortable asking for your assessment and feedback and you must provide information that is valued. So let’s focus on providing feedback.

Module 4:38 of 137

Feedback

How you provide feedback can make a significant difference in how your learner receives it and responds to it. Remember, the goal is to improve their performance compared to the gold-standard.

To emphasize how feedback can affect a student, patient or other learner, try playing the feedback game on the next slide.

Module 4:39 of 137

Feedback

The feedback game is designed to help you better understand feedback and some important points about providing feedback.

Click here to play.

Click here to skip the feedback game.

Feedback Game 1

Module 4:40 of 137

Feedback Game: Version 1

Guess a number between 1 and 10. Use your mouse to click on your guess. You have only three (3) chances to guess the correct number. Then click below to give up.

I give up. Proceed with module.

1 2 3 4 5 6 7 8 9 10

Module 4:41 of 137

You selected choice 1. You are totally wrong!

Back to game

Module 4:42 of 137

You selected choice 2. Not! Try again!

Module 4:43 of 137

You selected choice 3. Next time you try, bring your brain!

Module 4:44 of 137

You selected choice 4. You are not even close!

Module 4:45 of 137

You selected choice 5. Can’t you do better than that?

Module 4:46 of 137

You selected choice 6. Here’s a dime to call your mother – maybe she’ll think you are a doctor!

Module 4:47 of 137

You selected choice 7. That was a stupid guess!

Module 4:48 of 137

You selected choice 8. I have worked with babies with more brains than you!

Module 4:49 of 137

You selected choice 9. You are so far off base you are out of the field!

Module 4:50 of 137

You selected choice 10. How pathetic! Try again!

Module 4:51 of 137

Based on the feedback you received during this game…

– Did you get the correct number?– Was the feedback provided helpful in guiding

you to the right number?– Did you value the feedback you received?

Module 4:52 of 137

After playing this game how do you feel?Which of these adjectives most closely reflects how you felt after your three attempts?

Annoyed Frustrated Worried

Angry Demoralized Confused

Stupid PressuredMost of

the above

Module 4:53 of 137

This type of feedback is considered rude, unprofessional and unnecessary. It can be harmful and won’t help improve the learner’s ability.

This type of feedback is considered a “right of passage”* by some. This is part of the “hidden curriculum” and is a problem if you continue it. Remember, be part of the solution, not part of the problem. Avoid demeaning students, colleagues and staff.

*McIlwrick, Nair & Montgomery, Academ Psych 2006.

Module 4:54 of 137

Since that was not an example of effective feedback, let’s try again. Remove yourself from the last game and start again by guessing a number between 1 and 10. You have only three (3) chances to guess the correct number.

I give up, continue with module.

1 2 3 4 5 6 7 8 9 10

Module 4:55 of 137

You selected choice 1. Great try! Try again.

Module 4:56 of 137

You selected choice 2. That was really good.

Module 4:57 of 137

You selected choice 3. Great job!

Module 4:58 of 137

You selected choice 4. I like the way you guess. Want to try again?

Module 4:59 of 137

You selected choice 5. Excellent job. Keep up the good work.

Module 4:60 of 137

You selected choice 6. Well done.

Module 4:61 of 137

You selected choice 7. Nice try. Keep it up.

Module 4:62 of 137

You selected choice 8. Bravo!

Module 4:63 of 137

You selected choice 9. Way to step up to the plate.

Module 4:64 of 137

You selected choice 10. I think you did a great job.

Module 4:65 of 137

Based on the feedback you received during this game…

– Did you get the correct number?– Was the feedback provided helpful in guiding

you to the right number?– Did you value the feedback you received?

Module 4:66 of 137

How did this go around make you feel?

Satisfied Encouraged Calm

Pleasant Confident Clear

Smart GuidedMost of

the above

Module 4:67 of 137

You have just experienced what some might consider feedback. But this feedback was filled with empty praise.

Empty praise may make someone feel good but it does not lead to improved behavior because they don’t know what they did right. How often do you say to your students, “You did a good job,” and that’s all you say?

Module 4:68 of 137

Feedback

In the feedback game, effective feedback would provide specific guidance to get you to the desired outcome (the correct number or better tasting soup).

Effective feedback requires specific comments geared toward improvement in behavior and is based on direct observations.

Module 4:69 of 137

Now let’s try the game the right way: with effective and appropriate feedback. Guess a number between 1 and 10. Click on your choice. You have only three (3) chances to guess the correct number.

Proceed with module. 1 2 3 4 5 6 7 8 9 10

Module 4:70 of 137

You selected choice 1. This was a good guess but you need to increase you guess by 6.

Module 4:71 of 137

You selected choice 2. This was a good guess but if you increase your guess by 5 you well be where you need to be.

Module 4:72 of 137

You selected choice 3. This is a good guess. Increase your number by 4 and you will be at your target.

Module 4:73 of 137

You selected choice 4. This is a good guess. Add to that number 3 and you will be at the target number.

Module 4:74 of 137

You selected choice 5. This is an excellent guess and only two away from where you need to be. Increase your next guess by 2 to hit the target.

Module 4:75 of 137

You selected choice 6. This is an excellent guess and very close to the target number. Increase this guess by one and you are there!

Module 4:76 of 137

You selected choice 7. This is perfect. This is the exact number you should be at. Congratulations!

Module 4:77 of 137

You selected choice 8. This is an excellent guess. You are close to your target and you should reduce your next guess by one to get to the target number.

Module 4:78 of 137

You selected choice 9. This is an excellent guess and you are off by only two. Reduce you next guess by two to reach your target.

Module 4:79 of 137

You selected choice 10. This is a good guess. You need to reduce your guess by three in order to get to your target.

Module 4:80 of 137

Now that was more like it! With appropriate feedback you should have gotten to the correct answer in only two attempts.

You should have also learned how closely you guessed the first time and what you needed to do to get to the correct answer.

Module 4:81 of 137

Now select how you felt in the last version of the feedback game.

Satisfied Encouraged Calm

Pleasant Confident Clear

Smart GuidedMost of

the above

Module 4:82 of 137

Question

Select the statement that reflects the moral of the story. (click with your mouse to see the correct response for each.)

1. Feedback is very useful.

2. Feedback can help guide the learner to improve.

3. Inappropriate feedback can be harmful.

Module 4:83 of 137

Answer

All of the above…feedback is the key to helping your students and patients get to where they need to be.

Remember, feedback does not need to be malignant. It should improve a learner’s ability. Inappropriate feedback (negative, rude or full of empty praise without information) is useless and can be harmful to learners and patients.

Module 4:84 of 137

Think about a time you received feedback that was helpful and resulted in an improved behavior.

What made that feedback effective? Did you value the opinion of the person giving the feedback? Did you feel comfortable asking for feedback?

Feedback

Go back to play the feedback

Module 4:85 of 137

Characteristics of effective feedback include:– It is timely (linked closely to the observed behavior)

– It is specific– It is linked to goals/objectives/competencies– It suggests correct performance– It is focused and limited (not too much at one

time)– It is balanced

Overall feedback should be:– Expected– Provided often– Allows/plans for follow-up

Feedback

Module 4:86 of 137

Feedback

Just a point on being timely.

When providing feedback… Timing Is Everything! The most important and most valued feedback is that feedback provide immediately during/following a task or activity! It should not occur at random times or when too much time has passed. Keep it tied to the behavior!

Module 4:87 of 137

Feedback

This is especially true if the behavior is negative and needs immediate changing. If a student forgets to include the social history during a presentation, inform them of it immediately and then provide a summary of their entire performance (as feedback) after their full presentation is complete.

Module 4:88 of 137

Feedback

Feedback, just like other aspects of teaching, is best if planned. At the beginning of each month, provide students with the ground rules on when you will provide both formative and summative feedback.

Let them know that you will provide feedback often and that they should feel comfortable seeking feedback from you.

Module 4:89 of 137

Feedback

As a reminder:– Always focus on improving behaviors!– Only focus on 1-2 major points at a

time.– Involve the learner in the process.– Be specific, focused and limited.– Create a safe learning environment and

set the ground rules.

Module 4:90 of 137

Feedback

Here’s Dr. Dewey’s 5-step plan in giving feedback:

1. Observe and assess first

2. Identify behaviors or focus

3. Reinforce positive behaviors

4. Discuss changes/improvments

5. Summarize and create improvement plan

Module 4:91 of 137

Feedback in Five Steps

Observe & assess…

Identify focus…

I watched you as you prepared your soup today & compared to the guidelines on soup making I have some feedback for you.

I want to discuss the onion concentration and the balance of the chicken and stock.

Module 4:92 of 137

Reinforce positive behaviors…

Overall, your chicken soups tastes like chicken soup. The balance of chicken and stock is just right and I suggest you continue with this balance.

Module 4:93 of 137

Discuss changes…– (Suggest specific

changes in behavior. Focus on only 1 or 2 items.)

Regarding the onion flavor. It is a bit strong for the average chicken soup eater. May I suggest you reduce it by about a third. What do you think about that? I also noticed that the soup simmered at a low temperature and required more cooking time. Turn your flame up to a medium fire. Do you think this will reduce your cooking time?

Module 4:94 of 137

Summarize and create plan…

So, overall your soup is tasty and well balanced with chicken and stock. Reduce the onion flavor by 1/3 and turn up your simmering temperature to medium. We will make another attempt at making chicken soup next week. Please read this article on the health effects of chicken soup before the next session.

Module 4:95 of 137

Now lets hear some example of formative feedback with real medical examples.

The next slide has a few options you can chose by clicking on the corresponding sound icon. You may listen to as many as you like.

Module 4:96 of 137

Examples of Effective Formative Feedback

Example 1 (Pediatrics – Physical Examination)

Example 2 (Medicine- History Presentation)

Example 3 (Surgery – Suturing skills)

Example 4 (Psychiatry – Patient counseling)

Example 5 (Family Medicine - Professionalism

issue)

Module 4:97 of 137

Feedback

Notice, they do not take much time! Each of these were between 30 to 90 seconds. Feedback doesn’t have to take a lot of time to be effective or have significant impact on your learners and patients.

Providing feedback often is a very efficient use of your teaching time. Accurate feedback can prevent the need for repetition.

Module 4:98 of 137

Feedback

Let us reemphasize the importance of feedback when teaching invasive procedures. Based on your learning from the previous module on teaching procedural skills please remember:

– Provide step-by-step feedback during the procedure.– When something goes wrong…step in immediately!– Patient safety is still our first priority! – Never let a patient suffer in order to teach a learner.– There are ample times to teach. Knowing when is key!

Module 4:99 of 137

Question

List the five steps for providing feedback.

Module 4:100 of 137

Answer

1. Observe and assess first

2. Identify behaviors or teaching focus

3. Reinforce positive behaviors

4. Discuss changes

5. Summarize and create improvement plan

Module 4:101 of 137

Reflect on how you might apply the five steps in providing feedback to your medical students and patients.

Click on the icon to hear an example of a mid-rotation feedback session with a student not so interested in pediatrics and who can use some improvement in obtaining more thorough histories.

Feedback

Part 1: Setting up a mid-rotation feedback session (<1 minute)

Part 2: The feedback session (6 -7 minutes)

Module 4:102 of 137

Patient Feedback

Regarding patient feedback, please keep Covey’s habit of “seek to understand” in mind. Not every patient can do everything you ask of them. Why is this? Is there something they didn’t tell you? What influences or circumstances prevented or inhibited their compliance?

Do you seek to understand before you label patients as “non-compliant?”

Module 4:103 of 137

Patient Feedback

Pause here to reflect on what you would say to your patient who wasn’t able to get their blood pressure medication refilled before coming to your clinic visit.

Did you seek information first? Were your comments: Specific? Geared toward behavior change? Limited? Did it include the patient? Was it at their level? Click below to hear an example.

Module 4:104 of 137

Feedback

Make it known that feedback is occurring. You may have to state clearly, “Now I am going to provide you with some feedback.”

Feedback should be given:– Immediately as needed– Daily– Mid-rotation– End of rotation

Module 4:105 of 137

Feedback

There are several times during the day to provide students or patients with feedback.

– During/after presentations– While observing skills– While reviewing written records– When professionalism is demonstrated – After speaking with the staff, employees, etc.– While educating patients & family members– And more...think of a few for your team

Module 4:106 of 137

Feedback

Every teaching moment is an opportunity to provide

feedback!

You can make it happen!

Module 4:107 of 137

Feedback

“I have come to the frightening conclusion that I am the decisive element in the classroom...As a teacher, I possess tremendous power to make a child's life miserable or joyous. I can be a tool of torture or an instrument of inspiration. I can humiliate or humor, hurt or heal. In all situations, it is my response that decides whether a crisis will be escalated or de-escalated, and a child humanized or dehumanized.” ~ Ginott, 1976

Module 4:108 of 137

Things to Avoid!

To avoid pitfalls of providing feedback stay away from these problems:

– Not directly observing behaviors.– Confusing personality, cultural or ethnic issues

with behaviors.– Focusing on negatives without giving positives.– Giving feedback during times of stress or

anger. (Either yours or your learners.)

Module 4:109 of 137

Things to Avoid!

You will also want to avoid:– Waiting too long before giving feedback.– Giving feedback in a threatening environment.– Not giving any feedback and sending in an

evaluation!

Module 4:110 of 137

Feedback

Select the following statement that best describes your future use of feedback.

1. I plan to use feedback a lot to help guide my learners to better behaviors and performances.

2. I plan to use feedback some of the time to help guide my learners to better behaviors and performances.

3. I don’t plan to use feedback to help guide my learners to better behaviors and performances.

Module 4:111 of 137

Feedback-Summary

Feedback: A process by which a teacher provides a learner with the results of an assessment with the purpose of improving the learner’s performance.

Module 4:112 of 137

Feedback-Summary

Always provide formative feedback based on direct observation and assessment. Every teaching moment should be followed by feedback. Feedback can be written or verbal.

Summative feedback is the final feedback given to a learner at the end of their training rotation. Follow the same key guidelines for formative feedback.

Module 4:113 of 137

Evaluation

You have now learned to assess learners based on their standard competencies and learning objectives.

You have also learned about and heard some examples of how to provide feedback to patients and learners.

Now let’s look at how you complete your student evaluation forms after each rotation.

Module 4:114 of 137

Evaluation

If we go back to our soup example, the journal critique will most likely get back to you, the chef, in a written format. This is the evaluation form.

When you read your evaluation, you will hope that the reviewer provided an honest assessment based on the facts. Again, how would you feel if they slammed your soup and never even tasted it?

Module 4:115 of 137

Evaluation

So too with evaluations of learners. Base your assessment on the facts and document those facts. The form you use should be provided by the institution and is usually paper or electronic.

The evaluation form can take different formats including a paragraph of text, a rater sheet with numbers, grades or rankings, or a combination of both.

Module 4:116 of 137

Evaluation

Completing evaluation forms will become easier as you become more familiar with the evaluation form.

Many will cover the ACGME core competencies we learned earlier. These criteria can be very helpful in assessing your student. Need a reminder of the core competencies? Click here.

Skip reminder

Module 4:117 of 137

Competencies

The 6 ACGME core competencies are:1. Professionalism

2. Medical knowledge

3. Patient care

4. Systems-based practice

5. Interpersonal communication

6. Practice based learning and improvement

Module 4:118 of 137

Evaluation

Try to become familiar with the criteria by reviewing them before your student arrives on service. Then mid-month, review them with your student and then again at the end of the month.

Knowing the criteria for students helps you make a more accurate assessment of their knowledge, attitudes and skills.

Module 4:119 of 137

Evaluation

Accurate assessments are important in medical education. We want to be assured that students are evaluated fairly and adequately. Those that excel should be recognized, but we also want to know which students require remediation. The goal of the institution is that all students be prepared for their roles as interns.

Module 4:120 of 137

Evaluation

Completing an evaluation for a student that does not adequately reflect their ability can be misleading and damaging for the student and teacher.

Students need help seeing where they are in their professional development. They also need guidance to improve themselves so they can become more like a resident.

Module 4:121 of 137

Evaluation

Your comments also strongly influence the students’ grades and dean’s letters. Please think about your comments first and then complete evaluation forms.

Take a look at some example comments from student evaluations. Do you think these are appropriate? Do they provide information?

Module 4:122 of 137

Question

Which is the best? 1. Excellent job. Gave good conference &

talks. Needs to be more assertive.

2. Overall good job! Will make a wonderful House Officer.

3. Intelligent and empathic student. Responded well to feedback, reorganized history for concise and well organized presentations. Has excellent work ethic and potential as house officer.

Module 4:123 of 137

Answer

Remember the feedback game? Some of these comments are positive without explaining how the student rated compared to competencies or their objectives.

Overall, #3 has the most detail about the student and describes their work ethic, flexibility, knowledge, and humanism. A residency program director would feel more assured selecting this student over the other two even thought #1 & #2 are both positive.

Module 4:124 of 137

Think about an evaluation your previously wrote or one you received.

Did you find the information reflective of your performance compared to your learning objectives or competencies? Did the evaluator provide details about your performance that was good as well as those that needed improvement? How will you change your approach to completing evaluations?

Evaluation

Module 4:125 of 137

Evaluations

What if you have a student that is not stellar and is in fact poor or below average; not meeting their objectives or competencies?

Handling this situation may take some effort. You may need the help of the faculty attending for some situations. But most likely, the student just needs some guidance to get them on the right track.

Module 4:126 of 137

Evaluations

So how do you deal with students that are not enthusiastic about your sub-specialty? Or the student who lies? Or the student that never shows up? Or the student who is potentially dangerous to the patients?

In following our plan outlined, you must first seek to understand. Try to identify their issues so you identify the correct teaching focus and then discuss this with the student.

Module 4:127 of 137

Evaluations

Try to approach the student in a non-threatening way. In these situations expert communication skills using the reflective “I” can be helpful. Try to remain neutral and keep the focus on the observed behavior that needs improvement. Using the reflective “I” (“I understand this may be difficult for you to hear…”) and avoiding blame will keep emotions calm. Avoid getting emotionally involved and keep the tone of voice as one of concerned teacher.

Module 4:128 of 137

Evaluation

Dealing with difficult learners can be challenging. To learn more about dealing with problem learners read the article by Luca & Stallworth 2003 on the RATL resource page.

Module 4:129 of 137

Evaluation

In general, evaluations can also be both formative (during) and summative (final). For an example, written evaluations throughout the clinical years are technically formative, while the deans letter sent out at the end of the student’s clinical years is summative. It is the final written evaluation of the student’s progress.

Module 4:130 of 137

Evaluation

You can also think of the three individual monthly evaluations of a medicine rotation as being formative until the final grade at the end of the medicine core rotation, which is summative.

Whether formative or summative the key is that you complete your student evaluations based on observed behaviors and that the evaluations accurately reflect the student’s behaviors compared to given standards.

Module 4:131 of 137

Reflection

Think about an evaluation you received recently. Were the comments based on direct observations? Did the comments compare you to your expected role or activities (objectives)? Did the comments describe your competence? Were you aware of the issues discussed in your evaluation?

Now think about an evaluation you wrote and compare it to the questions above. Do you need to make any adjustments for the future?

Module 4:132 of 137

Summary

In summary, you assess students by comparing them to a standard – most likely their learning objectives and competencies.

Throughout your time with a student, you should provide them with feedback that is geared toward improving their behaviors and based on direct observations.

Module 4:133 of 137

Summary

You want your feedback to be well received – so make sure you communicate effectively and focus the information on direct observations. Try to avoid using “good job” unless you describe what was good and keep students and patients involved in the process.

When completing evaluation forms, use descriptive terms that provide insight into the student’s actual performance compared to the standards provided.

Module 4:134 of 137

Through the four modules, you have learned about professional identity, relationships in medicine, leadership skills and the basics of teaching students & patients.

We covered 1) effective communication, 2) organized & efficient teaching, 3) teaching psychomotor skills, 4) changing patient behaviors and 5) evaluation and feedback.

Closing Instructions

Module 4:135 of 137

Our goal was to provide an introductory level knowledge on teaching and leadership skills for 4th year medical students and 1st year residents.

After learning about the value of assessment, feedback and evaluation, we hope you will take the opportunity to provide us with a written evaluation of the RATL modules.

Module 4:136 of 137

Please take a few minutes to complete the following:

1. Posttest for Module 4: Evaluation and Feedback

2. Teacher Identity Scale

3. Evaluation for module 4

Thank you for participating.

Module 4:137 of 137

End of RATL Module 4End of RATL Module 4

Special Acknowledgements:

Mary Ann Nichols and Regina Russell, M.Ed.

Office for Teaching and Learning in MedicineVanderbilt University School of Medicine

ratl module 4: evaluation and feedback module created by: charlene m. dewey, m.d., m.ed., facp...

Documents

won kyoun park, m.d., sung-ho jung, m.d., ju yong lim, m.d

breast cancer genetics module m. kent froberg, m.d. 2009

overview of the orthopedics module albert quintos m.d., fpoa

nutrition module notes pediatric i – second year rebecca...

· physicians david b. argo. m.d. john e. bartsch, m.d....

kazuhiro tsuchiya, m.d., masamichi imai, m.d

jamal s al wakeel m.d., durdana hammad m.d., *abdul karaem...

cathy eng m.d., salil sethi m.d., george j.chang, m.d.,...

jack ansell, m.d. jack hirsh, m.d. nanette k. wenger, m.d

joan anzia , m.d. tony rostain , m.d

clinical skills verification rater training module 3 setting...

general anaesthesia m.fencl m.d. l.dadák m.d.

l liem , m.d. f huygen, m.d., ph.d marc russo , m.d. jp...

ravneet singh, m.d. depinder mann, m.d

vitamin a deficiency - carter center · module \ vitamin a...

cracking the usmle!! muhammad ali khan m.d. faraz khan luni...

daniel yoshor , m.d. michael raber peter kan, m.d. david xu,...

choosing your level of care andy jagoda, m.d., arthur m....

ssuhs.inssuhs.in/pdfdocuments/1574078262.pdf · arnab...

ratl module 3: teaching physical exams and procedural skills...