you be the judge…

You Be the Judge…

View the following performances

In small groups, take 1 minute to

assign a score from 1-10 using any means.

Be prepared to share your reasoning…

How Did It Go?

What did you notice?

Resident Evaluation

Shawn Alderman, MD

Faculty Development Fellow

Learning Objectives

• Examined why we evaluate

• Identified essentials of evaluations

• Listed common skills to assess

• Reviewed available evaluation tools

• Recognized potential pitfalls and biases

Take Home Points…

• Evaluations are azimuth checks

• Assess specific skills

• Incorporate standards

• Use your tool box

• Be aware of our personal biases

• Departmental communication is essential

Activity One

• Why are evaluations important?

• What problems have you encountered?

Stakeholders

Types of Evaluations

• Formative

• Summative

Activity Two

With the End in Mind

In your groups, discuss the things you consider before writing a learner’s evaluation

The Standard

A Standard is…

• …the yardstick

• …the bare minimum requirement

• …a requirement that must be met

• …a minimum skill set

Standards Must Be

Evaluation Essentials

• Formal - standardized, routine, expected

• Communication – dialogue, frequency

• Documentation - summative, written signed

• Due Process - standards, plans, consequences

Skill Evaluation

Activity Three

In your groups, discuss the skill sets and put them in a ranked order from highest to

lowest in importance

The Tool Box…

• Checklists

• Rating Scales

• Anecdotal Records

• Incident Reports

• RIME Method

• BSQs

• Global Assessments

• OSCEs

• Simulation

• Examinations

• Video Clinic

• 360 Degree Evals

Is the tool sharp?

• Use the right tool

• Inspect the tool

• Tool improvement

Activity Four

In your groups, review the evaluation tools.

How would you improve the tool?

Activity Five

In groups, match the error or bias type with the

appropriate description

Avoiding Error and Bias

• Know thyself

• Consult colleagues

• Use multiple tools

• Faculty development

Recent Incident Bias: Bias because of an isolated recent incident…

Central Tendency: Sitting on the fence …

Extreme Response Bias: Respondents tend to mark extremes …

Affirmation/Yea-Saying Bias: Give positive responses irrespective …

Incompetence Bias: Assign high ratings because of lack of confidence…

Leniency Bias: Being overly charitable

Halo Effect: Overall impression of a person affect rating on each item…

Contrast Bias: Rating against another person's performance…

Stringency Bias: Being overly critical

Our Evaluation System

•Standard focused

•Assessing skills

•Using all tools

•Minimizing error/bias

•Communicating

Commitment

• What are our areas of weakness?

• How can we improve our evaluation process?

• When will we start?

Take Home Points…

• Evaluations are azimuth checks

• Assess specific skills

• Incorporate standards

• Use your tool box

• Be aware of our personal biases

• Departmental communication is essential