Benchmarking Clinicians

Farrokh Alemi, Ph.D.

Why should it be done?

Hiring, promotion, and management decisions

Help clinicians improve

Intrusion in clinician’s practice?

Managers understand patient outcomes Practice profiles are constructed after the

fact, when the patient is gone.  Both the patient and managers can use

benchmarked data Poor clinicians are bad for the patient as well as

for the organization

How should the analysis be done?

Compare clinician to average peerCompare clinician to average peer taking

care of same kinds of patientsCompare clinician to expectations on

admissionCompare clinician and peers on patients

matched on certain features

In benchmarking,  a clinician's performanceis compared to an expected value.

Compare Clinician to Average Peer

Calculate peer providers and the clinician’s average and standard deviation

Compare using test of hypothesis with unequal means

Problem: Maybe misleading as providers see different

kinds of patients and the clinician with more severely ill patients will naturally have worst outcomes 

Example Data

123 internal medicine residents at the New York-Presbyterian Hospital in New York City. 

The outcomes examined included following outcomes: Patients' satisfaction measured by telephone interviews of at least 10

patients Disease-management profiles for average of 7 diabetes and 11

hypertensive patients.  Patient's condition Frequency of use of various medications

Faculty-evaluations on seven dimensions: History taking Physical examination Differential diagnosis Diagnostic and/or treatment plan Health care maintenance Act compassionately Team player 

Sample Report

Compare Clinician to Average Peer Caring for Same Kinds of Patients

Example Data

Comparing Clinicians to Expected Prognosis at Admission

Assess patients severityPredict prognosisCalculate pair-wise student-t comparison

of observed and expected values

Example Data

Comparing Clinicians When Patient's Severity of Illness Is Not Known

Example Data

Event Tree for Clinician’s Patients Is Kept, Outcomes Change

Is it reasonable to benchmark clinicians?

Measurement distorts goalsMeasurement leads to defensive behaviorNo adequate measure of severity maybe

availableToo much time spent on measurement

and too little on improvement

How Should Benchmarked Data Be Presented?

Before the meeting Schedule a feedback time and date as soon as

possible.  Check your data to make sure there are no errors. Add text, charts or graphics.   Supplement numeric data

with anecdotal information and the customer's voice (e.g. a short audio from a patient).

Distribute handouts ahead of meeting to participants

How Should Benchmarked Data Be Presented?

At the meeting Make it clear that the evaluation is confidential    Make a brief introduction of the purpose of the session  Acknowledge the limitation of the practice profiling

method Present the data and not the conclusions  Explicitly ask for clinician's evaluation of the data after

each section of the report is presented  DO NOT defend the practice profiling method, the

benchmarking effort or any aspect of your work  Thank the clinicians for their time and describe next

steps

How Should Benchmarked Data Be Presented?

After the meeting Summarize the comments and append it to the report. Describe resources available.   Send a written report to each clinician. Ask the clinicians to comment on:

What worked well and what needs improvement? Do they plan to change their practice and in what way? Was it worthwhile?

Set the time of next benchmarking report. .

Take Home Lesson

Expected Outcomes Can Be Benchmarked Using Severity of

Patients’ Illness