Using the Communication Assessment Tool (CAT) to Assess Communication

Skills ofFamily Medicine Residents

Linda Myerholtz, Ph.D., Lynn Simons , Psy.D. , Sumi Felix, M.D., Tuan Nguyen, M.D. , Julie

Brennan, Ph.D., Ana Rivera-Tovar, Ph.D., Pat Martin, PCC, Jeri Hepworth, Ph.D., Gregory

Makoul, Ph.D.

CAT 14-item patient satisfaction survey

Physician interpersonal and communication skills

Development based on sound psychometric methods

5-point rating scale: 1 = poor 2 = fair 3 = good 4 = very good 5 = excellent

Paper and pencil, phone, internet administration

Makoul G, Krupat E, Chang C. Measuring patient views of physician communication skills: Development and testing of the Communication Assessment Tool. Patient Educ Couns 2007; 67:333-342.

Individualized Resident Sample ReportYour

ResultsMean Results for all Mercy

FM Residents

National Sample

[Makoul, et al (2007) n = 950

Overall Mean Score 4.59 4.56 4.68 Standard Deviation 0.49 0.64 0.54Mean % of Items Rated as Excellent

64.78% 68.09% 76.30% Standard Deviation 6.54% 3.08% 11.10%

Mean Percent of Items Rated as "Excellent"

0%10%20%30%40%50%60%70%80%90%

1. G

reet

ed m

e in

a w

ay th

at m

ade

me

feel

com

forta

ble

2. T

reat

ed m

e w

ith re

spec

t

3. S

how

ed in

tere

st in

my

idea

s ab

out

my

heal

th

4. U

nder

stoo

d m

y m

ain

heal

th

conc

erns

5. P

aid

atte

ntio

n to

me

(look

ed a

t me,

liste

ned

care

fully

)

6. L

et m

e ta

lk w

ithou

t int

erru

ptio

ns

7. G

ave

me

as m

uch

info

rmat

ion

as I

wan

ted

8. T

alke

d in

term

s I c

ould

und

erst

and

9. C

heck

ed to

be

sure

I un

ders

tood

ever

ythi

ng

10.

Enco

urag

ed m

e to

ask

que

stio

ns

11.

Invo

lved

me

in d

ecis

ions

as

muc

h

as I

wan

ted

12.

Dis

cuss

ed n

ext s

teps

, inc

ludi

ng

any

follo

w-u

p pl

ans

13.

Show

ed c

are

and

conc

ern

14.

Spen

t the

righ

t am

ount

of t

ime

with

me O

VER

ALL

MEA

N

Resident X Mercy FM Overall Nov 2008 PGY2 Nov 2008

Objectives

Gather benchmarking data for the use of the CAT in Family Medicine residency programs

Examine differences based on: Year in training Native language of the resident (native English

speaking vs. non-native English speaking) Gender

Sample & Methods Six Family Medicine residency programs

Midwest & East coast Urban, suburban, rural communities 13-38 residents per program

127 residents

Data Collection: Nov 2008- Dec 2008 Paper and pencil version of the CAT

1,880 complete/useable surveys

Scoring: Mean ratings

Overall

By item

Percentage of “Excellent” ratings

Overall

By Item

Demographic Characteristics of Residents

% n

Gender: Male 44.1% 56

Female 55.9% 71

PGY Training Year: PGY 1 34.6% 44 PGY 2 30.0% 38 PGY 3 35.4% 45

Native English Speaking: Yes 56.7% 72 No 43.3% 55

Mean (SD) Minimum Maximum Age: 32.4 (5.8) 25 51

Results Overall mean percent “excellent” = 69.7% (SD = 40.4)

Items rated most frequently as “excellent” Paid attention to me (73.6%) Treated me with respect (72.8%) Showed care and concern (72.6%)

Items rated least frequently as “excellent” Encouraged me to ask questions (63.2%) Involved me in decisions (64.9%)

Consistent with Makoul et al.’s findings for practicing physicians

Training Year

Overall p = .015 PGY 1 vs. PGY 2 p =.018 ; PGY 1 vs. PGY 3 = p =.004 ; PGY 2 vs. PGY 3 = p = .55

77.0%

69.5%68.1%

60.0%

65.0%

70.0%

75.0%

80.0%

PGY 1 PGY 2 PGY 3

Overall Percent Rated as Excellent

Native Language

p = .06

71.8%

67.1%

60.0%

65.0%

70.0%

75.0%

80.0%

English Non-native English

Overall Percent Rated as Excellent

Gender

p = .81

69.2% 70.1%

60.0%

65.0%

70.0%

75.0%

80.0%

Male Female

Overall Percent Rated as Excellent

Limitations

Sample = Convenience sample of volunteering programs

Variability in the number of surveys collected per resident

Advantages

Measures patient’s perspective

User friendly administration & scoring

Provides empirical measure of core ACGME competency

Differentiates between residents

Easy to track changes over time

Benchmark data now available

Lessons Learned & Future Needs

Increase sample sizes

Evaluate changes over time

Consider efficacy of a minimum passing score 58% has been recommended2

Continue to expand benchmarking data

2 Wayne D, Cohen E, Makoul G, McGaghie W. The impact of judge selection on standard setting for a patient survey of physician communication skills. Acad Med 2008; 83: S17-20.