JGIM

EDITORIAL

Physician Education, Evidence and the Coming of Age of CME

T he article in this issue of the Jou rna l by Gifford and colleagues I affords an opportuni ty to reflect on the

impact of cont inuing medical education, or CME. Such a

s tudy raises several issues, namely: the increasing aware- ness of the need for CME and for s tudies such as this; the

methodology of CME research; and the implications for practice, the profession of medicine and research in phy- sician educat ion and learning.

The first issue, the need for CME--and its corollary, the need for evaluation of its effectiveness--has risen pri-

marily from a clear sense of what might be termed the dissonance in physician education. On the one hand,

CME resources are plentiful {including a wide range of products such as videotapes, conferences, and journals); CME accreditation is a well established process; and CME

credit systems for relicensure, recertification, hospital privileges, ar id/or professional membership are well ac- cepted. On the other hand, the need for remedial CME in selected physician populat ions is clear. For most physi- cians, the research-to-practice stage is often very slow. The latter phenomenon has been explored by Rogers 2 in

his s tudy of the adoption of innovation, by Fox et al. 3 in

their review of physicians ' changing and learning prac- tices, and by Lau et al 4 in the clinical area.

The dissonance has in t u r n given rise to a l i terature

spann ing nearly two decades assessing the effectiveness of CME. 5'6 The most recent of these 6 summarizes the find-

ings of over 100 randomized controlled trials (RCTs), the s tudy design believed by m a n y to produce the most credi- ble results, and provides the background for another s tudy in this issue by Gifford et al.1

The issue of the methodology, or methodologies, of CME research forms the second quest ion generated by

studies of this type. W-hile s tudy design is an impor tant

consideration, there are m a n y other factors that contrib- ute to methodologic rigor in CME (and other) research. Achieving them in s tudies of CME drives the field and forges scholarly enterprise, too often lacking in CME. Strong methodologies also have the potential of demon-

strat ing links between educat ion and heal th care out- comes.

Specific methodologic benchmarks for s tudies of CME interventions were established by colleagues at Mc- Master University in the 1980's7: (l) How well are the health professionals described? Is the sample representa- tive of the group of physicians discussed? Are other fac- tors considered such as the qualifications, experience, workload, and methods of payment of physicians? (2) How

well is the educat ional intervention described? Does it in- clude a needs assessment , behavioral objectives, a de-

scription of its content and format? Are issues of compli- ance, contaminat ion and co-intervention addressed? (3} Is

the issue of s tudy design addressed? While RCTs may generate the most credible results, other s tudy designs such as quasi r andom allocation may be acceptable, and

may be the only ones feasible. (4) How are outcomes mea- sured? Are the measures of health care outcomes em- ployed feasible? If they are. are they reliable and credible?

If not, are alternative measures (proxies) such as physi- cian performance or competency assessment measures pilot-tested and utilized? If so, are l inkages between these

"lower level" measures and health care outcomes estab- l ished? (5) Finally, unde r a more general heading, are is-

sues of statistical and educational or clinical significance addressed? Does the s tudy draw on a relevant li terature base; does it contr ibute to it at both practical and theoret-

ical levels? How does the paper by Gifford et al. compare to this

s tandard? On the whole - -and despite the clear bias inherent in

these benchmarks towards a quanti tat ive biomedical re- search model--very well. There are clear s t rengths to this study: the investigators have conducted a RCT, a difficult and laborious bu t worthwhile measure; efforts have been

made to encourage a high participation rate; evaluation measures are patient-centered, credible and pilot-tested, and assessors are bl inded to group allocation, Further,

participating neurologists are well described, us ing infor- mat ion of a demographic and descriptive nature , includ- ing pat ient volume and (importantly) insurance mix. Their

r andom selection and allocation is an essential consider- at ion in the generalizability of the results. Similarly, the

format of the educational in te rvent ion--a mailed, prob- lem-based learning program-- is well described, as is the issue of compliance with the program. Learning objec-

tives, in this case recommendat ions for managemen t of pat ients with movement disorders, formed the behavior

objectives of the program. On the negative side however, the authors did not re-

port if "subscribers" to the educat ional program were rep- resentative of the membership of the American Academy of Neurology (AAN). Thus, generalizability may be re- duced, No at tent ion was paid to cointervention (did the experimental group participate in other CME activities in

the same topic?), contamina t ion {could members of the experimental group distr ibute the CME program to con-

705

706 Editorial JGIM

trol g r o u p m e m b e r s ? ) or n e e d s a s s e s s m e n t (did p h y s i c i a n

m e m b e r s of t h e AAN ind ica t e t he s u b j e c t of m o v e m e n t

d i s o r d e r s a s a h i g h pr ior i ty l e a r n i n g n e e d ? Is t h e r e a doc-

u m e n t e d gap b e t w e e n real a n d r e c o m m e n d e d per for -

m a n c e in t h i s sub jec t? ) F u r t h e r , a des i rab le , b u t p e r h a p s

i m p r a c t i c a l a d d i t i o n to t h e s tudy , wou ld h a v e b e e n the a s -

s e s s m e n t of real-l ife p e r f o r m a n c e or h e a l t h ca re o u t c o m e s

a s a m e a s u r e of t h e i m p a c t o f t h i s p a r t i c u l a r CME in te r -

ven t ion . The Gifford s t u d y ' s focus on m e a s u r e of c o m p e -

t e n c e (wri t ten ques t ions ) i n s t e a d of p e r f o r m a n c e m e a -

s u r e s (e.g., object ive a n d obse rvab l e re fe r ra l p r a c t i c e s or

p r e s c r i b i n g pa t t e rns ) , m a y exp la in t h e d i f fe rence b e t w e e n

t h e f ind ing t h a t ma i l ed m a t e r i a l s "work" a n d a n o t h e r re-

c e n t review's f ind ing t h a t t h e y don ' t . 8 In t h e la t t e r review,

only object ive a s s e s s m e n t s of p h y s i c i a n p e r f o r m a n c e were

c o n s i d e r e d a s o u t c o m e s . Gifford a n d h i s co l l eagues m a k e

severa l po in t s , however , a b o u t the i r i n t e r v e n t i o n ' s effec-

t iveness : it w a s p r o b l e m - b a s e d , it w a s ma i l ed to m e m b e r s

of a p r o f e s s i o n a l society, a n d it c i tes l i t e ra tu re w h i c h l inks

c o m p e t e n c y a s s e s s m e n t m e a s u r e s to p e r f o r m a n c e .

Of t h e s t u d y ' s w e a k n e s s e s , t he m o s t s e r i o u s m a y be

in t he a rea of n e e d s a s s e s s m e n t . The lack of inc lus ion of a n

ear l ier su rvey of neuro log i s t ' s p e r c e p t i o n s of l ea rn ing n e e d s

in th i s area , or more objective, r ep re sen ta t ive d a t a of iden-

tilled p e r f o r m a n c e gaps , m a y a c c o u n t for w h a t t h e inves t i -

g a t o r s t e r m a "modes t " overal l i m p a c t of t h e p rog r a m.

This b r i e f c r i t ique of t h e p a p e r b r i n g s u s to a t h i r d

a n d f inal s e t of i s s u e s w h i c h the s t u d y ra i ses , r e la t ing to

t h e imp l i ca t i ons of CME for prac t ice , p r o f e s s i o n a l m e d i -

c ine a n d h e a l t h ca re delivery. First , t h e r e a re imp l i ca t i ons

h e r e of a p rac t i ca l n a t u r e for spec ia l ty socie t ies : mai led ,

p r o b l e m - b a s e d l e a r n i n g m a t e r i a l s m a y h a v e a n effect on

m e m b e r s ' c o m p e t e n c e , a t l e a s t for t h o s e w h o s u b s c r i b e to

a CME p r o g r a m . W h e t h e r t h i s c h a n g e in c o m p e t e n c y c a n

t r a n s l a t e in to c h a n g e s in p e r f o r m a n c e or h e a l t h ca re out -

comes , a n d w h e t h e r w h a t e v e r effect it does have c a n be

i n c r e a s e d by c lose r a t t e n t i o n to n e e d s , is unc l ea r , b u t de-

se rv ing of f u r t h e r s tudy . Second , th i s i s s u e of effective-

n e s s of CME c a n equa l ly be r a i s ed in r ega rd to m u c h of

w h a t we do in phys ic i an educat ion: didact ic sho r t courses ,

t h e m a i n s t a y of m o s t CME offices, a n d t h e r e v e n u e - g e n e r -

a t ing a r m of m a n y spec ia l ty socie t ies , a lso d e m o n s t r a t e s

little effect on p h y s i c i a n p e r f o r m a n c e on h e a l t h ca re ou t -

comes . 8 W h a t d o e s w o r k in c h a n g i n g t h e s e o u t c o m e s is

e i t he r too expens ive , too t ime c o n s u m i n g or (worse yet)

no t even t h o u g h t of a s CME - - - p r a c t i c e - b a s e d i n t e rven -

t i ons s u c h as r e m i n d e r s or c o m m u n i t y - b a s e d m a n e u v e r s

s u c h as a c a d e m i c de ta i l ing or op in ion l eaders . Third, t h i s

s t u d y a d d s to t h e a l r eady s izable b o d y of knowledge 9

a b o u t p h y s i c i a n l e a r n i n g a n d c o n t r i b u t e s to a n e m e r g i n g

a c a d e m i c d i sc ip l ine of CME. S u c h a d i sc ip l ine is, finally,

of s ign i f icance to t h e c o m p l e x a n d rap id ly c h a n g i n g US

h e a l t h ca re s y s t e m in w h i c h CME b r o a d l y def ined a n d rig-

o rous ly eva lua t ed m a y legi t imate ly p lay a role. DAVE

DAVIS, MD, Associate Dean, Office of Continuing Education Faculty of Medicine, University of Toronto, Ontario, Canada_

REFERENCES

1. Gifford DR, Mittman BS, Fink A, Lanto AB, Lee ML, Vickrey BG. Can a specialty society educate its member to think differently about clinical decisions? Results of a randomized trial, J Gen In- tern Med. 1996;11:664-72.

2. Rogers EM. Diffusion of Innovations. New York, NY: The Free Press. 1995.

3. Fox RD, Mazmanian PE, Putnam RW. Changing and Learning in the Lives of Physicians. New York, NY: Praeger Publications. 1989.

4. Lau J, Antman EM, Jimenez-Silva J, Kupelnick B, Mosteller F, Chalmers TC. Cumulative recta-analysis of therapeutic trials for myocardial infarction. N Engl J Med. 1992;327: 248-54.

5. Bertram DA, Brooks-Bertram PA. The evaluation of continuing medical education: a literature review. Health Education Mono- graphs. 1977;54:330~62.

6. Omnan AD, Thomson MA, Davis DA, Haynes RB. No magic bul- lets: a systematic review of 102 trials of interventions to improve professional practice. Can Med Assoc J. 1995:153:1423-31.

7. Chambers L, Davis DA, Dok C, et al, The impact of CME. Mobius. 1986; 6: 59~2.

8. Davis D, Thomson MA, Oxman AD. Haynes RB. Changing physi- cian performance: a systematic review of the effect of continuing medical education strategies. JAMA. 1995:274:9:700-5.

9. Taylor-Vaisey AL. Information needs of CME providers: Research and Development Resource Base in Continuing Medical Educa- tion. J Contin Educ Health Prof. 1995;15:2:117-21.