e0b49518b67a740d30

5
Cranial Computed Tomography Interpretation in Acute Stroke Physician Accuracy in Determining Eligibility for Thrombolytic Therapy David L. Schriger, MD, MPH; Mary Kalafut, MD, MS; Sidney Starkman, MD; Michelle Krueger, MD; Jeffrey L. Saver, MD Context.—Intracranial hemorrhage must be excluded prior to administration of thrombolytic agents in acute stroke. Objective.—To evaluate physician accuracy in cranial computed tomography scan int erpr etat ion for dete rmi ningeligibi lit y for thr ombo lyti c ther apy in acute str oke. Design.—Administration of randomly selected, randomly ordered series of 15 computed tomography scans from a pool of 54 scans that demonstrated intrace- rebral hemorrhage , acute infarction, intracerebr al calcications (impostor for hem- orr hage ), old cerebral infa rction (im post or for acut e infa rct ion) , and norm al find ings . Participants.—A convenience sample of 38 emergency physicians, 29 neu- rologists, and 36 general radiologists. Main Outcome Measures.  —Ph ysic ian determination of eligibili ty for thrombo- lytic therapy based on computed tomography scan interpretation. Results.—Av era ge cor re ct sco re by all ph ysi cians on all com put ed to mog ra phy scans was 77% (95% condence interval, 74%-80%). Of 569 computed tomogra- phy readings by emergency physicians, 67% were correct; of 435 readings by neurologists, 83% were correct; and of 540 readings by radiologists, 83% were cor rec t. Ove ral l sen sit iv ity for det ect ing hem or rha ge wa s 82% (9 5% con de nce in- te rv al, 78% -85 %) ; 17% of emer gen cy phy si cians, 40% of neurol ogi st s, and 52% of radiologists achieved 100% sensitivity for identication of hemorrhage. Conclusion.—Physicians in this study did not uniformly achieve a level of sen- sitivity for identication of intracerebral hemorrhag e sufficien t to permit safe selec- tion of candidates for thrombolytic therapy. JAMA. 1998;279:1293-1297 From the Eme rge ncy Medicine Cen ter (Drs Schriger, Starkman, and Krueger) and the Depart- ment of Neuro logy (Drs Kala fut, Starkman, and Saver), School of Medicine, and the Stroke Center (Drs Kalafut, Starkman, and Saver), University of California at Los Angeles. Dr Starkman has received contracts and grants as an investigator in acute stroke studies sponsored by Genen tech Inc, South San Francisco, Cal if, and Boeh ringe r-Ing elheim Pharmaceutical s Inc, Ridg e- fie ld,Conn. He als o is a memberof theAcuteIsche mic Stroke Advisory Board and the Stroke Speakers’ Bu- reau, and he has participated in stroke conferences spo nso red by Genen tec h Inc and Boe hri nge r- Ingelheim Pharmaceuticals Inc. Dr Saver has participated as an investigator in re- search trial s fund ed byGenentec h Incand Boehr inge r- Ingel heim Pharmaceut ical s Inc, and has recei ved hon ora ria from Gen ent ech Inc for edu cat ion al speeches. He also has served on a scientific advisory board for Boehringer-Ingelheim Pharmaceuticals Inc. Roch e Labo ratori es Inc,Nutley,NJ, hasa subs tantia l inter est in Gene ntechInc. Thesecompani es havesup- ported theactivitie s of theUniversit y of Calif orniaat Los Angeles Stroke Center with unrestricted educational grants. Corresponding author: David L. Schriger, MD, MPH, 924Westwoo dBlvd, Suit e 300,LosAngeles,CA 90024- 2924 (e-mail: [email protected] u). RECOMBINANT tissu e plasminogen activator has been approved for select patients within 3 hours of onset of acute ischemic stroke . 1 Since thromb olytic therapy may produce lethal bleeding in patients with intracranial hemorrhage, the presence of int rac ran ialbloodon the initial computed tomography (CT) scan has been an exclusion criterion in the 5 tri als of thr ombolytic therapy for stroke 2-6 and in the recommendations of expert panels. 7-9 Thus, before thrombo- lytic therapy may be given in stroke, a physi cianhighlyskilledin ident ifyi ng in- tracranial hemorrhage must interpret the CT scan. For editorial comment see p 1307. Early signs of major cerebral infarc- tion (sulcal effacement, mass effect, and edema) also are associated with an in- crea sed ris k for intr acer ebra l hemo r- rhage in patients who receive thrombo- lytictherapy, 10-12 andsomeguidelinesrec- ommend avoiding thrombolytic therapy when these findings are present. 7-9 We conducted this study to determine how well emerg ency physi cians , neuro lo- gists, and general radiologists identified cra nia l CT sca ns tha t hav e evidence of in- tracr anial hemor rhage . We also asses sed these physicians’ accuracy in interpret- ing other CT findings commonly seen in pat ien ts wit h acu te isc hemic str oketo ex- amine whether physicians could distin- guish CT scans that demonstrate subtle imaging abnormalities representing po- JAMA, April 22/29, 1998—Vol 279, No. 16 Crani al CT InterpretationSchri ger et al  1293 ©1998 American Medical Association. All rights reserved. Downloaded From: http://jama.jamanetwork.com/ on 02/25/2013

Upload: anggun-permatasari

Post on 03-Jun-2018

215 views

Category:

Documents


0 download

TRANSCRIPT

8/12/2019 e0b49518b67a740d30

http://slidepdf.com/reader/full/e0b49518b67a740d30 1/5

Cranial Computed TomographyInterpretation in Acute StrokePhysician Accuracy in Determining Eligibilityfor Thrombolytic Therapy

David L. Schriger, MD, MPH; Mary Kalafut, MD, MS; Sidney Starkman, MD;Michelle Krueger, MD; Jeffrey L. Saver, MD

Context.—Intracranial hemorrhage must be excluded prior to administration ofthrombolytic agents in acute stroke.

Objective.—To evaluate physician accuracy in cranial computed tomographyscaninterpretationfor determiningeligibility for thrombolytic therapy in acute stroke.

Design.—Administration of randomly selected, randomly ordered series of 15computed tomography scans from a pool of 54 scans that demonstrated intrace-rebral hemorrhage, acute infarction, intracerebral calcifications (impostor for hem-orrhage), old cerebral infarction (impostor for acute infarction), and normal findings.

Participants.—A convenience sample of 38 emergency physicians, 29 neu-rologists, and 36 general radiologists.

Main Outcome Measures. —Physician determination of eligibility for thrombo-lytic therapy based on computed tomography scan interpretation.Results.—Average correct score by all physicians on all computed tomography

scans was 77% (95% confidence interval, 74%-80%). Of 569 computed tomogra-phy readings by emergency physicians, 67% were correct; of 435 readings byneurologists, 83% were correct; and of 540 readings by radiologists, 83% werecorrect. Overall sensitivity for detecting hemorrhage was 82% (95% confidence in-terval, 78%-85%); 17% of emergency physicians, 40% of neurologists, and 52% ofradiologists achieved 100% sensitivity for identification of hemorrhage.

Conclusion.—Physicians in this study did not uniformly achieve a level of sen-sitivity for identification of intracerebral hemorrhage sufficient to permit safe selec-tion of candidates for thrombolytic therapy.

JAMA. 1998;279:1293-1297

From the Emergency Medicine Center (DrsSchriger, Starkman, and Krueger) and the Depart-ment of Neurology (Drs Kalafut, Starkman, andSaver), School of Medicine, and the Stroke Center(Drs Kalafut, Starkman, and Saver), University ofCalifornia at Los Angeles.

Dr Starkman has received contracts and grants asan investigator in acute stroke studies sponsored byGenentech Inc, South San Francisco, Calif, andBoehringer-Ingelheim Pharmaceuticals Inc, Ridge-field,Conn. He also is a memberof theAcuteIschemicStroke Advisory Board and the Stroke Speakers’ Bu-reau, and he has participated in stroke conferencessponsored by Genentech Inc and Boehringer-Ingelheim Pharmaceuticals Inc.

Dr Saver has participated as an investigator in re-search trials funded byGenentech Incand Boehringer-Ingelheim Pharmaceuticals Inc, and has receivedhonoraria from Genentech Inc for educationalspeeches. He also has served on a scientific advisoryboard for Boehringer-Ingelheim Pharmaceuticals Inc.

Roche Laboratories Inc,Nutley,NJ, hasa substantialinterest in GenentechInc. Thesecompanies havesup-ported theactivities of theUniversity of Californiaat LosAngeles Stroke Center with unrestricted educationalgrants.

Corresponding author: David L. Schriger, MD, MPH,924WestwoodBlvd, Suite 300,LosAngeles,CA 90024-2924 (e-mail: [email protected]).

RECOMBINANT tissue plasminogenactivator has been approved for selectpatients within 3 hours of onset of acuteischemic stroke.1 Since thrombolytictherapy may produce lethal bleeding inpatients with intracranial hemorrhage,the presence of intracranialbloodon theinitial computed tomography (CT) scanhas been an exclusion criterion in the5 trials of thrombolytic therapy forstroke2-6 and in the recommendations of 

expert panels.7-9

Thus, before thrombo-lytic therapy may be given in stroke, aphysicianhighlyskilledin identifying in-tracranial hemorrhage must interpretthe CT scan.

For editorial comment see p 1307.

Early signs of major cerebral infarc-tion (sulcal effacement, mass effect, andedema) also are associated with an in-creased risk for intracerebral hemor-rhage in patients who receive thrombo-lytictherapy,10-12 andsomeguidelinesrec-ommend avoiding thrombolytic therapywhen these findings are present.7-9

We conducted this study to determinehow well emergency physicians, neurolo-gists, and general radiologists identifiedcranial CT scans that have evidence of in-tracranial hemorrhage. We also assessedthese physicians’ accuracy in interpret-ing other CT findings commonly seen inpatients with acute ischemic stroketo ex-amine whether physicians could distin-guish CT scans that demonstrate subtleimaging abnormalities representing po-

JAMA, April 22/29, 1998—Vol 279, No. 16 Cranial CT Interpretation—Schriger et al   1293

©1998 American Medical Association. All rights reserved.

wnloaded From: http://jama.jamanetwork.com/ on 02/25/2013

8/12/2019 e0b49518b67a740d30

http://slidepdf.com/reader/full/e0b49518b67a740d30 2/5

tential contraindications to thromboly-tics from those that do not preclude safeadministration of these agents.

METHODS

Development of Scan Library

We reviewed interpretations of allcranial CT scans performed at a univer-sity teaching hospital from December1994 to January 1996 to identify scansexhibiting hemispheric parenchymalhemorrhage or early infarction. Scanswithcalcifications (usedas impostorsforhemorrhage), scans with old infarction(used as impostors for acute infarction),and normal scans without calcificationalso were identified. Scans with other

abnormalities (including subarachnoidand extracerebral hemorrhage), scanswith multiple findings, and scans thatcould not be definitively placed in 1 of these categories were excluded. Onlyscansfor which there wasunanimous di-agnostic agreement among the authors(anda consultingneuroradiologist) wereincluded.Using a consensus process, weclassified each scan that demonstratedhemorrhage as easy or difficult to inter-pret and each scan that demonstratedacute infarction as easy, intermediate,or difficult to interpret based on thesubtlety of the findings.

Subject RecruitmentThirty-eight emergency physicians

were recruited at the Scientific Assem-bly of the California Chapter of theAmerican College of Emergency Physi-cians in May 1996. Seventeen commu-nity-basedneurologistswhoperiodicallyattend at a university hospital neurol-ogyclinicweretestedinprivatesessionsand 12 additional neurologists weretested while attending the University of California at Los Angeles (UCLA)

Stroke Center symposium, held in Octo-ber 1996. Seven radiologists were con-tacted through the physician directors

of local community hospital radiologydepartments and tested at their hospi-tals. Twenty-nine other radiologistswere tested at the 49th Annual Midwin-ter Radiological/OncologyConference of the Los Angeles Radiological Society,held in January 1997. Board-certifiedand board-eligible radiologists who didnot have additional training in neurora-diology were tested.

Atthe meetings, a booth was set upinthe exhibits area of the conference witha sign reading “Test Your Skills at CT.”Subjects included physicians who spon-taneously approached the booth andthose who, when approached, agreed toparticipate in the study. At conferencetestsites,subjectswereofferedaT-shirtas an incentive to participate and wereinformed that the physician in each spe-cialty with the highest score would re-ceiveatextbookofhisorherchoice.Phy-sician-subjects who gave verbal in-formedconsenttoparticipatecompleteda single-sheet questionnaire regardingtheir age, years of clinical experience,residency training, board certification,and typical involvement in the readingofcranialCTscans.Aftercompletingthequestionnaire, each subject was shownthe CT scans on a view box in an indi-

vidual session with unlimited time tointerpret each scan. The study was ap-proved by the UCLA Institutional Re-view Board.

Scan Presentation

Physician-subjects were asked to as-sume that each scan was of a patient whoarrived atthe hospitalwithin thefirst fewhours after the onset of an acute hemi-spheric neurologic deficit (eg, aphasia,hemiparesis). As each CT scan was pre-

sented, the physician-subject was toldwhich side of the patient’s body was af-fected. Subjects were asked to accept

that each patient was eligible for throm-bolytictherapyprovidedtheCTscanhadno contraindications. For each patient,thephysician wasasked, “Basedsolely onscan findings, could thrombolytics beadministered to this patient?” Answerchoices were (1) yes; (2) no, because of hemorrhage; or (3) no,because of signs of acute infarction. A list of contraindica-tions (hemorrhage, early hypodensity,mass effect, and shift) and a list of find-ings that did not preclude the adminis-trationof thrombolytics (calcification, at-rophy, and old infarction) were providedto remind subjects of the criteria forthisstudy. Subjects were informed that 20%to 60% of the scans would have no con-traindication to thrombolytic therapy.

Eachsubjectwaspresentedwith5ini-tial scans: 2 difficult hemorrhages, 1 in-termediate acute infarction, 1 impostor,and 1 normal (Figure). Subjects who re-sponded correctly to all 5 scans wereplaced in an advanced track and werethen presented with 10 scans: 3 difficulthemorrhages, 3 difficult acute infarc-tions, 1 intermediate acute infarction, 2impostors, and 1 normal. Subjects whorespondedincorrectlyto1ormoreofthefirst 5 scans were placedin the standardtrackandpresentedwith3difficulthem-

orrhages,1 easyhemorrhage,1 interme-diate acute infarction, 2 easy acute in-farctions,1impostor,and2normalscans.The2-trackstrategywasdesignedtoen-sure that subjects were given scans thatwereappropriate to theirskilllevels andwould maximize the discriminative ca-pacity of the test.13,14

Testing Protocols

Two hundred protocols, each contain-ing a script of 25 scans, were created.

A B C D

Easy Hemorrhage,24 (100%) of 24

Difficult Hemorrhage,25 (76%) of 33

Easy Acute Infarction,17 (81%) of 21

Intermediate Acute Infarction,23 (53%) of 43

Single slices from 4 axial computed tomography (CT) scans. These illustrative slices depict the pathology that the physician needed to recognize. Subjects wereshown the patient’s entire scan in standard format. Data shown are number (percentage) of correct readings for each scan.

1294   JAMA, April 22/29, 1998—Vol 279, No. 16 Cranial CT Interpretation—Schriger et al

©1998 American Medical Association. All rights reserved.

wnloaded From: http://jama.jamanetwork.com/ on 02/25/2013

8/12/2019 e0b49518b67a740d30

http://slidepdf.com/reader/full/e0b49518b67a740d30 3/5

Each protocol specified that a subjectmust receive an initial series of 5 scans,with the scans to be included and theirorder of presentation determined usingthe “uniform”random numbers functionof STATA 5.0 (Stata Corp, College Sta-tion, Tex). By a similar method, addi-tional series of 10 scans each were pre-pared for the standard and advancedtracks.

Statistical Methods

We designed the experiment to pro-videstableestimatesoftheperformanceof each subject, the difficulty of eachscan, and the sensitivity for detectinghemorrhage, without unduly burdeningthe volunteers. Using a conservativesimulation, we determined that 30 sub-

 jects per specialty, each reading 15scans, would meet these goals. Ninety-five percent confidence intervals (CIs)surrounding percent correct valueswere calculated using robust clusteredlogistic regression, which accounts for

the fact that the scan readings may beassociated with the skills of the readerand, therefore, are not completely inde-pendent.15

RESULTS

All 29 neurologists, all 36 radiologists,and 74%of the 38 emergencyphysicianswereboard-certified(another13%oftheemergency physicians were senior resi-dents in emergency medicine). Emer-gency physicians averaged 9 years inpostresidency practice and 36 clinicalhours per week, neurologists averaged13 years in practice and 42 clinical hours

per week, and radiologists averaged 15years in practice and 42 clinical hoursper week.Twenty-four percent of emer-gency physicians routinely read cranialCT scans. All neurologists reportedreading CT scans; roughly half of themdid so before seeing the radiologist’s re-port. All of the radiologists spent sometime reading cranial CT scans (15% of clinical practice on average), although22% reported that the interpretation of cranial CT and magnetic resonance im-aging scans constituted less than 5% of their practice.

There were few violations in protocol.One emergency physician was inadvert-ently given only 14 scans.One radiologistwas placed in the standard track despiteachieving a perfect score on the first 5scans. Also, during testing of the firstgroup of radiologists it became clear that2 scans, an intermediate acute infarctionand a difficult hemorrhage, were causingunintended confusion. The difficult hem-orrhage hadan area that could have beenread as infarction, and subjects were ret-rospectivelygivencreditfor a correct an-swer provided theydidnot score thescan

as “give thrombolytics.” As a result of this, another radiologist was mistakenlyplaced in the standard track. These 2

cases were removed and replaced withalternate scans that did not have theseambiguities.

Among all103 physicians, 80 (78%) in-correctly interpreted at least 1 of the 5scans in the initial series (Table 1). Fiveemergency physicians, 6 neurologists,and 10 radiologists correctly respondedto all 5 scans and entered the advancedtrack, achieving average total scores of 67%, 80%, and 84%, respectively. Thus,even the best performers misinter-preted a substantial number of scans.

Across all examinations, neurologistsand radiologistswere 100%accurate for

identifying easy hemorrhages. Emer-gencyphysiciansidentified94%(95%CI,84%-98%)ofthese(Table2).Fordifficulthemorrhages, emergency physicianscorrectly read 56% (95% CI, 46%-67%),neurologistscorrectlyread78%(95%CI,68%-85%), and radiologists correctlyread 80% (95% CI, 71%-87%), for anoverall sensitivity for hemorrhage of 82% (95% CI, 78%-85%). Fifty-two per-cent of radiologists, 40% of neurologists,and 17% of emergency physicians cor-rectly identified all CT scans with evi-dence of hemorrhage.

Average correct score by all physi-cians on all examinations was 77%(95% CI, 74%-80%) (Table 1). Overall,of 569 CT scan readings by emergencyphysicians, 67% were correct; of 435readings by neurologists, 83% werecorrect; and of 540 readings by radiolo-gists, 83% were correct. Performanceby neurologists and radiologists wasweakly inversely related to years inpractice. Board certification in emer-gency medicine did not predict higherperformance. Emergency physicianswho routinely read the cranial CT

scans they ordered scored higher thanthose who did not (71% vs 59%; differ-ence of 12%; 95% CI, 4%-21%).

Percent correct by scan type de-creased sequentially from easy hemor-rhages (98%correct)througheasy acuteinfarctions, normal scans, normal scanswith calcification, difficulthemorrhages,intermediate acute infarctions, normalscans with old infarctions, and difficultacuteinfarctions(40%correct)(Table2).The correlation of percent correct withscan difficulty (easy, intermediate, ordifficult) substantiatesthevalidity ofourclassification schema.

Subjects in the advanced track (thosewho responded correctly to the first 5scans) consistently scored higher on

each type of scan than those in the stan-dard track (Table 3). For all but those inthe lowest stratum of overall perfor-mance, easy hemorrhages and easyacute infarctions were read with nearperfect sensitivity. Scores for othertypes of scans improved with increasingoverall skill.

COMMENT

In the United States, recombinant tis-sue plasminogen activator is approvedfor use in acute ischemic stroke whenadministered within 180 minutes of symptom onset.1 Duringthis time inter-valthepatientmustrecognizethesymp-toms, get to a hospital, undergo evalua-tion, and have a CT scan performed andinterpreted. There will be situationswhen the 3-hour limit is rapidly ap-proaching, no neuroradiologist is avail-able, and an emergency physician, neu-rologist,orgeneralradiologististheonlyphysician available to interpret the CTscan.Ourstudyaskswhetherphysiciansin these specialties are capable, withoutadditional training, of interpreting theCT scan with sufficient sensitivity to de-

Table 1.—Performance on First 5 Scans and All Scans by Subject

No. (%) Correct

% of Physicians

Emergency Medicine(n = 38)

Neurology(n = 29)

General Radiology(n = 36) Total

First 5 scans*0 0 0 3 1

1 (20) 5 0 3 3

2 (40) 26 3 8 14

3 (60) 37 28 22 29

4 (80) 18 48 31 315 (100) 13 21 33 22

Average score (SD) 62 (22) 77 (16) 75 (25) 71 (23)

All 15 scans0-7 (0-46) 8 0 0 3

8-9 (53-60) 34 0 8 15

10-11 (67-73) 34 21 14 24

12-13 (80-87) 24 55 56 43

14-15 (93-100) 0 24 22 15

Average score (SD) 67 (13) 83 (9) 83 (11) 77 (14)

*Scans were 2 difficult hemorrhages, 1 intermediate acute infarction, 1 normal scan, and 1 impostor.

JAMA, April 22/29, 1998—Vol 279, No. 16 Cranial CT Interpretation—Schriger et al   1295

©1998 American Medical Association. All rights reserved.

wnloaded From: http://jama.jamanetwork.com/ on 02/25/2013

8/12/2019 e0b49518b67a740d30

http://slidepdf.com/reader/full/e0b49518b67a740d30 4/5

termine if thrombolytics may be admin-istered safely.

Overall sensitivity for intracerebralhemorrhage, an absolute contraindica-tion to thrombolytic therapy, was 82%.Twolevelsofsensitivityforhemorrhage

may be expected of physicians who in-terpret cranial CT scans, one based on asingle-case perspective, the other basedon a population perspective. From thesingle-case perspective, the administra-tion of a thrombolytic agent to a patientwith an intracerebral hemorrhage maybe lethalor have other catastrophiccon-sequences. The potential consequencesto the patient and the physician of ad-ministering thrombolytic therapy afterfailing to recognize hemorrhage on theCT scan mandate that sensitivity for in-tracerebral hemorrhage be extremelyhigh, certainly over 95% and ideallyhigher than 99%.

A population-based perspective pro-videsamorerelaxedrequirementforac-ceptable sensitivity. For instance,in the624-patientNational Instituteof Neuro-logical Disorders and Stroke study,there were 10 more survivors in thetreatment group than in the controlgroup at 3 months.4 Assuming that 15%ofallacutehemisphericstrokesarehem-orrhagic,16 that patients withsmall hem-orrhagic strokes have prognoses equalto those with ischemic stroke, and that

all patients with hemorrhagic strokewho receive thrombolytics die, any sen-sitivityfor hemorrhagehigher than 75%will preserve a potential net death ben-efit in the treated group (unpublisheddata, D.L.S.). By similar reasoning, the

sensitivity for hemorrhage may de-crease to as low as 22% before negatingimprovements in neurologic outcome asmeasured with the Barthel Index.

We weigh heavily thesingle-caseper-spective and believe that, on average,thesample of physicians in ourstudy didnot have the skills needed to recognizehemorrhage on CT scans and determinewhich patients may safely receivethrombolytic therapy. One implicationof this finding is that physicians in thesespecialties should not assume that thestandard of care dictates that theyshould all be able to make these deci-sions independently.

Sensitivity for identification of earlysigns of major infarction was variable.Easy acute infarctions were identifiedby most physicians, but even the bestperformers failed to identify a third of the difficult acute infarctions. The clini-calimportance of this finding in patientswithstrokewithinandbeyond3hoursof symptom onset remains to be deter-mined.10,17,18

Poorspecificitywoulddepriveeligiblepatients theopportunityto benefit from

thrombolytic therapy. While this is lesscatastrophic than administering throm-bolytic agents to patients with contrain-dications, our study demonstrates thatmany physicians will have trouble dif-ferentiating hemorrhage from calcifi-

cation and acute infarction from old in-farction.It is possible that our study results

could represent a biased estimate of na-tional average performance. Physiciansin our study may not be representativeof the national population of physiciansin each specialty, leading to samplingbias. Furthermore, physicians taking atest may achieve results different fromthosetheyachieveinactualpractice.Mo-tivation in the simulated testing situa-tion could be higher (eg, years of condi-tioning toperformwell, desireto winthetextbook)or lower(eg,no patient’slifeisat stake) than in clinical practice. Wecannot predict the direction of this bias.

The inability of participants to viewCT scans on a computer and measureimage density to differentiate calciumfrom blood may have adversely affectedspecificity on normal scans with calcifi-cations.Higher thanideal ambient light-inginthetestingareamayhavereducedoverall performance. We intentionallyprevented physicians from stating “un-sure” in response to our questions be-cause we were interested in determin-

Table 2.—Percent Correct by Scan Category

Type of ScanNo. ofScans

Emergency Physicians Neurologists General Radiologists

% CorrectNo. of

ReadingsRange Correct

by Scan, % % CorrectNo. of

ReadingsRange Correct

by Scan, % % CorrectNo. of

ReadingsRange Correct

by Scan, %

Normal 10 83 109 57-100 90 81 77-100 89 98 70-100

Imposter (calcification) 4 77 39 44-100 78 32 43-100 71 49 39-92

Imposter (old infarction) 4 43 42 27-80 59 32 42-100 70 33 20-88

Hemorrhage (easy) 8 94 98 85-100 100 69 100 100 78 100

Hemorrhage (difficult) 10 57 124 25-93 78 99 40-100 80 128 43-93

Acute infarction (easy) 4 82 45 66-100 100 34 100 93 29 80-100Acute infarction (intermediate) 8 46 97 14-97 77 70 42-100 85 95 33-100

Acute infarction (difficult) 8 13 15 0 44 18 0-100 50 30 0-100

Total 56*   67   569   . . .† 83   435   . . . 83   540   . . .

*There were 54 scans in the initial pool, and 2 were added when 2 problem scans were replaced.†Ellipses indicate data not applicable.

Table 3.—Percent Correct by Scan Type Stratified by Subject’s Overall Performance

nEasy

HemorrhageDifficult

HemorrhageEasy Acute

InfarctionIntermediate

Acute InfarctionDifficult Acute

Infarction NormalAcute Impostor

(Calcification)Impostor

(Old Infarction)

Advanced Track (N = 21*)

No. of scans read per subject . . .† 5 . . . 2 3 2 3

Highest scorers, %‡ 8 . . . 90 . . . 100 66 100 100 100

Lower scorers, % 13 . . . 86 . . . 88 23 96 83 50

Standard Track (N = 82)

No. of scans read per subject 3 3 2 2 . . . 3 2

Higher scorers, % 34 100 81 100 84 . . . 94 77 79

Intermediate scorers, % 32 100 60 98 60 . . . 84 59 38

Lowest scorers, % 16 88 35 59 29 . . . 67 67 29

*Although 23 subjects achieved perfect scores on the initial series, 2 were inadvertently placed in the standard track.†Ellipses indicate scan type not included in test.‡Subjects were stratified by overall percent correct into roughly equal groups.

1296   JAMA, April 22/29, 1998—Vol 279, No. 16 Cranial CT Interpretation—Schriger et al

©1998 American Medical Association. All rights reserved.

wnloaded From: http://jama.jamanetwork.com/ on 02/25/2013

8/12/2019 e0b49518b67a740d30

http://slidepdf.com/reader/full/e0b49518b67a740d30 5/5

ing what would happen if the physicianwere the only person available topromptly read the scan. This approachmost likely decreased specificity (thosephysicians who expressed uncertaintyabout a scan often scored it as infarctionorhemorrhagebecausetheydidnotwishto miss contraindications), but shouldnot have affected sensitivity.

The few breaks in protocol also could

affect our estimates, but any decrease inperformanceresultingfromthe2ambigu-ousscanswaslikelybalancedbythe2sub-

 jects who were mistakenly placed in thestandard track (where they likely scoredhigher than they would have in the ad-vancedtrack).Giventhemagnitudeofourresults,and the factthat manyphysicianswere able to attain a perfect sensitivityfor hemorrhage, we do not believe thatthese testing biases are large enough toalter our findings substantially.

Our convenience sampling method isthe main threat to the external validityof thestudy. While board certification is

an imperfect proxy forskill at interpret-

ingcranialCTscans,allneurologistsandradiologists were board certified, andthe percentage of board-certified emer-gencyphysiciansinthesampleexceededthe national percentage of full-timeemergency physicians who are boardcertified.Whilewecannotprovethatoursample was representative of the na-tional population of physicians in thesespecialties, we observed that physicians

who acknowledged that they were un-comfortable reading CT scans oftendeclined participation, suggesting thatour study most likely did not underesti-mate physician performance. In addi-tion, overall physician performance cor-related with performance on each scantype, which suggests that the examina-tion had construct validity.

In conclusion, it appears that whilesome members of each of these physi-cian groups are capable of identifyinghemorrhage with perfect or near-per-fect sensitivity, the majority of thosetested are not. Board certification in

emergency medicine,neurology, or gen-

eral radiology is an inadequate markerfor such competence. Physicians in-volved in the care of patients with acutestroke should ensure that the interpre-tion of the CT scan reliably identifies in-tracranial hemorrhage when present.This may be accomplished by providingphysicianswith enhanced trainingin theinterpretation of cranial CT scans or byimplementing teleradiography or other

systems that facilitate immediate scaninterpretation by qualified readers.

The National Stroke Association (NSA) allowedus to use their exhibit booth for execution of thisstudy, and underwrote the cost of the T-shirts andmedical books. The NSA supported these activitieswith funds provided by an educational grant fromKnoll Pharmaceutical Company, Whippany, NJ.

Weofferoursincerethankstothephysicianswhoparticipated in thisstudy. We alsothank WilliamH.Rogers, PhD, for help with the modeling of samplesizeconsiderations; Michael Zucker,MD, andEmilyNicklinfor helpfindingand duplicatingscans;PabloVillablanca, MD, neuroradiologist, for aid in scaninterpretations; Danica Barron for data entry anddata checking; and Julea Leshar McGhee for help

withrecruitment andenrollment at thestudy sites.

References

1.  Nightingale SL. t-PA approved for acute ische-mic stroke [From the Food and Drug Administra-tion]. JAMA. 1996;276:443.2.   Hacke W, Kaste M, Fieschi C, et al. Intravenousthrombolysiswith recombinant tissueplasminogenactivator for acute hemispheric stroke: the Euro-pean Cooperative Acute Stroke Study (ECASS).JAMA. 1995;274:1017-1025.3.  Multicenter Acute Stroke Trial–Italy (MAST-I)Group.Randomisedcontrolledtrial of streptokinase,aspirin, andcombinationof both intreatmentof acuteischaemic stroke. Lancet. 1995;346:1509-1514.4.   NationalInstituteof Neurological DisordersandStrokert-PAStrokeStudyGroup.Tissueplasmino-gen activator for acute ischemic stroke.  N Engl J Med. 1995;333:1581-1587.

5.   Donnan GA, Davis SM, Chambers BR, et al.Streptokinase for acute ischemic stroke with rela-tionship to timeof administration.JAMA. 1996;276:961-966.6.   Multicenter Acute Stroke Trial–Europe StudyGroup. Thrombolytic therapy withstreptokinaseinacute ischemic stroke.   N Engl J Med. 1996;335:145-150.

7.  Special Writing Committee of the Stroke Coun-cil, American Heart Association. Guidelines forthrombolytic therapy foracute stroke. Stroke. 1996;27:1711-1718.8.  Quality Standards Subcommittee of the Ameri-can Academy of Neurology. Practice advisory:thrombolytic therapy for acute ischemic stroke:summary statement. Neurology. 1996;47:835-839.9.   National Stroke Association consensus state-ment:stroke:the firsthours: emergencyevaluationand treatment guidelines. In:  Stroke Clinical Up-dates. Englewood, Colo: National Stroke Associa-tion; 1997.10.  National Institute of Neurological Disordersand Stroke rt-PA Stroke Study Group. Intracere-bralhemorrhageafterintravenoust-PAtherapyfor

ischemic stroke. Stroke. 1997;28:2109-2118.11.  Moulin T, Besson G, Crepin-Leblond T, et al.HemorrhagictransformationsinMAST-Etrial:pre-dictive factors [abstract].   Cerebrovascular Dis.1996;6:182.12.  Larrue V,von KummarR, delZoppo G,Bluhm-kiE. Hemorrhagic transformationin acuteischemicstroke: potential contributing factors in the Eu-

ropean Cooperative Acute Stroke Study.  Stroke.1997;28:957-60.13.   Ebel RL, Frisbie DA.   Essentials of Educa-tional Measurement. 4th ed. Englewood Cliffs, NJ:Prentice-Hall International Inc; 1986:335-339.14.  Wainer H, Dorans NJ, Flangher R, et al. Com-

 puterized Adaptive Testing: A Primer . Hillsdale,NJ: Lawrence Erlbaum Associates Inc Publishers;1990.15.   Stata Corporation.  Stata 5.0 Statistical Soft-

 wareUser’s Guide.CollegeStation,Tex:StataCorp;1997:235-239.16. SaccoR. Classificationof stroke. In:FisherM, ed.Clinical Atlas of Cerebrovascular Disorders.  Lon-don, England: Mosby–Year Book Inc; 1994:2.1-2.23.17.   von Kummer R, Allen KL, Holle R, et al. Acute

stroke: usefulness of early CT findings beforethrombolytic therapy.   Radiology.   1997;205:327-333.18.  National Institute of Neurological Disordersand Stroke rt-PA Stroke Study Group. General-ized efficacy t-PA for acute stroke: subgroupanalysis of the NINDS stroke trial.  Stroke. 1997;28:2119-2125.

JAMA, April 22/29, 1998—Vol 279, No. 16 Cranial CT Interpretation—Schriger et al   1297

©1998 American Medical Association. All rights reserved.