comparison of computer display monitors for computed radiography diagnostic application in a...

3
148 Australasian Physical & Engineering Sciences in Medicine Volume 27 Number 3, 2004 SCIENTIFIC NOTE Comparison of computer display monitors for computed radiography diagnostic application in a radiology PACS L. Sim, K. Manthey, P. Esdaile and M. Benson Radiology Department, Princess Alexandra Hospital, Brisbane, Australia Abstract A study to compare the performance of the following display monitors for application as PACS CR diagnostic workstations is described. 1. Diagnostic quality, 3 megapixel, 21 inch monochrome LCD monitors. 2. Commercial grade, 2 megapixel, 20 inch colour LCD monitors. Two sets of fifty radiological studies each were presented separately to five radiologists on two occasions, using different displays on each occasion. The two sets of radiological studies were CR of the chest, querying the presence of pneumothorax, and CR of the wrist, querying the presence of a scaphoid fracture. Receiver Operating Characteristic (ROC) curves were constructed for diagnostic performance for each presentation. Areas under the ROC curves (AUC) for diagnosis using different monitors were compared for each image set and the following results obtained: Set 1: Monochrome AUC = 0.873 +/- 0.026; Colour AUC = 0.831 +/- 0.032; Set 2: Monochrome AUC = 0.945 +/- 0.014; Colour AUC = 0.931 +/- 0.019; Differences in AUC were attributed to the different monitors. While not significant at a 95% confidence level, the results have supported a cautious approach to consideration of the use of commercial grade LCD colour monitors for diagnostic application. Key w ords PACS, CR, Diagnostic monitor, receiver operating characteristic Introduction Recent developments in image display technology have created ready availability of good quality, high resolution, commercial grade LCD colour monitors. There is increased interest in using these monitors for diagnostic applications in radiology diagnostic workstations as they offer a large cost advantage over the high resolution, high brightness monochrome displays more conventionally deployed in diagnostic workstations. Princess Alexandra Hospital implemented a Picture Archive Communication System (PACS) in 1999/2000. High brightness monochrome CRT monitors (1megapixel and 1.5megapixel (Mp)) were provided for diagnostic workstation application at that time. Replacement of these monitors is now coming due and has stimulated this study to evaluate the feasibility of using commercial grade colour LCD monitors for computed radiography (CR) diagnostic workstation application. Corresponding author: L. Sim, PACS Support Office, Radiology Department, Princess Alexandra Hospital, Ipswich Road, Woolloongabba, Brisbane, Qld, 4102, Australia Tel: 07 3240 7411, Fax: 07 3240 7357 Email: [email protected] Received: 16 June 2004; Accepted: 3 September 2004 Copyright © 2004 ACPSEM/EA Material and methods Two types of monitors have been compared in this study: 1. Diagnostic quality, 3 Mp, 21 inch monochrome LCD with brightness set to 300 cd m -2 . 2. Commercial grade, 2 Mp, 20 inch colour LCD with Brightness set to 210 cd m -2 . A matched pair of each type of monitor was attached to an Agfa DS3000 diagnostic workstation to provide two dual screen workstations for the comparison study. The colour monitors were adjusted using Verilum® software (available at http://www.image-smiths.com) to deliver gamma correction according to the DICOM TM Part 14 Grayscale Standard Display Function 1 . The two monitors in each set were respectively matched and calibrated for output according to acceptance test procedures for Princess Alexandra Hospital, with the exception of the brightness criterion for the commercial grade monitor pair. Normal acceptance is a maximum brightness of 300 cd m -2 . These monitors could only achieve 210 cd m -2 output. Suitable CR images were selected from the PACS archive, de-identified and assembled into two sets. A total of 50 images per set were selected, with 25 classified as “normal” and 25 classified as “positive” for the condition. This classification was on the basis of the existing radiology report. The reports had been produced during normal clinical diagnosis with access to full patient details. The readers in this study did not have access to the reports or to the patient record. The image sets were:

Upload: l-sim

Post on 10-Jul-2016

218 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Comparison of computer display monitors for computed radiography diagnostic application in a radiology PACS

148

Australasian Physical & Engineering Sciences in Medicine Volume 27 Number 3, 2004

SCIENTIFIC NOTE

Comparison of computer display monitors for computed

radiography diagnostic application in a radiology PACS

L. Sim, K. Manthey, P. Esdaile and M. Benson

Radiology Department, Princess Alexandra Hospital, Brisbane, Australia

AbstractA study to compare the performance of the following display monitors for application as PACS CR diagnostic

workstations is described. 1. Diagnostic quality, 3 megapixel, 21 inch monochrome LCD monitors. 2. Commercial

grade, 2 megapixel, 20 inch colour LCD monitors. Two sets of fifty radiological studies each were presented separately

to five radiologists on two occasions, using different displays on each occasion. The two sets of radiological studies

were CR of the chest, querying the presence of pneumothorax, and CR of the wrist, querying the presence of a scaphoid

fracture. Receiver Operating Characteristic (ROC) curves were constructed for diagnostic performance for each

presentation. Areas under the ROC curves (AUC) for diagnosis using different monitors were compared for each image

set and the following results obtained: Set 1: Monochrome AUC = 0.873 +/- 0.026; Colour AUC = 0.831 +/-

0.032; Set 2: Monochrome AUC = 0.945 +/- 0.014; Colour AUC = 0.931 +/- 0.019; Differences in AUC were

attributed to the different monitors. While not significant at a 95% confidence level, the results have supported a

cautious approach to consideration of the use of commercial grade LCD colour monitors for diagnostic application.

Key w ords PACS, CR, Diagnostic monitor, receiver

operating characteristic

Introduction

Recent developments in image display technology have

created ready availability of good quality, high resolution,

commercial grade LCD colour monitors. There is increased

interest in using these monitors for diagnostic applications

in radiology diagnostic workstations as they offer a large

cost advantage over the high resolution, high brightness

monochrome displays more conventionally deployed in

diagnostic workstations.

Princess Alexandra Hospital implemented a Picture

Archive Communication System (PACS) in 1999/2000.

High brightness monochrome CRT monitors (1megapixel

and 1.5megapixel (Mp)) were provided for diagnostic

workstation application at that time. Replacement of these

monitors is now coming due and has stimulated this study

to evaluate the feasibility of using commercial grade colour

LCD monitors for computed radiography (CR) diagnostic

workstation application.

Corresponding author: L. Sim, PACS Support Office, RadiologyDepartment, Princess Alexandra Hospital, Ipswich Road,Woolloongabba, Brisbane, Qld, 4102, AustraliaTel: 07 3240 7411, Fax: 07 3240 7357Email: [email protected]: 16 June 2004; Accepted: 3 September 2004Copyright © 2004 ACPSEM/EA

Material and methods

Two types of monitors have been compared in this

study:

1. Diagnostic quality, 3 Mp, 21 inch monochrome LCD

with brightness set to 300 cd m-2.

2. Commercial grade, 2 Mp, 20 inch colour LCD with

Brightness set to 210 cd m-2.

A matched pair of each type of monitor was attached to an

Agfa DS3000 diagnostic workstation to provide two dual

screen workstations for the comparison study. The colour

monitors were adjusted using Verilum® software (available

at http://www.image-smiths.com) to deliver gamma

correction according to the DICOMTM Part 14 Grayscale

Standard Display Function1. The two monitors in each set

were respectively matched and calibrated for output

according to acceptance test procedures for Princess

Alexandra Hospital, with the exception of the brightness

criterion for the commercial grade monitor pair. Normal

acceptance is a maximum brightness of 300 cd m-2. These

monitors could only achieve 210 cd m-2 output.

Suitable CR images were selected from the PACS

archive, de-identified and assembled into two sets. A total

of 50 images per set were selected, with 25 classified as

“normal” and 25 classified as “positive” for the condition.

This classification was on the basis of the existing

radiology report. The reports had been produced during

normal clinical diagnosis with access to full patient details.

The readers in this study did not have access to the reports

or to the patient record. The image sets were:

Page 2: Comparison of computer display monitors for computed radiography diagnostic application in a radiology PACS

Australas. Phys. Eng. Sci. Med. Vol. 27, No 3, 2004 Sim et al � PACS CR diagnostic monitor comparison

149

Set 1: Chest CR, selected for query pneumothorax.

Set 2: Wrist CR, selected for query scaphoid fracture.

The images were randomised in order of presentation to

each of five radiologists. Each radiologist viewed each

image set using one type of monitors and then the other

after a suitable time delay. Presentation order was reversed

for the second reading and the radiologists were not told

that the second reading set comprised the same images as

the first. The times taken for the radiologists to complete

the reading tasks for each image set were recorded.

The radiologists were asked to classify the studies as

“normal” or “positive” using a five point rating scale for the

condition. Data for analysis was classified as:

5. Definitely present

4. Probably present

3. Possibly present

2. Probably not present

1. Definitely not present

Data from each radiologist was combined for each

image set on each monitor type, and four ROC2,3,4,5, curves

were constructed, using ROCIT6 and JROCfit7 for

analysis. The radiology report based classification of

positive or negative for the condition provided the standard

for the ROC analysis.

The physical environment for each workstation was

standardised as far as possible. The DS3000 workstations

were identical and the inferences drawn from the results

rely on a basic assumption that the only differences in the

results of the two readings by each radiologist are due to the

different monitor types.

Results

The ROC curves for each image set, read on the two

monitor types are compared in Figure 1 and Figure 2. The

mean AUC for each curve is tabulated in Table 1, along

with the standard error. The differences in mean AUC for

each image set read on the two monitor types and the

derived z values indicating the level of statistical

significance for the differences are also tabulated. The

calculated z values are corrected for correlation between

image sets using the method of Hanley and McNeil8.

The mean times required to complete the 50 study

reading tasks for each monitor type were:

Monochrome: 26.6 minutes (SD = 10.8 minutes)

Colour LCD: 27.2 minutes (SD = 9.1 minutes)

Discussion

Inspection of Figure1 and Figure 2 demonstrates

differences in plotted ROC curves produced from data

obtained using the colour monitors against data obtained

using the monochrome monitors for both image sets. The

higher value of AUC obtained with the monochrome

monitors is consistent with expectations, i.e. the

monochrome monitors have a higher intrinsic spatial

resolution, higher brightness specification and are produced

C R - C he s t

0

0 . 2

0 . 4

0 . 6

0 . 8

1

0 0 . 2 0 . 4 0 . 6 0 . 8 1

F P F

TPF

M o n oC o l o u r

Figure 1. Comparison of ROC curves for each monitor type(colour and monochromatic) for the chest CR image set. (TPF –True Positive Fraction; FPF – False Positive Fraction).

C R - S c a p h o i d F r a c t u r e

0

0 . 2

0 . 4

0 . 6

0 . 8

1

0 0 . 2 0 . 4 0 . 6 0 . 8 1

F P F

TPF

M o n oC o l o u r

Figure 2. Comparison of ROC curves for each monitor type(colour and monochromatic) for the scaphoid fracture CR imageset. (TPF – True Positive Fraction; FPF – False PositiveFraction).

Image setAUC (SE)

(Colour)

AUC (SE)

(Mono)�(AUC) z value

Chest CR 0.831 (0.032) 0.873 (0.026) 0.042 1.044

Scaphoid 0.931 (0.019) 0.945 (0.014) 0.014 0.560

(z = 1.96 corresponds to a 95% confidence level)

Table 1. Areas under the Receiver Operating Characteristic Curves (AUC) for two image sets (chest and scaphoid) fortwo monitor types (colour and monochrome). The Standard Error (SE) of the AUC is calculated taking into accountcorrelation between the AUCs according to the method of Hanley & McNeil8. �(AUC) is the difference between thetwo areas and z indicates the statistical significance of this difference in areas.

Page 3: Comparison of computer display monitors for computed radiography diagnostic application in a radiology PACS

Australas. Phys. Eng. Sci. Med. Vol. 27, No 3, 2004 Sim et al � PACS CR diagnostic monitor comparison

150

specifically to the requirements of the medical imaging

industry.

There was no significant difference in the time required

to complete the reading tasks on the different monitors.

The standardisation of reporting environment together

with the experimental design that saw the same radiologists

reading the same studies independently on the two monitor

types, supports the inference that differences in results are

due to the differences in the display monitors. However, the

statistical significance of these results as quantified by the z

value is not high, and in the case of the scaphoid fracture

image set, is considered equivocal. For the chest image set,

the calculated z value, on a one directional test of

significance, implies that this result would be achieved, by

chance alone, approximately once in 7 samples. For the

scaphoid fracture image set the corresponding figure is

once in approximately 3.5 samples.

The absence of strong statistical significance in these

results may be due to a number of factors, including:

� No significant performance difference between

monitor types for the study sets.

� The nature of the studies selected (i.e. for the

scaphoid fractures the threshold for classification

as positive or negative may be low as a function of

the actual clinical condition).

� The comparison standard used. In this study the

original radiologist report classification was taken

as the standard. Given inter radiologist variation of

interpretation; a more effective standard may be

classification by a panel of radiologists.

These results point to a need for further investigation

using more highly discriminating image content.

Conclusions

Differences in AUC for the chest image set are

attributed to the different monitors. Differences in the AUC

for the scaphoid fracture image set are considered

equivocal. The results for the CR Chest images have

supported a cautious approach at Princess Alexandra

Hospital, to consideration of the use of commercial grade

LCD colour monitors for CR based diagnostic application.

Acknowledgements

The authors gratefully acknowledge the efforts of the

radiology staff from Princess Alexandra Hospital in

completing the study reads, the provision by Agfa Geveart

of the two DS3000 workstations used in this study and the

acceptance testing of all monitors used, by the Biomedical

Technology Services group within Queensland Health.

References

1. NEMA, Digital Imaging and Communications in Medicine(DICOM) Part 14: Grayscale Standard Display Function, PS

3.14, 2003.

2. Metz, C. E., Basic Principles of ROC Analysis, Seminars in

Nuclear Medicine. 8(4): pp283-298, 1978.

3. Hanley, J. A., Receiver Operating Characteristic (ROC)Methodology, Critical Reviews in Diagnostic Radiology.

29(3): pp307-335, 1989.

4. Swaving, M., van Houweingen, H., Ottes, F. P., and

Steerneman, T., Statistical Comparison of ROC Curves fromMultiple Readers, Medical Decision Making. 16(2) pp143-

152, 1996.

5. Park, S. H., Goo, J. M. and Jo, C., Receiver OperatingCharacteristic (ROC) Curve: Practical Guide forRadiologists, Korean, J Radiol. 5(1): pp11-18, 2004.

6. ROCIT, http://xray.bsd.uchicago.edu/cgi-bin/roc_software.cgi,

accessed June 2004.

7. JROCfit, http://www.rad.jhmi.edu/roc, accessed June 2004.

8. Hanley, J. A. and McNeil, B. J., A Method of Comparing theAreas Under receiver Operating Characteristic CurvesDerived from the Same Cases, Radiology. 148(3): pp839-843,

1983.