comparison of computer display monitors for computed radiography diagnostic application in a...
TRANSCRIPT
148
Australasian Physical & Engineering Sciences in Medicine Volume 27 Number 3, 2004
SCIENTIFIC NOTE
Comparison of computer display monitors for computed
radiography diagnostic application in a radiology PACS
L. Sim, K. Manthey, P. Esdaile and M. Benson
Radiology Department, Princess Alexandra Hospital, Brisbane, Australia
AbstractA study to compare the performance of the following display monitors for application as PACS CR diagnostic
workstations is described. 1. Diagnostic quality, 3 megapixel, 21 inch monochrome LCD monitors. 2. Commercial
grade, 2 megapixel, 20 inch colour LCD monitors. Two sets of fifty radiological studies each were presented separately
to five radiologists on two occasions, using different displays on each occasion. The two sets of radiological studies
were CR of the chest, querying the presence of pneumothorax, and CR of the wrist, querying the presence of a scaphoid
fracture. Receiver Operating Characteristic (ROC) curves were constructed for diagnostic performance for each
presentation. Areas under the ROC curves (AUC) for diagnosis using different monitors were compared for each image
set and the following results obtained: Set 1: Monochrome AUC = 0.873 +/- 0.026; Colour AUC = 0.831 +/-
0.032; Set 2: Monochrome AUC = 0.945 +/- 0.014; Colour AUC = 0.931 +/- 0.019; Differences in AUC were
attributed to the different monitors. While not significant at a 95% confidence level, the results have supported a
cautious approach to consideration of the use of commercial grade LCD colour monitors for diagnostic application.
Key w ords PACS, CR, Diagnostic monitor, receiver
operating characteristic
Introduction
Recent developments in image display technology have
created ready availability of good quality, high resolution,
commercial grade LCD colour monitors. There is increased
interest in using these monitors for diagnostic applications
in radiology diagnostic workstations as they offer a large
cost advantage over the high resolution, high brightness
monochrome displays more conventionally deployed in
diagnostic workstations.
Princess Alexandra Hospital implemented a Picture
Archive Communication System (PACS) in 1999/2000.
High brightness monochrome CRT monitors (1megapixel
and 1.5megapixel (Mp)) were provided for diagnostic
workstation application at that time. Replacement of these
monitors is now coming due and has stimulated this study
to evaluate the feasibility of using commercial grade colour
LCD monitors for computed radiography (CR) diagnostic
workstation application.
Corresponding author: L. Sim, PACS Support Office, RadiologyDepartment, Princess Alexandra Hospital, Ipswich Road,Woolloongabba, Brisbane, Qld, 4102, AustraliaTel: 07 3240 7411, Fax: 07 3240 7357Email: [email protected]: 16 June 2004; Accepted: 3 September 2004Copyright © 2004 ACPSEM/EA
Material and methods
Two types of monitors have been compared in this
study:
1. Diagnostic quality, 3 Mp, 21 inch monochrome LCD
with brightness set to 300 cd m-2.
2. Commercial grade, 2 Mp, 20 inch colour LCD with
Brightness set to 210 cd m-2.
A matched pair of each type of monitor was attached to an
Agfa DS3000 diagnostic workstation to provide two dual
screen workstations for the comparison study. The colour
monitors were adjusted using Verilum® software (available
at http://www.image-smiths.com) to deliver gamma
correction according to the DICOMTM Part 14 Grayscale
Standard Display Function1. The two monitors in each set
were respectively matched and calibrated for output
according to acceptance test procedures for Princess
Alexandra Hospital, with the exception of the brightness
criterion for the commercial grade monitor pair. Normal
acceptance is a maximum brightness of 300 cd m-2. These
monitors could only achieve 210 cd m-2 output.
Suitable CR images were selected from the PACS
archive, de-identified and assembled into two sets. A total
of 50 images per set were selected, with 25 classified as
“normal” and 25 classified as “positive” for the condition.
This classification was on the basis of the existing
radiology report. The reports had been produced during
normal clinical diagnosis with access to full patient details.
The readers in this study did not have access to the reports
or to the patient record. The image sets were:
Australas. Phys. Eng. Sci. Med. Vol. 27, No 3, 2004 Sim et al � PACS CR diagnostic monitor comparison
149
Set 1: Chest CR, selected for query pneumothorax.
Set 2: Wrist CR, selected for query scaphoid fracture.
The images were randomised in order of presentation to
each of five radiologists. Each radiologist viewed each
image set using one type of monitors and then the other
after a suitable time delay. Presentation order was reversed
for the second reading and the radiologists were not told
that the second reading set comprised the same images as
the first. The times taken for the radiologists to complete
the reading tasks for each image set were recorded.
The radiologists were asked to classify the studies as
“normal” or “positive” using a five point rating scale for the
condition. Data for analysis was classified as:
5. Definitely present
4. Probably present
3. Possibly present
2. Probably not present
1. Definitely not present
Data from each radiologist was combined for each
image set on each monitor type, and four ROC2,3,4,5, curves
were constructed, using ROCIT6 and JROCfit7 for
analysis. The radiology report based classification of
positive or negative for the condition provided the standard
for the ROC analysis.
The physical environment for each workstation was
standardised as far as possible. The DS3000 workstations
were identical and the inferences drawn from the results
rely on a basic assumption that the only differences in the
results of the two readings by each radiologist are due to the
different monitor types.
Results
The ROC curves for each image set, read on the two
monitor types are compared in Figure 1 and Figure 2. The
mean AUC for each curve is tabulated in Table 1, along
with the standard error. The differences in mean AUC for
each image set read on the two monitor types and the
derived z values indicating the level of statistical
significance for the differences are also tabulated. The
calculated z values are corrected for correlation between
image sets using the method of Hanley and McNeil8.
The mean times required to complete the 50 study
reading tasks for each monitor type were:
Monochrome: 26.6 minutes (SD = 10.8 minutes)
Colour LCD: 27.2 minutes (SD = 9.1 minutes)
Discussion
Inspection of Figure1 and Figure 2 demonstrates
differences in plotted ROC curves produced from data
obtained using the colour monitors against data obtained
using the monochrome monitors for both image sets. The
higher value of AUC obtained with the monochrome
monitors is consistent with expectations, i.e. the
monochrome monitors have a higher intrinsic spatial
resolution, higher brightness specification and are produced
C R - C he s t
0
0 . 2
0 . 4
0 . 6
0 . 8
1
0 0 . 2 0 . 4 0 . 6 0 . 8 1
F P F
TPF
M o n oC o l o u r
Figure 1. Comparison of ROC curves for each monitor type(colour and monochromatic) for the chest CR image set. (TPF –True Positive Fraction; FPF – False Positive Fraction).
C R - S c a p h o i d F r a c t u r e
0
0 . 2
0 . 4
0 . 6
0 . 8
1
0 0 . 2 0 . 4 0 . 6 0 . 8 1
F P F
TPF
M o n oC o l o u r
Figure 2. Comparison of ROC curves for each monitor type(colour and monochromatic) for the scaphoid fracture CR imageset. (TPF – True Positive Fraction; FPF – False PositiveFraction).
Image setAUC (SE)
(Colour)
AUC (SE)
(Mono)�(AUC) z value
Chest CR 0.831 (0.032) 0.873 (0.026) 0.042 1.044
Scaphoid 0.931 (0.019) 0.945 (0.014) 0.014 0.560
(z = 1.96 corresponds to a 95% confidence level)
Table 1. Areas under the Receiver Operating Characteristic Curves (AUC) for two image sets (chest and scaphoid) fortwo monitor types (colour and monochrome). The Standard Error (SE) of the AUC is calculated taking into accountcorrelation between the AUCs according to the method of Hanley & McNeil8. �(AUC) is the difference between thetwo areas and z indicates the statistical significance of this difference in areas.
Australas. Phys. Eng. Sci. Med. Vol. 27, No 3, 2004 Sim et al � PACS CR diagnostic monitor comparison
150
specifically to the requirements of the medical imaging
industry.
There was no significant difference in the time required
to complete the reading tasks on the different monitors.
The standardisation of reporting environment together
with the experimental design that saw the same radiologists
reading the same studies independently on the two monitor
types, supports the inference that differences in results are
due to the differences in the display monitors. However, the
statistical significance of these results as quantified by the z
value is not high, and in the case of the scaphoid fracture
image set, is considered equivocal. For the chest image set,
the calculated z value, on a one directional test of
significance, implies that this result would be achieved, by
chance alone, approximately once in 7 samples. For the
scaphoid fracture image set the corresponding figure is
once in approximately 3.5 samples.
The absence of strong statistical significance in these
results may be due to a number of factors, including:
� No significant performance difference between
monitor types for the study sets.
� The nature of the studies selected (i.e. for the
scaphoid fractures the threshold for classification
as positive or negative may be low as a function of
the actual clinical condition).
� The comparison standard used. In this study the
original radiologist report classification was taken
as the standard. Given inter radiologist variation of
interpretation; a more effective standard may be
classification by a panel of radiologists.
These results point to a need for further investigation
using more highly discriminating image content.
Conclusions
Differences in AUC for the chest image set are
attributed to the different monitors. Differences in the AUC
for the scaphoid fracture image set are considered
equivocal. The results for the CR Chest images have
supported a cautious approach at Princess Alexandra
Hospital, to consideration of the use of commercial grade
LCD colour monitors for CR based diagnostic application.
Acknowledgements
The authors gratefully acknowledge the efforts of the
radiology staff from Princess Alexandra Hospital in
completing the study reads, the provision by Agfa Geveart
of the two DS3000 workstations used in this study and the
acceptance testing of all monitors used, by the Biomedical
Technology Services group within Queensland Health.
References
1. NEMA, Digital Imaging and Communications in Medicine(DICOM) Part 14: Grayscale Standard Display Function, PS
3.14, 2003.
2. Metz, C. E., Basic Principles of ROC Analysis, Seminars in
Nuclear Medicine. 8(4): pp283-298, 1978.
3. Hanley, J. A., Receiver Operating Characteristic (ROC)Methodology, Critical Reviews in Diagnostic Radiology.
29(3): pp307-335, 1989.
4. Swaving, M., van Houweingen, H., Ottes, F. P., and
Steerneman, T., Statistical Comparison of ROC Curves fromMultiple Readers, Medical Decision Making. 16(2) pp143-
152, 1996.
5. Park, S. H., Goo, J. M. and Jo, C., Receiver OperatingCharacteristic (ROC) Curve: Practical Guide forRadiologists, Korean, J Radiol. 5(1): pp11-18, 2004.
6. ROCIT, http://xray.bsd.uchicago.edu/cgi-bin/roc_software.cgi,
accessed June 2004.
7. JROCfit, http://www.rad.jhmi.edu/roc, accessed June 2004.
8. Hanley, J. A. and McNeil, B. J., A Method of Comparing theAreas Under receiver Operating Characteristic CurvesDerived from the Same Cases, Radiology. 148(3): pp839-843,
1983.