comparison of spatial resolution and contrast uniformity of … · department of radiology, faculty...
TRANSCRIPT
Comparison of Spatial Resolution and Contrast Uniformity of Various Printers
by
Milan Madhavji
A thesis submitted in conformity with the requirements for the degree of Masters of Science in Oral Radiology
Department of Radiology, Faculty of Dentistry University of Toronto
© Copyright by Milan Madhavji 2010
ii
Comparison of Spatial Resolution and Contrast Uniformity of
Various Printers
Milan Madhavji
Masters of Science in Oral Radiology
Department of Radiology, Faculty of Dentistry
University of Toronto
2010
Abstract
For several common inkjet, laser and thermal dye printers, a method of evaluating prints that is
not associated with the level of dental expertise of the observer is introduced. In addition, an
automated analysis that mimics the observations made by observers is tested. Metrics that are
evaluated in this study include spatial resolution, contrast uniformity, the type of paper, and
overall observer preference. The results demonstrate that observer preference is associated with a
high print contrast uniformity and with the use of glossy paper, but not with increased spatial
resolution. The automated analysis produced results that were in general agreement with the
observer data for spatial resolution, which concluded that the Lexmark C543DN printer produced
prints with the highest spatial resolution. A thermal dye printer (Kodak CMI1000) produced
prints with the highest contrast uniformity, and the print most favored by observers overall was
produced by the Kodak ESP-9 inkjet printer on Kodak Everyday Glossy Photo paper.
iii
Acknowledgments
Thank you to Drs. Ernest Lam, Michael Pharoah and Bryan Tompson for comprising my thesis
review committee and allowing me the freedom to pursue this study as I saw fit, to Drs. Jim Lai
and Richard Bohay for volunteering for my examining committee, to my family for supporting
me over the many years during which I have been receiving an education, to the observers that
donated their valuable time to this study, and to the contributors and maintainers of free open
source software, without which, a project such as this one would not have been possible without
an inordinately large investment of time and expertise.
Special thanks to Guido Van Rossum and the Python community for the fabulous Python
programming language and extensions, which have made my fledgling attempts at programming
a joy, and have opened my eyes to the possibilities of computing as it applies to my field.
Most of all, I would like to thank my wife, Monica, who has graciously and cheerfully taken care
of everything in our lives while I have been sitting at my computer until the wee hours of the
morning every day over the past 3 years.
iv
Table of Contents
Contents
Acknowledgments .......................................................................................................................... iii
Table of Contents ........................................................................................................................... iv
List of Tables ............................................................................................................................... viii
List of Figures ................................................................................................................................ xi
List of Appendices ....................................................................................................................... xvi
Chapter 1 ......................................................................................................................................... 1
1 Introduction ................................................................................................................................ 1
1.1 Overview ............................................................................................................................. 1
1.2 Prior work ........................................................................................................................... 1
1.2.1 Comments on prior work ........................................................................................ 3
1.3 Existing systems of objective print evaluation ................................................................... 4
1.4 Printers on the market ......................................................................................................... 5
1.5 Print mechanisms ................................................................................................................ 6
1.5.1 Laser printers .......................................................................................................... 6
1.5.2 Inkjet Printers .......................................................................................................... 7
1.5.3 Thermal dye printers ............................................................................................... 8
1.6 Anecdotal example of a poor quality digital image ............................................................ 8
1.7 Factors affecting print quality ........................................................................................... 10
1.7.1 Spatial resolution .................................................................................................. 10
1.7.2 Aliasing and the Moiré pattern ............................................................................. 11
1.7.3 Contrast uniformity ............................................................................................... 12
1.7.4 Paper type .............................................................................................................. 13
v
1.7.5 Printer Drivers ....................................................................................................... 14
Chapter 2 ....................................................................................................................................... 16
2 Aims ......................................................................................................................................... 16
Chapter 3 ....................................................................................................................................... 17
3 Objectives ................................................................................................................................. 17
Chapter 4 ....................................................................................................................................... 18
4 Null Hypotheses ....................................................................................................................... 18
Chapter 5 ....................................................................................................................................... 19
5 Methods .................................................................................................................................... 19
5.1 Overview ........................................................................................................................... 19
5.2 Ethics approval .................................................................................................................. 20
5.3 Test image ......................................................................................................................... 20
5.3.1 Components of the test image ............................................................................... 20
5.3.2 Test image generation ........................................................................................... 21
5.3.3 Test image references ........................................................................................... 22
5.3.4 Sample test images ................................................................................................ 23
5.4 Printer setup ...................................................................................................................... 24
5.4.1 Printer selection .................................................................................................... 24
5.4.2 Media selection ..................................................................................................... 25
5.4.3 Excluded printer and media types ......................................................................... 25
5.4.4 Print setups used for this study ............................................................................. 25
5.5 Production of test prints .................................................................................................... 26
5.5.1 Spectrum of images produced for each print setup ............................................... 27
5.5.2 Empirical confirmation of continuity of images at different resolutions .............. 28
5.6 Observations by human subjects ....................................................................................... 30
5.6.1 Observer group composition ................................................................................. 30
vi
5.6.2 Instructions for observers ...................................................................................... 30
5.6.3 Data collection ...................................................................................................... 31
5.7 Automated analysis ........................................................................................................... 31
5.7.1 Analysis program .................................................................................................. 31
5.7.2 Scanning test prints ............................................................................................... 31
5.7.3 Illustrative comparison of a pre-printed test image to a scanned print ................. 32
5.7.4 Analytic steps ........................................................................................................ 35
5.7.5 Data visualization and determination of spatial resolution ................................... 41
Chapter 6 ....................................................................................................................................... 49
6 Results ...................................................................................................................................... 49
6.1 Observer spatial resolution data ........................................................................................ 49
6.1.1 Effect of observer grouping on observation results .............................................. 49
6.1.2 Observer test-retest reliability ............................................................................... 50
6.1.3 Effect of print setup on spatial resolution observations ........................................ 50
6.2 Observer Favourites .......................................................................................................... 54
6.3 Automated analysis of spatial resolution .......................................................................... 57
6.4 Automated analysis of contrast uniformity ....................................................................... 61
6.5 Higher contrast uniformity is associated with more popular images ................................ 64
6.6 Higher spatial resolution is not associated with more popular images ............................. 65
6.7 The automated and observed spatial resolution results are highly correlated .................. 66
Chapter 7 ....................................................................................................................................... 68
7 Discussion ................................................................................................................................ 68
7.1 Dental education levels produced no differences in spatial resolution observations ........ 68
7.2 General agreement of automated analysis with observations of spatial resolution .......... 68
7.3 Observer preference for glossy prints ............................................................................... 68
7.4 Reconciliation of overall observer preference with measured results .............................. 69
vii
7.5 Cost vs. quality .................................................................................................................. 69
7.6 Study limitations ............................................................................................................... 70
7.7 Future directions ............................................................................................................... 70
7.7.1 Technical considerations ....................................................................................... 70
7.7.2 Clinical considerations .......................................................................................... 71
Chapter 8 ....................................................................................................................................... 72
8 Conclusion................................................................................................................................ 72
Bibliography ................................................................................................................................. 73
viii
List of Tables
Table 1: Summary of the 11 print setups used in this study. The abbreviations are used to
describe these print setups for the remainder of the document. The costs per page were based on
8% ink coverage. ........................................................................................................................... 25
Table 2: Print IDs and source test image sizes produced for each of the 11 print setups. 55 prints
were generated (5 test images x 11 print setups = 55 total images). The source test image ID is
the reference ID of the test image with the parameters shown at the bottom of the table (light
grey regions). The Print ID is the reference ID for each print produced on each printer for a
specific source test image ID. For example, print ID 27 corresponds to source test image 63
(1500 pixels per 5 inches of image width for a source test image at 300 pixels per inch) printed
on the CMI1000 thermal dye printer on glossy paper. The function of these IDs was to track the
data for each print in the database. ................................................................................................ 26
Table 3: Determination of equivalency of line pair group numbers across various theoretical
prints with different resolutions. The purpose of this table is to demonstrate the equivalency of
the line pair densities between line pair groups printed at different resolutions on different
images. For example, since the group of line pairs labeled "1" in image A and the group of line
pairs labeled 16 in group E both have 100 line pairs per printed inch, they are equivalent. The
group numbers correspond to the groups in Section 2 of Figure 10. ............................................ 28
Table 4: Measured maximum line pairs per printed inch for 4 different resolutions on 3 different
print setups on the Lexmark C543DN printer. With the exception of the number of prints
analyzed, the values have units of line pairs per printed inch. ..................................................... 29
Table 5: Composition of observer groups ..................................................................................... 30
Table 6: The data set that corresponds to the sample scanned image being described in this
section. .......................................................................................................................................... 42
Table 7: The values corresponding to the coloured data columns will be plotted in the graphs that
follow. The colour coding corresponds to the line colours in the plots that follow in Figure 22. 45
ix
Table 8: ANOVA summaries of tests of equality of the mean line pairs per printed inch of the 7
observer groups for each print. The differences between observer groups were not significant for
any of the prints. F represents the F-ratio (which measures how different the means are relative to
the variability within each group). Sig represents the calculated significance level that tests the
null hypothesis that no observer group is different than any other observer group. The null
hypothesis could not be disproven for these observer groups. ..................................................... 49
Table 9: Tabulated data of line pairs per printed inch for each observed print. N is the number of
observations. The remainder of the values in the table are line pairs per printed inch. (The
observed line pairs per printed inch were calculated using the formula for converting observed
group numbers in prints to line pairs per printed inch in section 5.5.1) ....................................... 51
Table 10: One-way ANOVA for the pooled data that tests the null hypothesis that observed print
spatial resolutions are the same between all prints. The null hypothesis was proven false. F is the
F-ratio; Sig is the significance level for the testing of the null hypothesis; df is degrees of
freedom. (The observed line pairs per printed inch were calculated using the formula for
converting observed group numbers in prints to line pairs per printed inch in section 5.5.1) ...... 53
Table 11: Levene's test evaluates the null hypothesis that variances between sample sets are
equal. The null hypothesis was false. df1 and df2 represent the two degrees of freedom and Sig
represents the significance level of the null hypothesis test. ........................................................ 53
Table 12: The matrix of p-values less than 0.05 for significant differences between prints setups,
as determined by observers. The Lexmark C543DN printer produced prints of significantly
different spatial resolution than the other printers in the study, except for the Epson C68 printer
with glossy paper. ......................................................................................................................... 54
Table 13: Overall favourite prints selected by observers, in descending order of preference. The
top 3 prints represented over 80% of those selected by observers................................................ 55
Table 14: The computed analysis of spatial resolution of all printouts produced for this study.
With the exception of the number of measurements, all values have a unit of line pairs per
printed inch. (The line pairs per printed inch were calculated using the formula for converting
group numbers from the automated analysis of prints to line pairs per printed inch in section
5.5.1) ............................................................................................................................................. 58
x
Table 15: One-way ANOVA that tests the null hypothesis that all prints have an equivalent
spatial resolution, based on the automated analysis. The null hypothesis was false. F represents
the F-ratio; Sig represents the significance level of the null hypothesis test; df represents the
degrees of freedom. ....................................................................................................................... 60
Table 16: Levene's test evaluates the null hypothesis that variances between sample sets are
equal. The null hypothesis was false. df1 and df2 represent the degrees of freedom and Sig
represents the significance level of the null hypothesis test. ........................................................ 60
Table 17: The matrix of p-values less than 0.05 for significant differences between prints setups,
as determined by the automated analysis. The Lexmark C543DN printer produced prints of
significantly different spatial resolution than the other printers in the study, except for the Epson
C68 printer with glossy paper and the HP 4050 with plain paper. ............................................... 61
Table 18: The computed data for contrast uniformity for all prints in this study. N represents the
number of observations. The remainder of the values are shrink factors (representing the factor
that image had to be shrunk by from its original size to produce a uniformity within 4 pixel
intensity units (out of 256 total) , averaged over all the grayscale steps. ..................................... 62
xi
List of Figures
Figure 1: Laser printing steps. 1) Print data is fed to the printer from a computer. 2) The print
processor converts the data into a suitable format understandable by the printer. A high-voltage
"corona wire"(3) charges the photoreceptor drum(4) with positive charge. 5) The laser beam is
reflected off a moving mirror that scans over the drum, producing areas of negative charge on the
drum. (The negative areas will be black on the future printout and the positive areas will be
white.) Gradually an image is produced on the photoreceptor drum. 6) An ink roller transfers
positively charged ink to the negatively charged areas on the photoreceptor drum. 7) A sheet of
paper is fed towards the drum and receives an intense positive charge from another corona wire.
8) When the paper reaches the drum, the negatively charged ink is transferred to the positively
charged paper surface. 9) The inked paper passes through two hot rollers (the fuser unit), which
fuses the ink to the page. 10) the warm page emerges from the printer. ........................................ 6
Figure 2: An example of a poor-quality print of a digital radiograph received by the Faculty of
Dentistry at the University of Toronto in August 2009. ................................................................. 9
Figure 3: A magnified section of the low quality print from Figure 2. ........................................ 10
Figure 4: Evaluation of spatial resolution. In this print, 20 alternating black and white line pairs
are shown in each of the four groups, starting with the widest lines on the left (group 4) and
ending with the narrowest on the right (group 1). The spatial resolution of this print is limited to
objects that are spaced as far apart as those in group 2, because that is the last group in which 20
line pairs are distinguishable. In group 1, several of the line pairs have converged, resulting in an
apparent decrease in the number of visible line pairs. This is because the spatial resolution of the
print is lower than the actual resolution of the source image (in which 20 line pairs were present
in all groups). ................................................................................................................................ 11
Figure 5: Moiré patterns demonstrated in a screen shot of a line pair test image (red dashed
boxes). The frequency of pixels on the screen did not match that of the line pairs in the original
image, resulting in this aliasing artifact. The source image had uniformly spaced line pairs in it.
....................................................................................................................................................... 12
xii
Figure 6: Excellent and poor contrast uniformity compared. Prints with excellent contrast
uniformity display a uniform gross appearance (a), and a uniform microscopic appearance (b).
Prints with poor contrast uniformity display either (or both) an inhomogeneous gross appearance
(c) and an inhomogeneous magnified appearance (d). ................................................................. 13
Figure 7: A source image composed of steps(left) is rendered and printed as a dithered pattern
(right). All the ink in the print is black, but from a distance, the shaded steps are apparent. ....... 14
Figure 8: Different dithering patterns are produced in these magnified sections of two prints from
a Xerox 8860 colour laser printer. All the parameters were kept constant except for the print
drivers. The differences in dither patterns are especially evident at the lighter side of the scale. 14
Figure 9: Sequence of events for methods and analysis of data in this study. .............................. 19
Figure 10: This test image is 900 pixels wide x 594 pixels high. There are 20 line pairs per group
and 6 groups (numbered 1-6 in black or white, from right to left), with the group number
corresponding to the width of half a line pair in pixels. The test image is generated by a custom
algorithm and output as PDF files on letter sized paper (8.5 x 11 inches) with a consistent image
with of 5 inches, regardless of the original resolution of the image. The properties of the test
image are shown in white on a green background: 1.) visual reference to the test image ID and
print ID (which are necessary for observers to record their observations). 2.) Instructions for
observers, and labels of each line pair group. These are the values that observers will select when
making observations. 3.) The column of reference grey values that are halfway between the
maximum and minimum values in the corresponding row. 4.) Stripe 1: Alternating broad black
and white stripes that each correspond to 1 group of 20 line pairs. 5.) Stripe 2: Alternating black
and white line pairs. There are 20 line pairs per group. This is the row that observers evaluate. 6.)
Stripe 3: Alternating 50% grey and white line pairs. 7.) Stripe 4: Alternating 50% grey and black
line pairs. 8.) The data matrix barcode for automated analysis reference, which allows the
program to record the analytic data in the appropriate location in the database. .......................... 20
Figure 11: A test image that is 3000 pixels wide. The group numbers represent the same number
of pixels in half a line pair in both images. For example, group 10 has twenty pairs of black and
white lines that are each 10 pixels wide. ....................................................................................... 23
Figure 12: A test image that is 24000 pixels wide. ....................................................................... 24
xiii
Figure 13: A resized version of a digital source image that is produced for printing an analysis. 33
Figure 14: A scan of a print produced with the digital image in the previous figure. Note the
mottling, loss of line pair resolution and reduced contrast range. ................................................ 34
Figure 15: A stylistic example of the pre-analysis processing that was sequentially applied to
scanned images. The scanned image was de-rotated so that the top of the original image was
horizontal. The image was deskewed so that the right edge was vertical. The excess scanned
space around each image was cropped out. This resultant image was saved as an 8 bit lossless
TIF file, and was subsequently analyzed by the automated system. ............................................ 34
Figure 16: The image is divided into 7 rows of equal height. Row 0, image text; Row 1, group
orientation stripes; row 2: white/black line pairs; row 3: white/50% grey line pairs; row 4: 50%
grey/black line pairs; row 5: black to white grayscale (30 steps); row 6: white to black grayscale
(30 steps). ...................................................................................................................................... 36
Figure 17: Row #1 is used to determine the width of each column, each containing a group of 20
line pairs. The rows are numbered in red, and the columns corresponding to each group of line
pairs is numbered from right to left in blue. Each blue number indicates the width of half a line
pair in pixels in the original image. For example, in group 5, each black line is 5 pixels wide in
the source image. .......................................................................................................................... 37
Figure 18: Comparison of a perfect data plot (top) and a scanned data plot (middle). The plotted
data for the scan is expressed in numeric form on the bottom of the figure. The meaning of each
variable in the table are: PixelPair: The group number. Also refers to the number of pixels in half
of a pixel pair. So a pixelPair value of 5 means that the black line is 5 pixels wide and the white
line is 5 pixels wide in the original test image. (The size of the pair on the printed image varies
because prints are made at different resolutions). Length: size of the column in scanned pixels.
Minimum: the smallest pixel intensity value measured in the entire column. Maximum: the largest
pixel intensity value measured in the entire column. Average: the average of all pixel intensities
in the column. range: the difference in pixel intensities between the maximum value in the
column and the minimum value in the column. maxPeaksAverage: the average value of the 20
highest peaks in this column. (The number 20 comes from the number of line pairs per column).
The unit of measure is pixel intensity value (ranging from 0-255 for an 8bit grayscale image).
xiv
minPeaksAverage: the average value of the 20 lowest valleys in this column. (The number 20
comes from the number of line pairs per column). The unit of measure is pixel intensity value.
peaksAverageRange: the difference in pixel intensity value between maxpeaksAverage and
minPeaksAverage. ........................................................................................................................ 38
Figure 19: The cropped region of the image from the previous figure that will be further
examined. ...................................................................................................................................... 40
Figure 20: Each group is subdivided into 20 evenly spaced subdivisions. The maxima and
minima in each group are evaluated and averaged to produce minPeaksAverage and
maxPeaksAverage values for each group. The difference between these two values is the
peaksAverageRange for each group. ............................................................................................ 41
Figure 21: Rows 1 and 2 of the scan of an plain paper inkjet print that is being examined with the
automated analysis are shown at top. This print was not included in the actual data analysis, and
is only presented as an example to demonstrate the automated analysis method. A zoomed
section of groups 1-6 from the scan are shown at bottom. There is a marked visual difference
between groups 4 and 5 in this scan. ............................................................................................. 43
Figure 22: The various variables from table are plotted on adjacent charts. Note the similarity
between them and the general dip that occurs between groups 4and 5. ....................................... 47
Figure 23: Ordered plot of observer results for spatial resolution evaluations. Higher spatial
resolutions are better. The y-axis represents the various print setups and the x-axis represents the
line pairs per printed inch for each print setup. The pink boxes represent groups of statistically
similar print setups. ....................................................................................................................... 52
Figure 24: Overall favorite prints selected by observers. ............................................................. 56
Figure 25: Breakdown of favorite prints by observer dental education level. .............................. 57
Figure 26: Ordered plot of automated spatial resolution evaluations. The print setups are on the
y-axis and the line pairs per printed inch for each print setup are on the x-axis. Higher spatial
resolutions are better. The pink boxes represent groups of statistically similar print setups. ....... 59
xv
Figure 27: Ordered plot of automated contrast uniformity measurements. Lower values indicate
higher uniformity, because the image needed less anti-aliased shrinking to achieve contrast
homogeneity. Statistically similar groups of print setups are grouped together in pink boxes. ... 63
Figure 28: Observers’ favorite images superimposed over the contrast uniformity plot. Popular
images were associated with high contrast uniformity. ................................................................ 64
Figure 29: Print popularity ratings superimposed over ordered plotted results of the spatial
resolution observations. Popular images were not associated with high spatial resolutions. ....... 65
Figure 30: Print popularity ratings superimposed over the ordered plotted results of the
automated analysis. Note the similarity of the ordering of the results of the automated analysis to
that of the observer-generated results. .......................................................................................... 66
Figure 31: Scatter plot and best fit line for correlated observed and automated mean spatial
resolutions. The R value corresponds to Pearson's correlation coefficient. The formula for
calculating the expected automated mean from the observed mean is shown in the inset. the p-
value represents the 2-tailed significance. .................................................................................... 67
xvi
List of Appendices
The upload of digital files is permitted with theses, so the appendices are distributed with this
thesis on two DVDs for review by committee members and shall uploaded directly to the U of T
Electronic Thesis archive when submitted.
DVD 1:
Appendix A: Digital_thesis : This is the digital copy of this document.
Appendix B: Ethics_review: These are the documents related to ethics approval for this
project.
Appendix C: Image_generation_code: This is the Python source code for a modified
version of the program that was used to produce the test images for this project. The
program version included in this appendix does not produce Datamatrix barcodes, and is
fixed to a default number of 30 grayscale steps. The functionality is otherwise unchanged
from the original image production program.
Appendix D: Image_analysis_code: This is the Python source code for a modified version
of the program that was used to analyze the scans of images produced by various print
setups for this project. The program version included in this appendix does not read
Datamatrix barcodes and can only analyze images with 30 grayscale steps. The
functionality is otherwise unchanged from the original image analysis program.
Appendix E: Runnable_programs: These are executable programs that can be run on any
Windows computer to produce test images or analyze scanned prints.
Appendix F: Presentations: These are the PowerPoint and poster presentations associated
with this project.
Appendix G: print_and_scan_setups: These are the web page captures of the parameters
for each print setup and the single scanner setup.
Appendix H: Stastistics_files: These are the SPSS and Microsoft Excel files that contain
the anonymized data for this project.
Appendix I: Test_images_for_printing: These are the PDF files of the 5 test images used
in this study.
xvii
Appendix J1: Scans_of_prints_part_1: These are the first 15 of 55 scanned images
produced for this project. To reduce the space requirements, the 5 prints from each print
setup were zipped into a zip file. Note that the unzipped versions of these files are in a
suitable format for analysis by the automated program found in Appendix E
DVD 2:
Appendix J2: Scans_of_prints_part_2: These are the last 40 of 55 scanned images
produced for this project. To reduce the space requirements, the 5 prints from each print
setup were zipped into a zip file. Note that the unzipped versions of these files are in a
suitable format for analysis by the automated program found in Appendix E
1
1
Chapter 1
1 Introduction
1.1 Overview
In dental radiology, there are two basic image storage and viewing systems available: analog
radiographs and digital radiographs. X-ray films have existed since the advent of radiography at
the turn of the 20th
century. The first commercial digital dental radiography systems became
available in the late 1980s (Mouyen et al., 1989). As the spatial resolution of systems improved
and the prices decreased, digital radiography assumed a larger proportion of the dental
radiographic market. In two surveys of dentists in 2007, market levels of 19.7% in Indiana
(Brian and Williamson, 2007) and 36% in Hawai'i (Brady, 2007) were found for digital intraoral
radiography systems.
Digital images require a computer and monitor for image viewing, and this means that
periodically, printed copies of digital images need to be produced.
The purpose of this thesis is to examine various consumer-grade printers with respect to spatial
and contrast resolutions, two factors that affect the print quality of the production of digital
radiographs.
1.2 Prior work
Bley et al. (2003) used receiver operating characteristic curves to evaluate the diagnostic quality
of paper prints vs. film. Digital radiographs were printed on paper with an inkjet printer and on
film with a thermal laser printer. Five observers (1 fellow + 4 residents) evaluated 60 high-
contrast and 60 low-contrast objects. They scored the presence of a hole in each object on a 1
(not visible) to 5 (definitely present) scale. Receiver operating characteristics for film and prints
were 0.926 (SE 0.015) and 0.863 (SE 0.024) respectively. The conclusion was that film is
superior to paper prints for evaluating radiographs
Gijbels et al. (2004) evaluated the subjective image quality of digital panoramic images on
monitors vs. thermal prints vs. inkjet prints on paper and transparencies. Fifteen digital
2
2
panoramic radiographs were viewed on a monitor, printed on direct thermal print film, and from
an inkjet printer onto transparencies, glossy paper, satin paper, and matte paper. Five observers
(2 oral radiologists, 2 dental students, 1 periodontist) evaluated the presence or absence of
various anatomic landmarks on a 1 (impossible to see) to 5 (clearly visible) scale. One observer
performed the test again, 4 weeks later, to test intra-observer variability. Subjectively, the order
of preference was as follows: direct thermal prints > monitor > inkjet transparencies > inkjet
glossy paper > inkjet satin paper > inkjet matte paper.
Haak et al. (2003) evaluated the influence of displayed image size on radiographic detection of
approximal caries. Charge coupled device (CCD) images of 160 unrestored posterior teeth were
assessed on 2 thin film transistor liquid crystal (TFT) and cathode ray tube (CRT) monitors at 3
sizes (1:1-full size, 1:2, 1:7-real life size). The results were compared to histologic data for all
teeth. Five observers (dentists at university with at least 2 years experience) evaluated caries
depth on a 0 (no caries) to 5 (inner dentin) scale. Larger image sizes (1:1 and 1:2) produced
significantly improved caries detection than small images (1:7) on monitors. Viewing images on
CRT and TFT monitors did not produce statistically significant differences.
Guerrant et al. (2001) compared thermal printed digital panoramic images to monitor displays.
Sixty panoramic images were displayed on a computer monitor and thermal printed. Each pair of
images was viewed simultaneously. Four calibrated observers qualitatively analyzed 13 anatomic
features on each pair of images. Each feature on each image was scored on a nominal scale in
terms of subjective quality and diagnostic utility. Within the parameters of the study, thermal
prints of digital panoramic images and images viewed on a monitor were both acceptable for
diagnostic utility and were not significantly different from each other.
Otis and Sherman (2005) compared the diagnostic accuracy of film vs. prints for approximal
caries detection. Fifteen posterior bitewing films were digitized with a transparency scanner at
300 pixels per inch. Prints on glossy paper were made with an inkjet printer at 1:1 and 4:1
(enlarged) sizes. Fourteen dentists with between 6 and 26 years of experience) evaluated caries
on a 0 (none) to 4 (caries to pulp) scale. Film and paper prints produced similar caries detection
levels. For paper prints, the enlarged prints produced more accurate results than the 1:1 print for
enamel caries. There were no differences for dentin caries
3
3
Benediktsdottir and Wenzel (2004) evaluated digital panoramic radiographs on monitor vs.
glossy paper vs. transparent film for the assessment of third molar position. Two digital
panoramic systems produced images for 164 third molars. Findings at surgery were the gold
standard. Four pre-calibrated observers evaluated the impaction state, tooth position, number of
roots and root morphology. Each evaluation consisted of categorical or nominal variables. Inter-
observer variation was greater than the variation between viewing modalities. Therefore, within
the parameters of this study, inkjet glossy prints, inkjet transparencies, and images on a monitor
could not be shown to be significantly different.
Schulze et al. (2008) compared the quality of calibrated customary printers on glossy paper.
Three inkjet printers were compared to 2 thermal dye sublimation printers and a consensus
observation between 2 people on standard viewing monitor was used as a gold standard. Sixteen
raters provided subjective ratings of 3 dental radiographs via 35 single questions with answers on
nominal scales. The test pattern was only used for calibration. Receiver operator characteristic
(ROC) curves showed no significant difference overall between all the printers. Subjectively, one
inkjet printer (HP Tetenal inkjet printer) was significantly different than the other printers. All
printers produced good diagnostic performance compared to a standard viewing monitor.
1.2.1 Comments on prior work
The gold standard for the printing of digital radiographic images is laser film (Bley et al., 2003)
but the high cost ($25,000-$50,000 plus materials and maintenance) does not make it feasible for
typical dental office. Thermal dye sublimation printers (in the $650 range) have received mixed
reviews; they have been subjectively rated as being both superior to and inferior to images on a
monitor (Guerrant et al., 2001), and superior or equivalent to inkjet prints on glossy paper
(Gijbels, et al. 2004). The cost-per-print for a typical dye sublimation printer such as the Kodak
8500 thermal dye printer (which is marketed under various names including the Kodak CMI1000
colour medical imager and the Codonics EP-1000) is approximately $2.25. These cost estimates
all assume that all the prints work perfectly and there is no wastage.
A general approach to the evaluation of printed and displayed digital images has been the use of
subjective analysis coupled with ROC curves and kappa tests for inter-operator variability
4
4
(Lyttkens et al., 1994; Guerrant et al., 2001; Bley et al., 2003; Benediktsdottir and Wenzel,
2004; Otis and Sherman, 2005; Schulze et al., 2008). However, inter-operator agreement tends to
be low, and generally overshadows differences between experimental variables (such as
comparisons between printers, between different paper stocks, or a printed image vs. an image
on a computer monitor).
A fundamental problem with having operators judge whether something is "better" is that the
definition of "better" is poorly defined. What one operator may find "better" (a glossy image, for
example), may be construed as "worse" by another operator. For glossy images, for example,
there may be more specular reflections. In designing a method for evaluation of test prints, one
objective must be to eliminate the concept of "better" from a subjective perspective by relying on
objective findings by observers, as opposed to evaluation of actual dental radiographs, which
adds an extra layer of complexity to the print analysis.
However, the diagnostic value of a print (which is not assessed in this study) should not be
confused with its esthetic performance (which is evaluated in this study). The diagnostic value of
a print is not something that an observer experiences directly, but is the result of comparison of
the observations of prints versus the ground truth. Therefore, the parameters tested in this study
should not be assumed to be surrogate measures for diagnostic ability; their usefulness shall only
be determined in subsequent studies that focus on the diagnostic performance of prints produced
by the same print setups in this study.
1.3 Existing systems of objective print evaluation
In 2001, the first world-wide industry standard for digital print quality was introduced as "ISO-
13660:2001, Information, technology - office equipment - measurement of image quality
attributes for hardcopy output - binary monochrome text and graphic images" (McDowell, 2004).
It was intended to provide a practical, objective means of communication about 14 basic image
quality parameters and provided measurement methods that could be automated. However, the
visual significance of a measurement difference is not addressed in the standard. The standard is
device-independent, meaning that any image produced by any system can be used. The image
quality parameters are intended to evaluate monochrome text and graphics, and measurements
can be made on any character, line or test area, meaning that no specific test images are required.
5
5
ISO-13660 and the newer ISO-19751 (which incorporates evaluations of gray-level and full-
colour imaging technologies) are effective professional-strength tools developed internationally
by engineers from digital printing equipment manufacturers, and its implementation is both
elegant and practical. The primary use of these standards, however is to allow digital press
manufactures a standardized method for benchmarking their digital presses.
The main drawbacks of the ISO standards in the context of the present study pertain to the target
audience and to the intended function of the analysis. ISO-13660 is not intended to apply to
pictorial art, but is instead optimized for black colourant forming the image on a white substrate
(Briggs et al, 1999). The measurement system for the ISO standard must be accurately calibrated
to provide a base level of uniformity for each physical setup. In contrast, the system proposed in
this study is designed to be used by dentists in their offices with uncalibrated printers and the
relatively low-cost Epson V700 series scanners. The measurements in the ISO system rely on
specific software coupled to the calibrated hardware, and the actual measurements themselves
are not fully automated. A skilled user of the software must analyze the prints. In contrast, the
automated system proposed in this study automates the evaluation process to a large extent.
A useful adjunct to the current study, however, would be to perform various ISO-19751
measurements on the prints produced and evaluated in this study to ascertain whether clear
parallels between the two systems exist.
1.4 Printers on the market
The most popular printers on the market, (according to google trends1) are laser printers and
inkjet printers. A technique for determining the optimum print parameters for laser and inkjet
printers on plain paper is of importance, because it is frequently possible to coerce a higher
quality image out of an existing printer. An example of a beneficial change that can be made for
free is the updating of a print driver from PCL5 to the more recent PCL6.
1
http://www.google.com/trends?q=inkjet%2C+laser+printer%2C+thermal+printer%2C+laser+film+printer&ctab=0&
geo=us&geor=all&date=all&sort=1
6
6
1.5 Print mechanisms
A brief discussion about printer mechanisms for laser, inkjet and thermal dye printers follows.
1.5.1 Laser printers
Laser printers use the same basic technology as photocopiers. However, as opposed to the
copying of an original document onto a light sensitive drum (as in a photocopier), a laser is used
to convert print data from a computer into an image on a light sensitive drum, which is then
processed as it would be in a photocopier. The steps in the production of a laser print are shown
in figure 1, below.
Figure 1: Laser printing steps. 1) Print data is fed to the printer from a computer. 2) The print
processor converts the data into a suitable format understandable by the printer. A high-voltage
"corona wire"(3) charges the photoreceptor drum(4) with positive charge. 5) The laser beam is
reflected off a moving mirror that scans over the drum, producing areas of negative charge on the
drum. (The negative areas will be black on the future printout and the positive areas will be
white.) Gradually an image is produced on the photoreceptor drum. 6) An ink roller transfers
positively charged ink to the negatively charged areas on the photoreceptor drum. 7) A sheet of
7
7
paper is fed towards the drum and receives an intense positive charge from another corona wire.
8) When the paper reaches the drum, the negatively charged ink is transferred to the positively
charged paper surface. 9) The inked paper passes through two hot rollers (the fuser unit), which
fuses the ink to the page. 10) the warm page emerges from the printer. 2
The use of dry toner in a laser printer means that no diffusion or spreading of ink occurs,
regardless of the type of paper used. This means that ordinary paper stock is capable of
producing sharp prints. However, the heat required to fuse the ink to the paper limits the paper
choice to substrates that are capable of handling the elevated temperature.
The process for colour laser prints is the same as that for black and white prints, except that
instead of just one pass through the image creation mechanism, four passes are required (one for
each of the cyan, magenta, yellow and black inks) before the image is fused onto the page. This
means that colour laser prints take significantly longer to produce and are more expensive than
black and white laser prints.
1.5.2 Inkjet Printers
Inkjet printers use a matrix of jets or nozzles to fire small bursts of ink onto paper as the print
head scans across a page. The technology for firing the ink varies from company to company.
For example, Canon heats the ink, causing it to boil and explode toward the paper in small
bubbles, so Canon printers are marketed under the brand name Bubble Jet. Epson printers use the
piezoelectric effect (the ability of a crystal to change shape in response to an electric current) to
essentially "hammer" small drops of ink towards the paper.
The position of each dot on a page is determined by the print processor, and the placement of the
dot is achieved by a combination of feeding the paper through the printer, and the simultaneous
continuous back-and-forth scanning of the print head during printing. Therefore, the accuracy of
the print is dependent on the accuracy with which the paper can be fed through the printer.
2 The image in Figure 1 is from http://www.explainthatstuff.com/laserprinters.html
8
8
Because the ink used in inkjet printers is wet, it will diffuse upon contact with the paper. The
amount of diffusion is controlled by the type of paper used as a substrate. Specially coated inkjet
papers have much less ink diffusion than ordinary paper stock, resulting in a crisper image.
1.5.3 Thermal dye printers
Thermal dye printers use a technique called dye sublimation , which involves a ribbon being
heated as it passes under a thermal print head. The elevated temperature converts the ink on the
ribbon from a solid to a gaseous state, which diffuses onto the adjacent paper, where it then cools
and is retransformed into the solid state. The amount of heat applied to the print head determines
the amount of ink that is transferred from the ribbon to the paper.
The ribbon used in a colour dye sublimation printing consists of three separate colour panels
(yellow, magenta, and cyan), which are each applied to the paper in a multi-pass operation.
The advantage of the dye sublimation process is that continuous colour tones can be produced.
However, a disadvantage is that the diffusion of the ink in the gaseous phase produces slightly
smudgy images that are less crisp. The ribbons cannot be reused because each print job removes
ink from a specific part of the ribbon. The net result is a high amount of wastage, and therefore, a
high overall cost per print. The paper used for dye sublimation printing is specially coated to
accept the sublimated ink. The ink ribbons and paper are sold together, because each is
optimized for the other.
1.6 Anecdotal example of a poor quality digital image
The printout in Figure 2 (below) was received by the Faculty of Dentistry at the University of
Toronto in August 2009. It is a printout of two digital intraoral radiographs.
9
9
Figure 2: An example of a poor-quality print of a digital radiograph received by the Faculty of
Dentistry at the University of Toronto in August 2009.
The resolution of the author’s scan of this image was high enough to capture the smallest dots
produced in the print. The actual image appears to have been generated with a halftone pattern
on a monochrome laser printer, and represents the typical output from an older machine or a
newer machine being driven by old print drivers. The image has been resized for the web and
reprinting in this thesis, so the details may not be immediately apparent. However, the streaky
quality and poor contrast uniformity is evident. For reference: the grey bar beneath each of the
radiographs in figure 2 is supposed to be a uniform density.
A magnified section of figure 2 is shown in figure 3, below.
10
10
Figure 3: A magnified section of the low quality print from Figure 2.
A question that could be asked of the image in Figure 3 is whether the result could have been
improved, and if so, whether that potential improvement in quality could be objectively
ascertained. This question is essentially the motivation of this thesis, because an answer to that
question would form the foundation of an informed recommendation to anyone that is
considering purchasing a printer to print dental radiographs.
1.7 Factors affecting print quality
1.7.1 Spatial resolution
Spatial resolution is defined as the minimum distance between distinguishable objects in an
image. Alternating pairs of black and white lines (also known as line pairs) of decreasing width
are evaluated to determine the narrowest line pairs that remain distinguishable. The width of the
11
11
narrowest distinguishable line pairs represent a measure of the print's spatial resolution. An
example of spatial resolution determination on a sample print is shown in figure 4.
Figure 4: Evaluation of spatial resolution. In this print, 20 alternating black and white line
pairs are shown in each of the four groups, starting with the widest lines on the left (group 4) and
ending with the narrowest on the right (group 1). The spatial resolution of this print is limited to
objects that are spaced as far apart as those in group 2, because that is the last group in which 20
line pairs are distinguishable. In group 1, several of the line pairs have converged, resulting in an
apparent decrease in the number of visible line pairs. This is because the spatial resolution of the
print is lower than the actual resolution of the source image (in which 20 line pairs were present
in all groups).
1.7.2 Aliasing and the Moiré pattern
Aliasing is a signal processing effect that causes different signals to become indistinguishable
when sampled. At a very basic level, consider the following times noted on a clock: 1pm, 2pm,
3pm, 4pm. If someone was asked how often the clock was read, a reasonable answer would be
"one hour intervals." However, an equally reasonable answer is "25 hour intervals" or even "49
hour intervals". The sampled data (the times that were noted on the clock) could have been
produced by multiple sampling rates (1, 25 and 49 hour intervals). Aliasing artifacts become
12
12
apparent when the sampling rate does not exactly match the frequency of the signal. For
example, if your goal was to report all the times that a clock struck, but you were only able to
observe the clock in 75 minute intervals, you would mistakenly think that clocks only struck 5
times in a 24 hour period. In images, this sort of sampling error shows up as a Moiré pattern (see
figure 5, below).
Figure 5: Moiré patterns demonstrated in a screen shot of a line pair test image (red dashed
boxes). The frequency of pixels on the screen did not match that of the line pairs in the original
image, resulting in this aliasing artifact. The source image had uniformly spaced line pairs in it.
1.7.3 Contrast uniformity
Contrast uniformity is a measure of the ability of a printer to produce smooth solid colours on a
print. There are two factors that play a role in contrast uniformity. At a gross level, a uniform
source image composed of a single shade should be reproduced without inhomogeneities. At a
magnified level, the image should remain homogenous instead of degenerating into clustered
regions of high and low contrast. Figure 6 presents examples of good and poor contrast
uniformity.
13
13
Figure 6: Excellent and poor contrast uniformity compared. Prints with excellent contrast
uniformity display a uniform gross appearance (a), and a uniform microscopic appearance (b).
Prints with poor contrast uniformity display either (or both) an inhomogeneous gross appearance
(c) and an inhomogeneous magnified appearance (d).
1.7.4 Paper type
A plethora of papers are available for printing. Each has a specific set of properties, including
brightness, smoothness, weight, coating type, glossiness, printer compatibility and printing
temperature range. The choice of paper affects the print output, but the magnitude of the effect is
related to the printer technology used. For example, the sharpness of an inkjet print is
significantly improved by the use of coated paper specifically designed to minimize ink
diffusion. However, the sharpness of a laser print is relatively unaffected by the type of paper
used.
A particularly obvious property of paper from an esthetic perspective is gloss, which is actually
the ability of some papers to reflect light in the specular direction (in much the same way that
mirrors reflect light). This is actually a function of the surface topography of the paper. Paper
with a smooth surface appears glossy, and paper with rough surfaces (in the micrometer range)
reflect no specular light, and therefore appears Matte.
14
14
1.7.5 Printer Drivers
Printer drivers are the programs that determine the placement of ink on the page by the printer so
that the source image can be as accurately presented as possible. A printer such as a
monochrome laser printer (which only produces black dots of various sizes) relies on a dithering
algorithm in the print driver in order to produce grayscales. Dithering is the use of patterns of
dots to mimic various solid shades (see figure 7).
Figure 7: A source image composed of steps(left) is rendered and printed as a dithered pattern
(right). All the ink in the print is black, but from a distance, the shaded steps are apparent.
Newer print drivers tend to have more sophisticated dithering algorithms than old drivers. An
example of differing dither patterns produced on the same printer with different print drivers is
shown in figure 8.
Figure 8: Different dithering patterns are produced in these magnified sections of two prints
from a Xerox 8860 colour laser printer. All the parameters were kept constant except for the
print drivers. The differences in dither patterns are especially evident at the lighter side of the
scale.
15
15
16
16
Chapter 2
2 Aims
The aims of this study are to:
a. determine which printer setup produces images with the highest spatial resolution and
contrast uniformity.
b. to ascertain which prints observers subjectively prefer from the entire set of prints
produced by the various print setups.
c. to validate the automated method of print resolution determination based on the
observations made by observers. The gold standard was a test image (produced by a
custom image generation program) that was visible on a monitor.
d. to determine whether the dental education levels of the observers have any effects on
their observations.
17
17
Chapter 3
3 Objectives
The objectives of this study are to:
1. produce a simple test image that can be printed at any dimension on any printer in a
standardized way. Computers have many variables associated with them (such as
different operating systems, programs capable of printing, etc.). In order to minimize the
variation produced by these factors, a standardized method for printing across all
platforms must be implemented.
2. produce a program that automates the analyses of scans of printouts of test images.
Automation eliminates subjectivity from the analysis, and thus potentially obviates the
need for observers to evaluate printouts.
3. compare the findings from an automated system with those produced by a panel of
observers. Validation of the automated system by the observations made by observers
will mean that future analyses can potentially be carried out by the automated system
alone.
4. provide a simple metric by which prints can be compared in terms of spatial and contrast
resolution, without the need for complex standardization procedures. The reason for the
omission of standardization procedures is to mimic the clinical situation in which the
dentist typically unboxes a new printer and uses it with factory default settings and no
calibration. Therefore, all the printers in this test were evaluated at their factory default
settings.
5. determine whether groups of observers with different levels of education produce
different results on evaluations that do not require a specific diagnostic skill
18
18
Chapter 4
4 Null Hypotheses
1. Hoa: There are no differences in spatial resolution for images printed on different
printer/paper combinations.
2. Hob: There are no differences in contrast uniformity for images printed on different
printer/paper combinations.
3. Hoc: Observers do not prefer specific prints from the test set.
4. Hod: There are no differences between the automated analysis and observer-based
analyses of test images.
5. Hoe: There are no differences between observer groups with different levels of dental
education
19
19
Chapter 5
5 Methods
5.1 Overview
An overview of the methods used in this project follow the sequence shown in Figure 9.
Figure 9: Sequence of events for methods and analysis of data in this study.
20
20
5.2 Ethics approval
This study was approved by the University of Toronto Health Sciences Ethics Review Board.
See the appendices for the approval letter.
5.3 Test image
5.3.1 Components of the test image
A test image was created that could be unambiguously analyzed by a computer without operator
intervention. As well, it was necessary for the image to be simply interpreted or analyzed by an
observer. Consequently, the complexity of the image needed to be minimized, and the structure
of elements in the image needed to be very rigidly defined.
The constituents of the typical test image are shown in Figure 10 .
Figure 10: This test image is 900 pixels wide x 594 pixels high. There are 20 line pairs per
group and 6 groups (numbered 1-6 in black or white, from right to left), with the group number
21
21
corresponding to the width of half a line pair in pixels. The test image is generated by a custom
algorithm and output as PDF files on letter sized paper (8.5 x 11 inches) with a consistent image
with of 5 inches, regardless of the original resolution of the image. The properties of the test
image are shown in white on a green background: 1.) visual reference to the test image ID and
print ID (which are necessary for observers to record their observations). 2.) Instructions for
observers, and labels of each line pair group. These are the values that observers will select when
making observations. 3.) The column of reference grey values that are halfway between the
maximum and minimum values in the corresponding row. 4.) Stripe 1: Alternating broad black
and white stripes that each correspond to 1 group of 20 line pairs. 5.) Stripe 2: Alternating black
and white line pairs. There are 20 line pairs per group. This is the row that observers evaluate. 6.)
Stripe 3: Alternating 50% grey and white line pairs. 7.) Stripe 4: Alternating 50% grey and black
line pairs. 8.) The data matrix barcode for automated analysis reference, which allows the
program to record the analytic data in the appropriate location in the database.
5.3.2 Test image generation
The test image was generated so that it could be reproduced in an automated fashion in order to
minimize inconsistency: A custom Python3 program (see appendix C for program source code
and Appendix E for a runnable Microsoft Windows program) was written by the author to
produce a test image, incorporating the following specific input parameters:
1. Print identification
2. Image identification
3. Image width in pixels
4. Image width in inches
5. Number of line pairs per group
6. number of grayscale steps for contrast evaluation
3 The program was written in the cross-platform, freely-available open sourced python programming language.
www.python.org
22
22
The output format was a PDF document for each test image. As PDF documents are viewed and
printed consistently on different platforms, they lend themselves to an application such as this
one.
5.3.3 Test image references
Pertinent information regarding the parameters and settings necessary to produce each print were
recorded and stored in a relational database. Each individual printout has a unique print ID, as
well as a reference to the original image ID that is shown on each print. To minimize data input
errors for scanned prints, the print and data IDs were incorporated into a 2D barcode at the
bottom right of the image. This barcode could be scanned and read by the automated analysis
system (described later). The barcode type is DataMatrix4. The barcode is included in the scan of
the print, so the data for the print reference can be automatically extracted, allowing the correct
filing of the image results in the database without operator intervention. Only the references are
stored on the printout. The actual parameters are stored in the database, to minimize the size of
the barcode. A list of the parameters that are stored for each print includes:
1. Printer type
2. Printer settings
3. Paper type
4. Actual width as printed
5. Actual height as printed
6. What program was used to send the print command
7. Reference to the original test image
8. Reference to scans made of this print
9. Reference to analysis data (both automated and observer-based)
Appendix G contains the actual settings and values for each of these parameters.
4 DataMatrix was chosen because the tools necessary for processing and producing a DataMatrix barcode are not
proprietary, and are freely available and open sourced.
23
23
5.3.4 Sample test images
Two examples of test images generated with the custom image generation program (see
appendices) are shown in Figures 11 and 12.
Figure 11: A test image that is 3000 pixels wide. The group numbers represent the same number
of pixels in half a line pair in both images. For example, group 10 has twenty pairs of black and
white lines that are each 10 pixels wide.
24
24
Figure 12: A test image that is 24000 pixels wide.
5.4 Printer setup
5.4.1 Printer selection
An informal survey by the author revealed that the overwhelming majority of dentists that own
digital radiographic equipment purchased dedicated inkjet printers for printing the occasional
digital radiograph, typically for insurance companies. A minority purchased dedicated thermal
dye printers, and the group that did not purchase a dedicated printer often relied on prints from
the general office laser printer.
Given the findings of the author’s survey, the types of printers that were examined were:
- Inkjet
o 3 colour + black : Epson Stylus C68
o 4 colour + black: Kodak ESP9
- Laser
o Black/white: HP LaserJet 4050
o Colour: Lexmark C543DN, Xerox 8860
- Thermal dye: Kodak CMI1000
25
25
5.4.2 Media selection
Although glossy prints have been shown to be subjectively superior to matte prints (Gijbels et
al., 2004), there are no objective reasons to choose one medium over another. Therefore,
multiple media types need to be tested. Some media are only compatible with one printer type,
and in general, higher quality media for inkjet printers are incompatible with laser printers, and
vice versa.
As such, the media comparisons are generally confined to a single printer. That is, the ability of a
printer to produce better or worse results on various media can be evaluated, but since a
particular medium is not available for all printers, the effect of the medium itself cannot be
established across a broad range of printers.
5.4.3 Excluded printer and media types
Laser film printers and transparencies will not be examined because they produce images on
transmissive media, which cannot be evaluated in a uniform way in a study that deals primarily
with reflective media. Prior research has already established that laser film printers on
transmissive film produce subjectively better images than inkjet prints on both transparencies
and glossy paper (Bley et al. 2003; Gijbels et al., 2004).
5.4.4 Print setups used for this study
All printers used their manufacturer-recommended or compatible inks, so the hardware and
consumable variables in this study were limited to the printer and paper types. The 11 print
setups examined in this study are outlined in Table 1. The estimated cost per page (at 8% page
coverage) was determined based on the retail prices of the paper and ink consumed, and the
manufacturers' published ink consumption data.
Table 1: Summary of the 11 print setups used in this study. The abbreviations are used to
describe these print setups for the remainder of the document. The costs per page were based on
8% ink coverage.
Abbreviation Printer Type Printer Model Paper Type
Ink cost per page
(cents)
Paper cost per page
(cents)
Total cost per page
(cents)
CMI1000(glossy) Thermal dye Kodak CMI1000 Kodak professional photo glossy 136.95 86.00 222.95
ESP9(glossy) 4+1 inkjet Kodak ESP9 Kodak everyday photo glossy 6.80 20.95 27.75
ESP9(plain) 4+1 inkjet Kodak ESP9 plain 6.80 1.00 7.80
26
26
C68(glossy) 3+1 inkjet Epson C68 Epson glossy photo 31.89 49.00 80.89
C68(plain) 3+1 inkjet Epson C68 plain 31.89 1.00 32.89
C543(plain) colour laser Lexmark C543DN plain 44.06 1.00 45.06
C543(plain++) colour laser Lexmark C543DN HP premium choice laser 44.06 5.12 49.18
C543(glossy) colour laser Lexmark C543DN HP presentation glossy 44.06 5.76 49.82
8860PCL(plain) colour laser Xerox 8860 PCL plain 3.89 1.00 4.89
8860PS(plain) colour laser Xerox 8860 PS plain 3.89 1.00 4.89
4050(plain) B/W laser HP 4050 plain 2.98 1.00 3.98
5.5 Production of test prints
For each of the 11 prints setups, 5 prints at different source image resolutions were produced. All
images were printed at a width of 5 inches, and the source image widths were 1500, 3000, 6000,
12000 and 24000 pixels to produce prints with source image resolutions of 300, 600, 1200, 2400
and 4800 pixels per inch, respectively. Table 2, below shows the print IDs assigned to each print,
as well as the print setup to which each print belongs.
Table 2: Print IDs and source test image sizes produced for each of the 11 print setups. 55 prints
were generated (5 test images x 11 print setups = 55 total images). The source test image ID is
the reference ID of the test image with the parameters shown at the bottom of the table (light
grey regions). The Print ID is the reference ID for each print produced on each printer for a
specific source test image ID. For example, print ID 27 corresponds to source test image 63
(1500 pixels per 5 inches of image width for a source test image at 300 pixels per inch) printed
on the CMI1000 thermal dye printer on glossy paper. The function of these IDs was to track the
data for each print in the database.
Source test image ID 63 61 72 73 74 Print setup
print ID
27 28 29 30 31 CMI1000(glossy)
32 33 34 36 37 ESP9(glossy)
38 39 40 41 42 ESP9(plain)
43 44 45 46 47 C68(glossy)
48 49 50 51 52 C68(plain)
65 66 67 69 70 C543(plain)
71 72 73 74 75 C543(plain++)
76 77 78 79 81 C543(glossy)
92 91 90 89 88 8860PCL(plain)
97 96 95 94 93 8860PS(plain)
102 101 100 106 103 4050(plain)
Source test image width (pixels)
1500 3000 6000 12000 24000
Source test image width (inches)
5 5 5 5 5
27
27
Source test pixels per inch (p.p.i)
300 600 1200 2400 4800
5.5.1 Spectrum of images produced for each print setup
An important feature of the test images is the maintenance of uniform line pair dimensions
between test images of different resolutions. In order to allow test images at various resolutions
for the same print setup to be directly compared to the results from images at different
resolutions, the parameters for the size of the test image, resolution of the test image, and print
size were recorded. These images were tested for uniformity prior to embarking on the study as
follows:
The formula for determining the number of line pairs per printed inch for a specific line pair
group on a specific print was calculated using the following equation:
line pairs per printed inch (for group x) =
This line pairs per printed inch calculated in the equation above are independent of the print
resolution and size, so group numbers in different prints that have the same calculated line pairs
per printed inch are equivalent The numeric value of 2 in the denominator of the equation above
accounts for the fact that, at minimum, 2 pixels are necessary to produce a line pair.
An example of the formula applied to theoretical prints of various resolutions (that are easy to
conceptually calculate) is shown in Table 3 below.
28
28
Table 3: Determination of equivalency of line pair group numbers across various theoretical
prints with different resolutions. The purpose of this table is to demonstrate the equivalency of
the line pair densities between line pair groups printed at different resolutions on different
images. For example, since the group of line pairs labeled "1" in image A and the group of line
pairs labeled 16 in group E both have 100 line pairs per printed inch, they are equivalent. The
group numbers correspond to the groups in Section 2 of Figure 10.
Image reference label
Printed image width in inches
Line pairs per group
Group number being examined
Image width in pixels
Resolution of print (pixels per inch)
Line pairs per printed inch
A 5 20 1 1000 200 100
B 5 20 2 2000 400 100
C 5 20 4 4000 800 100
D 5 20 8 8000 1600 100
E 5 20 16 16000 3200 100
It can be seen that if all other factors are kept the same, doubling the resolution of a test print
means that the corresponding group number that will have the same number of line pairs per
printed inch would be doubled as well. Therefore, in this example, Group 1 in image A is
equivalent to Group 2 in image B, and so forth, up to being equivalent to Group 16 in image E.
The usefulness of this feature is that any print at any given resolution can be used as a test print
for observations, as long as the resolution is neither too low nor too high. If resolution is too
low, print quality may exceed the output produced by the print. And if it is too high, observers
may have difficulty making an observation if lines are too closely spaced (as shown in see
Figure 7 to observe the difference that resolution can make for observers).
5.5.2 Empirical confirmation of continuity of images at different resolutions
One printer (Lexmark C543DN) was tested by the automated analysis of spatial resolution to
ensure that the same resolution values were obtained for prints printed at 4 different resolutions:
600, 1200, 2400, 4800 pixels per inch (p.p.i.). Note that the 300 p.p.i. image was excluded from
this test because the resolution of the printer exceeded that of the smallest line pair produced by
29
29
this image. Three different print setups were tested on this printer, resulting in a total of 12 cutoff
measurements, which are summarized in Table 4 below.
Table 4: Measured maximum line pairs per printed inch for 4 different resolutions on 3 different
print setups on the Lexmark C543DN printer. With the exception of the number of prints
analyzed, the values have units of line pairs per printed inch.
Number of prints analyzed
Mean Line
pairs/ printed
inch Std.
Deviation Std.
Error
95% Confidence Interval for
Mean Printer setup Lower
Bound Upper Bound
Colour laser, C543DN, plain paper
4 150.0 0.0 0.0 150.0 150.0
Colour laser, C543DN, HP premium choice laser paper
4 150.0 0.0 0.0 150.0 150.0
Colour laser, C543DN, HP presentation glossy paper
4 152.5 5.0 2.5 144.5 160.5
Total 12 150.8 2.9 0.8 149.0 152.7
These results lend credence to the assertion that the images produced at different resolutions do
indeed exist on a continuum and that groups with purported equivalent line pairs per printed inch
do actually appear the same in prints of varying overall resolution.
While it is true that the use of a printer to empirically evaluate the equivalency of different
groups of line pairs on different prints may introduce a level of measurement error, the point is to
demonstrate that it works in actual prints, as opposed to in theory (because applying this method
to the digital source images directly produces perfect agreement via the formula in section 5.5.1).
30
30
5.6 Observations by human subjects
As mentioned in section 5.5.1, the test images and prints in this study exist on an overlapping
scale to produce a spectrum of line pair groups for each print setup5.
5.6.1 Observer group composition
One image was selected from each print setup for evaluation of spatial resolution by a panel of
35 observers. The first 5 volunteers of each observer type (listed in Table 6) that consented to
participate were enlisted for the study. At the end of the evaluation, observers were asked to
select a single overall favorite from the prints presented. The total number of observations was
385 for spatial resolution (35 individuals times 11 observations each) and 35 favorites selected (1
favorite per observer). The composition of the observer panel is shown in Table 5 below.
Table 5: Composition of observer groups
Group Name Number in group
Oral radiologist 5
Non-radiology dental specialist 5
General dentist 5
Graduate dental student 5
Undergraduate dental student 5
Dental assistant 5
Laypersons 5
Total 35
5.6.2 Instructions for observers
To standardize the observation process, all the observers were required to read the same set of
instructions prior to beginning the evaluation. The specific instructions presented to observers
are shown in Appendix F. The consent form that each observer received is shown in Appendix F
5 The print results from selected print setups (3 x C543 print setups) were used to validate this assertion. See section
5.5.2 for details on how the equivalency of various groups between prints was ascertained.
31
31
5.6.3 Data collection
The observers entered their observations into a secure custom-coded web page6 that
automatically saved each value as it was entered. The data were stored in a database until
analysis. The data were then formatted, anonymized and transferred to SPSS Version 17 (SPSS
Inc., Chicago, IL) for statistical analysis.
5.7 Automated analysis
All 55 prints produced for this study were scanned, processed and analyzed.
5.7.1 Analysis program
A custom analysis program was written in the Python programming language to analyze the
scans from each print. The code for the program is shown in the appendices. It is important to
note that this program produced perfect analytic results on the computer-generated test image
produced by the program shown in the appendices. This means that the differences seen from a
perfect result are the result of the printing/scanning process. The same scan setup was used for
all the prints, and they were all scanned at the same time, so the systematic error introduced by
the scanning process should be uniform, which means that any differences noted by the
automated program should be caused by differences inherent in each print setup.
The steps for automated analyses of the test images are: scanning, pre-processing for the analysis
program (de-skewing, derotating, cropping, conversion to 8 bit), and processing via the analysis
program.
5.7.2 Scanning test prints
All printouts were scanned with a high resolution flatbed scanner (Epson V700 photo, Epson
Seiko Corporation, Tokyo, Japan). Prior to embarking on this study, the scanning quality
threshold at which no further analytical benefit existed was established as follows: A pilot study
consisting of 5 test prints at various resolutions (300, 600, 1200, 2400, 4800 p.p.i.) produced by
an Epson C68 inkjet printer (Epson c68 inkjet, Seiko Epson Corporation, Tokyo, Japan) on plain
6 https://secure.milannium.com/thesis2/observations
32
32
paper did not demonstrate any statistical difference in automated analyzed results for images
scanned at 1200 p.p.i. and those scanned at 3200 p.p.i. or 6400 p.p.i., because a 1200 p.p.i. scan
seemed to be beyond the limit of dot sizes produced by the test printer. As a precaution, the scan
resolution was fixed at 3200 p.p.i. for the study.
The following steps were taken to ensure reproducibility. The same scanner and scan setup was
used for every scan. No contrast or brightness adjustments were made to the source image. Prior
to scanning, swatches with the blackest and whitest available printed media are used to calibrate
the scanner for contrast range. None of the prints produced blacks or whites that approached the
limits established by the black and white calibration swatches. One test print was scanned and
analyzed multiple times in order to measure the variation present in the experimental methods.
The scans were saved in as 16 bit grayscale .tif images at a resolution of 3200 p.p.i. Each image
was approximately 160 Megabytes in size. No filters or adjustments were enabled during the
scanning process, other than the determination of the maximal contrast range. Once the scan
parameters were established, they remained constant for the entire study. Therefore, if any errors
or biases existed in the scanner, they were systematic, and therefore did not contribute to
observed differences between experimental groups.
5.7.3 Illustrative comparison of a pre-printed test image to a scanned print
A resized example of an original test image is shown in Figure 13, below. The Moiré pattern
seen on the narrower lines is an artifact of resizing the image for display or for printing. If the
original PDF file is closely examined, uniform line pairs are seen in all areas of the image.
33
33
Figure 13: A resized version of a digital source image that is produced for printing an analysis.
A resized scanned image of a plain paper print produced by an inkjet printer is shown in Figure
14. Note the decrease in uniformity of the solid blocks on the image, as well as the decrease in
contrast range (the blacks and whites both appear greyer) compared to the original image in
figure 13.
34
34
Figure 14: A scan of a print produced with the digital image in the previous figure. Note the
mottling, loss of line pair resolution and reduced contrast range.
The image in figure 14 has been de-rotated, cropped and de-skewed manually as part of the
preprocessing of the scan. See figure 15 for an example of de-rotation, deskewing and cropping
of an image.
Figure 15: A stylistic example of the pre-analysis processing that was sequentially applied to
scanned images. The scanned image was de-rotated so that the top of the original image was
35
35
horizontal. The image was deskewed so that the right edge was vertical. The excess scanned
space around each image was cropped out. This resultant image was saved as an 8 bit lossless
TIF file, and was subsequently analyzed by the automated system.
The analysis program expects 8 bit images, so the image depth was reduced from 16 bits to 8 bits
after the pre-processing step was performed. All the images used for data analysis in this study,
as well as the results of the analyses and the programs written to perform them are available on
the data DVD that accompanies this thesis.
This image will be used to demonstrate how the automated analysis works. Please note that this
example was produced before the introduction of the 2D barcode to the test images, so it does
not look exactly like an actual test print used in the study. However, this image is functionally
identical to the actual test images used in the study.
5.7.4 Analytic steps
The test pattern was defined with rigid parameters in order to facilitate easy automated analysis.
The steps performed by the analysis program are described below.
The test image is divided into 7 equal-sized rows, which are numbered in red for illustration
purposes in figure 16, below.
36
36
Figure 16: The image is divided into 7 rows of equal height. Row 0, image text; Row 1, group
orientation stripes; row 2: white/black line pairs; row 3: white/50% grey line pairs; row 4: 50%
grey/black line pairs; row 5: black to white grayscale (30 steps); row 6: white to black grayscale
(30 steps).
As human observers only rated row 2, the automated analysis will only be described for that row
in this description of the methodology. The analysis for rows 3 and 4 is the same as that of row 2.
Prior to detailed image analysis, each image is opened and read, and the raw data for each row is
dumped to a .csv (comma separated variable) file. For the remainder of the analysis, only the
averaged data generated for the raw data file and stored in the .csv file is used.
The first step after the division of the image into 7 sections the determination of the boundaries
of each measurement group. The alternating black and white numbers in the top stripe of the test
image correspond to the number of pixels in half a line pair.
Row 1 is used to determine the boundaries of adjacent groups. In order for the algorithm to work
as planned, the right side of the image must be cropped exactly. If there is excess white scanned
page data to the right of the image, the algorithm fails, simply because the algorithm looks for
37
37
alternating areas of black and white in row 1. If there are white pixels to the right of the last real
row, the algorithm interprets that as an additional row. An illustrative example of the division of
the image into vertical columns is shown in figure 17, below.
Figure 17: Row #1 is used to determine the width of each column, each containing a group of 20
line pairs. The rows are numbered in red, and the columns corresponding to each group of line
pairs is numbered from right to left in blue. Each blue number indicates the width of half a line
pair in pixels in the original image. For example, in group 5, each black line is 5 pixels wide in
the source image.
The leftmost column in the image (column 7 in the example in Figure 17) is a reference column.
The shade of the rectangle in this column is always exactly halfway between the alternating
colours in the line pairs for the corresponding row. So in row 2, which alternates between black
and white, the leftmost column (column 7 in this case) is 50% grey.
In Figure 18, below, the raw data for row 2 is plotted for a perfect image and for the scan of the
printed image:
38
38
Figure 18: Comparison of a perfect data plot (top) and a scanned data plot (middle). The plotted
data for the scan is expressed in numeric form on the bottom of the figure. The meaning of each
39
39
variable in the table are: PixelPair: The group number. Also refers to the number of pixels in half
of a pixel pair. So a pixelPair value of 5 means that the black line is 5 pixels wide and the white
line is 5 pixels wide in the original test image. (The size of the pair on the printed image varies
because prints are made at different resolutions). Length: size of the column in scanned pixels.
Minimum: the smallest pixel intensity value measured in the entire column. Maximum: the largest
pixel intensity value measured in the entire column. Average: the average of all pixel intensities
in the column. range: the difference in pixel intensities between the maximum value in the
column and the minimum value in the column. maxPeaksAverage: the average value of the 20
highest peaks in this column. (The number 20 comes from the number of line pairs per column).
The unit of measure is pixel intensity value (ranging from 0-255 for an 8bit grayscale image).
minPeaksAverage: the average value of the 20 lowest valleys in this column. (The number 20
comes from the number of line pairs per column). The unit of measure is pixel intensity value.
peaksAverageRange: the difference in pixel intensity value between maxpeaksAverage and
minPeaksAverage.
A cropped section of the data from figure 11 is magnified for further analysis, as shown in figure
19.
40
40
Figure 19: The cropped region of the image from the previous figure that will be further
examined.
The magnified region of the cropped section from Figure 19 is shown below in a new plot,
Figure 20.
41
41
Figure 20: Each group is subdivided into 20 evenly spaced subdivisions. The maxima and
minima in each group are evaluated and averaged to produce minPeaksAverage and
maxPeaksAverage values for each group. The difference between these two values is the
peaksAverageRange for each group.
For the test images, 20 line pairs are produced for each column. Therefore, each column is
subdivided into 20 subdivisions. The maximum and minimum for each subdivision is
determined, and the average pixel intensity of these subdivisions is recorded.
The summarized data for each row is saved into a relational database and into a .csv file for easy
reference. These files are available from the principal investigator.
5.7.5 Data visualization and determination of spatial resolution
Visualizations of the various metrics that are produced in the analysis will be shown below for
comparative purposes:
The sample data set that will be used for visualizations that follow is shown in Table 6.
42
42
Table 6: The data set that corresponds to the sample scanned image being described in this
section.
The actual scan (including a zoomed portion of the smallest line pairs) is shown in figure 21
below.
43
43
Figure 21: Rows 1 and 2 of the scan of an plain paper inkjet print that is being examined with
the automated analysis are shown at top. This print was not included in the actual data analysis,
and is only presented as an example to demonstrate the automated analysis method. A zoomed
section of groups 1-6 from the scan are shown at bottom. There is a marked visual difference
between groups 4 and 5 in this scan.
For the scan depicted in figure 21, it should be noted that that the quality of the line pairs
changes markedly between groups 5 and 4. The line pairs are clearly distinguishable in group 5,
with few black "ladder rungs" visible. A 'ladder rung' represents a region of continuity between
two black lines in adjacent line pairs. The presence of a ladder rung means that the white line in
the affected line pair is discontinuous. There are significant numbers of rungs in group 4 and
below.
Every group is supposed to have 20 line pairs in it, so even though groups 2 and 1 appear to have
alternating line pairs present, they appear to have the wrong number of line pair groups present
(based on visual inspection of the scan), which means that multiple line pairs are being
combined, resulting in resolution loss.
44
44
Various calculations (minimum, maximum, average, range, standard deviation, average of 20
peak maxima, average of 20 peak minima, range between average peak minima and maxima)
were performed on the pixel intensity data belonging to each group of line pairs. The aggregated
data is shown in Table 7. The columns of the data set are colour-coded to match the data series in
the plots that follow.
45
45
Table 7: The values corresponding to the coloured data columns will be plotted in the graphs
that follow. The colour coding corresponds to the line colours in the plots that follow in Figure
22.
The data were plotted, as shown in the four graphs in Figure 22, to ascertain which of the
calculated metrics would provide the most robust measurement that would be analogous to a
human's interpretation.
46
46
47
47
Figure 22: The various variables from table are plotted on adjacent charts. Note the similarity
between them and the general dip that occurs between groups 4and 5.
The peaksAverageRange measurement was selected as the de facto measurement value, although
the preceding graphs have demonstrated a simple range or average would have produced similar
values. The reason for selecting the peaksAverageRange value is to ensure that one small spike
in the group does not end up being representative of the entire group of 20 line pairs. This way,
the values for each line pair in each group are averaged, which provides a more consistent result.
The value calculated above (group 5 is the cutoff value in this example) is the value that will be
used in comparisons between observers and the automated system, because they are the same
unit. However, when comparing prints of images of different resolutions to each other, the
resolution of the original image must be taken into account. For example, the plots above would
48
48
not be able to superimpose the data for, say, an image that has a resolution of 300 pixels per inch
over the data for an image with a resolution of 150 pixels per inch. In order to normalize the data
to account for original source image resolutions, the formula from section 5.5.1 is applied to
convert the result into the maximum line pairs per printed inch that are clearly visible on a print.
49
49
Chapter 6
6 Results
6.1 Observer spatial resolution data
The data points for spatial resolution observations were imported into the SPSS program. For the
purpose of the comparison, the 11 main print setup observations were separated from the 3 high-
resolution print observations that were also included in the test print series to determine intra-
observer reproducibility on prints of differing resolutions.
6.1.1 Effect of observer grouping on observation results
The effect of dental education level (observer groupings) was assessed to determine whether this
had any effect on the outcome. For each print, the results for each observer group were compared
to each other. A summary of the ANOVA results of line pairs per inch vs. observer group for
each print are shown in Table 8.
Table 8: ANOVA summaries of tests of equality of the mean line pairs per printed inch of the 7
observer groups for each print. The differences between observer groups were not significant for
any of the prints. F represents the F-ratio (which measures how different the means are relative
to the variability within each group). Sig represents the calculated significance level that tests the
null hypothesis that no observer group is different than any other observer group. The null
hypothesis could not be disproven for these observer groups.
Prints F Sig.
Colour laser, C543DN, Plain paper 0.578 0.745
Colour laser, C543DN, HP premium choice laser paper 0.628 0.706
Colour laser, C543DN, HP presentation glossy paper 0.466 0.827
4+1 inkjet, ESP9, plain paper 0.742 0.621
4+1 inkjet, ESP9, everyday glossy photo paper 0.367 0.894
Colour laser, 8860 PCL, plain paper 1.874 0.121
Colour laser, 8860 Postscript, plain paper 0.877 0.524
Thermal dye, CMI1000, glossy professional paper 0.565 0.754
Black/white laser, 4050, plain paper 0.409 0.867
3+1 inkjet, C68, glossy photo paper 0.918 0.497
3+1 inkjet, C68, plain paper 0.578 0.744
50
50
The differences between groups were not significant for any of the prints evaluated. Therefore,
for the remainder of the analysis, stratification by group type was eliminated; all the observer
data for a specific print were considered together.
6.1.2 Observer test-retest reliability
The ability of observers to reproduce their observations was tested on four observers, who made
a second set of observations (for a total of 44 repeated observations) at least two months after the
initial series of observations. The Pearson correlation coefficient was 0.332 for test-retest
reliability, which was statistically significant at the 0.05 level (2-tailed), which suggested that
observers had a poor ability to reproduce their observations.
6.1.3 Effect of print setup on spatial resolution observations
The pooled observer results for each print were compared to ascertain whether different print
setups produced statistically significant differences in observed spatial resolution. The tabulated
data and plot for each observed print is shown in table 9, below. The plot of the means and
standard errors for each print setup follows in Figure 23.
51
51
Table 9: Tabulated data of line pairs per printed inch for each observed print. N is the number of
observations. The remainder of the values in the table are line pairs per printed inch. (The
observed line pairs per printed inch were calculated using the formula for converting observed
group numbers in prints to line pairs per printed inch in section 5.5.1)
N Mean Std.
Deviation Std. Error
95% Confidence Interval for Mean
Maximum
Print setup Lower Bound
Upper Bound
Minimum
Colour laser, C543DN, Plain paper
35 132.8 29.8 5.0 122.6 143.1 70.6 200.0
Colour laser, C543DN, HP premium choice laser paper
35 123.4 32.1 5.4 112.4 134.5 63.2 200.0
Colour laser, C543DN, HP presentation glossy paper
35 131.3 35.4 6.0 119.2 143.5 66.7 200.0
4+1 inkjet, ESP9, plain paper
35 92.7 22.4 3.8 85.0 100.4 42.9 150.0
4+1 inkjet, ESP9, everyday glossy photo paper
35 98.3 25.9 4.4 89.5 107.3 0.5 120.0
Colour laser, 8860 PCL, plain paper
35 80.3 16.9 2.9 74.5 86.1 46.1 100.0
Colour laser, 8860 Postscript, plain paper,
35 80.6 42.0 7.1 66.2 95.1 42.9 300.0
Thermal dye, CMI1000, glossy professional paper
35 81.3 34.4 5.8 69.5 93.1 33.3 150.0
Black/white laser, 4050, plain paper
35 99.1 34.4 5.8 87.3 110.9 46.2 200.0
3+1 inkjet, C68, glossy photo paper
35 128.5 121.1 20.5 87.0 170.1 50.0 600.0
3+1 inkjet, C68, plain paper
35 93.4 24.9 4.2 84.8 101.9 46.2 120.0
52
52
Figure 23: Ordered plot of observer results for spatial resolution evaluations. Higher spatial
resolutions are better. The y-axis represents the various print setups and the x-axis represents the
line pairs per printed inch for each print setup. The pink boxes represent groups of statistically
similar print setups.
One-way ANOVA for the pooled data that tests the hypothesis that observed print spatial
resolutions are the same between all the prints is shown in Table 10.
53
53
Table 10: One-way ANOVA for the pooled data that tests the null hypothesis that observed print
spatial resolutions are the same between all prints. The null hypothesis was proven false. F is the
F-ratio; Sig is the significance level for the testing of the null hypothesis; df is degrees of
freedom. (The observed line pairs per printed inch were calculated using the formula for
converting observed group numbers in prints to line pairs per printed inch in section 5.5.1)
Observed line pairs / printed inch for each print
Sum of Squares
df Mean Square F Sig.
Between Groups 156690.868 10 15669.087 7.171 .000
Within Groups 817175.561 374 2184.961
Total 973866.430 384
The null hypothesis that all observed print spatial resolutions are the same is false. Levene's test
of homogeneity of variance was applied to the data to determine which post-hoc test to use, as
shown in Table 11.
Table 11: Levene's test evaluates the null hypothesis that variances between sample sets are
equal. The null hypothesis was false. df1 and df2 represent the two degrees of freedom and Sig
represents the significance level of the null hypothesis test.
line pairs / printed inch
Levene Statistic df1 df2 Sig.
2.945 10 374 .001
The hypothesis that the variances were homogenous was false, so Dunnett's T3 post-hoc test was
applied.
The matrix of p-values less than 0.05 for significant differences between print setups is shown in
Table 12:
54
54
Table 12: The matrix of p-values less than 0.05 for significant differences between prints setups,
as determined by observers. The Lexmark C543DN printer produced prints of significantly
different spatial resolution than the other printers in the study, except for the Epson C68 printer
with glossy paper.
CM
I10
00
(glossy)
ESP9
(glossy)
ESP9
(plain
)
C6
8(glo
ssy)
C6
8(p
lain)
C5
43
(plain
)
C5
43
(plain
++)
C5
43
(glossy)
88
60
PC
L(plain
)
88
60
PS(p
lain)
40
50
(plain
)
CMI1000(glossy)
0.000 0.000 0.000 ESP9(glossy)
0.000 0.001 0.000
ESP9(plain)
0.000 0.032 0.002 C68(glossy)
C68(plain)
0.000 0.002 0.000 C543(plain) 0.000 0.000 0.000
0.000
0.000 0.000 0.002
C543(plain++) 0.000 0.032 0.001
0.002
0.000 0.001 C543(glossy) 0.000 0.002 0.000
0.000
0.000 0.000 0.014
8860PCL(plain)
0.000 0.000 0.000 8860PS(plain)
0.000 0.001 0.000
4050(plain)
0.002
0.014
A 1200 by 1200 dpi colour laser printer (Lexmark C543DN) is significantly different from all
the other printers except the Epson C68 inkjet printer with glossy paper. The type of paper used
in the C543DN generally did not affect the outcome, so for this colour laser printer, the output
spatial resolution is not affected by the paper type.
6.2 Observer Favourites
The tabulated data for the 35 favorites chosen by the observers is shown in Table 13. The data is
represented visually in a pie chart in Figure 24.
55
55
Table 13: Overall favourite prints selected by observers, in descending order of preference. The
top 3 prints represented over 80% of those selected by observers.
Print setup Frequency Percent Cumulative Percent
4+1 inkjet, Kodak ESP9, everyday glossy photo
18 51.4 51.4
Thermal dye, CMI1000, glossy professional photo
7 20.0 71.4
3+1 inkjet, Epson C68, Epson glossy photo
4 11.4 82.9
Colour laser, Lexmark C543DN, HP presentation glossy
3 8.6 91.4
Colour laser, Lexmark C543DN, HP premium choice laser
1 2.9 94.3
Colour laser, 8860 Postscript, plain
1 2.9 97.1
Black/white laser, HP 4050, plain
1 2.9 100.0
Total 35 100.0
A pie chart of the observer favorites are shown in Figure 24.
56
56
Figure 24: Overall favorite prints selected by observers.
Observers chose the glossy print from the Kodak 4+1 inkjet printer over half the time, and
selected the glossy prints from the Kodak CMI100 thermal dye printer and the Epson C68
thermal dye printer 20% and 11% of the time, respectively. In all, 91% (32/35) of the selected
favourites were on glossy paper, which suggests that observers have a marked preference for
glossy paper (given that over half of the paper used in this study was plain paper). No non-glossy
print was selected by more than one observer.
Observer favorites were not uniform between the observer groups. The cumulative plot of
observer favourites is shown in Figure 25.
51%
20%
11%
9%
9%
4+1 inkjet, Kodak ESP9, everyday glossy photo paper
Thermal dye, Kodak CMI1000, glossy professional photo
3+1 inkjet, Epson C68, Epson glossy photo
Colour laser, Lexmark C543DN, HP presentation glossy
Other (Aggregated: non-glossy)
91% glossy
9% not glossy
57
57
Figure 25: Breakdown of favorite prints by observer dental education level.
6.3 Automated analysis of spatial resolution
The computed analysis of spatial resolution examined all 55 prints produced for this study.
The results of the automated analysis of spatial resolution are shown in Table 14.
58
58
Table 14: The computed analysis of spatial resolution of all printouts produced for this study.
With the exception of the number of measurements, all values have a unit of line pairs per
printed inch. (The line pairs per printed inch were calculated using the formula for converting
group numbers from the automated analysis of prints to line pairs per printed inch in section
5.5.1)
Number of measurements Mean
Std. Deviation
Std. Error
95% Confidence Interval for Mean
Minimum Maximum
Lower Bound
Upper Bound
3+1 inkjet, C68,. plain paper 5 98.6 14.0 6.2 81.3 116.0 75.0 109.1
Colour laser, C543DN, plain paper
4* 150.0 0.0 0.0 150.0 150.0 150.0 150.0
Colour laser, C543DN, HP premium choice laser paper
4* 150.0 0.0 0.0 150.0 150.0 150.0 150.0
Colour laser, C543DN, HP presentation glossy paper
4* 152.5 5.0 2.5 144.5 160.5 150.0 160.0
4+1 inkjet, ESP9, plain paper 5 97.0 17.3 7.8 75.5 118.5 75.0 120.0
4+1 inkjet, ESP9, everyday glossy photo paper
5 103.0 18.6 8.3 79.9 126.1 75.0 120.0
colour laser, 8860 PCL, plain paper 5 83.5 11.9 5.3 68.7 98.2 75.0 100.0
colour laser, 8860 postscript, plain paper
5 96.8 12.8 5.7 80.9 112.7 75.0 109.1
Thermal dye, CMI1000, glossy professional paper
5 79.9 6.8 3.1 71.4 88.4 75.0 88.9
Black/White laser, 4050, plain paper 4* 148.2 33.7 16.8 94.6 201.8 100.0 171.4
3+1 inkjet, C68,. glossy photo paper 5 107.0 19.9 8.9 82.3 131.7 75.0 120.0
*If the smallest line pairs produced by a low-resolution print resolved fully, that print was
automatically excluded from the analysis. This typically occurred with the laser printers.
The plot of the data from Table 14 is shown in Figure 26, below.
59
59
Figure 26: Ordered plot of automated spatial resolution evaluations. The print setups are on the
y-axis and the line pairs per printed inch for each print setup are on the x-axis. Higher spatial
resolutions are better. The pink boxes represent groups of statistically similar print setups.
The results of the one-way ANOVA testing the equivalency of each print setup in terms of the
maximum producible line pairs per printed inch are shown in Table 15.
60
60
Table 15: One-way ANOVA that tests the null hypothesis that all prints have an equivalent
spatial resolution, based on the automated analysis. The null hypothesis was false. F represents
the F-ratio; Sig represents the significance level of the null hypothesis test; df represents the
degrees of freedom.
Computed line pairs / printed inch for each print
Sum of Squares df Mean Square F Sig.
Between Groups 36267.131 10 3626.713 14.750 .000
Within Groups 9835.188 40 245.880
Total 46102.318 50
The hypothesis that the prints have the same spatial resolution is false. Levene's test of
homogeneity of variance was performed to determine which post-hoc test to use (see Table 16).
Table 16: Levene's test evaluates the null hypothesis that variances between sample sets are
equal. The null hypothesis was false. df1 and df2 represent the degrees of freedom and Sig
represents the significance level of the null hypothesis test.
line pairs / printed inch
Levene Statistic df1 df2 Sig.
2.915 10 40 .008
The null hypothesis that the variances were homogenous was false. Therefore, Dunnett's T3 was
used as a post-hoc test.
The matrix of p-values that are less than 0.05 for significant findings is shown in Table 17.
61
61
Table 17: The matrix of p-values less than 0.05 for significant differences between prints setups,
as determined by the automated analysis. The Lexmark C543DN printer produced prints of
significantly different spatial resolution than the other printers in the study, except for the Epson
C68 printer with glossy paper and the HP 4050 with plain paper.
CM
I10
00
(glossy)
ESP9
(glossy)
ESP9
(plain
)
C6
8(glo
ssy)
C6
8(p
lain)
C5
43
(plain
)
C5
43
(plain
++)
C5
43
(glossy)
88
60
PC
L(plain
)
88
60
PS(p
lain)
40
50
(plain
)
CMI1000(glossy)
0.000 0.000 0.000
ESP9(glossy)
.049
ESP9(plain)
0.036 0.036 0.023
C68(glossy)
0.049
C68(plain)
0.018 0.018 0.009
C543(plain) 0.000
0.036
0.018
0.004 0.012
C543(plain++) 0.000
0.036
0.018
0.004 0.012
C543(glossy) 0.000 0.049 0.023
0.009
0.001 0.004
8860PCL(plain)
0.004 0.004 0.001
8860PS(plain)
0.012 0.012 0.004
4050(plain)
As was the case for the observer results, the 1200x1200dpi Lexmark C543DN colour laser
printer was significantly different from most of the other printers in the study, except for the
Kodak ESP9 on glossy paper and the 1200x1200dpi HP4050 Black/white laser printer.
6.4 Automated analysis of contrast uniformity
The uniformity of each grayscale step in the bottom two rows of each image was analyzed. Each
image was shrunk by an incrementing factor with anti-aliasing7 until the uniformity of the
grayscale steps overall was less than 4 brightness units (out of 256 total, or 0.156%). This
brightness range was selected because it represented the threshold that a wide gamut of shrink
factors between different print setups. If too low of a threshold was selected, all the values would
have clustered at the high end of the shrink value spectrum, and of too high of a threshold was
7 In digital signal processing, anti-aliasing is the technique of minimizing the distortion artifacts known as aliasing
when representing a high-resolution signal (or image) at a lower resolution.
62
62
selected, the calculated values would have clustered at the low end of the shrink value spectrum.
The shrink factors that produced this level of uniformity were averaged for each of the 5 prints in
a series, for each of the 11 prints setups. The results are shown in Table 18, below:
Table 18: The computed data for contrast uniformity for all prints in this study. N represents the
number of observations. The remainder of the values are shrink factors (representing the factor
that image had to be shrunk by from its original size to produce a uniformity within 4 pixel
intensity units (out of 256 total) , averaged over all the grayscale steps.
N Mean
Std. Deviation
Std. Error
95% Confidence Interval for Mean
Minimum Maximum
Lower Bound
Upper Bound
3+1 inkjet, Epson C68, plain
5 16.0 0.0 0.0 16.0 16.0 16.0 16.0
Colour laser, Lexmark C543DN, plain 5 22.0 1.4 0.6 20.2 23.8 20.0 24.0
Colour laser, Lexmark C543DN, HP premium choice laser
5 21.2 1.1 0.5 19.8 22.6 20.0 22.0
Colour laser, Lexmark C543DN, HP presentation glossy
5 22.4 0.9 0.4 21.3 23.5 22.0 24.0
4+1 inkjet, Kodak ESP9, plain
5 16.4 0.9 0.4 15.3 17.5 16.0 18.0
4+1 inkjet, Kodak ESP9, everyday glossy photo
5 12.0 0.0 0.0 12.0 12.0 12.0 12.0
colour laser, Xerox 8860 PCL, plain 5 24.0 0.0 0.0 24.0 24.0 24.0 24.0
colour laser, Xerox 8860 postscript, plain 5 24.0 0.0 0.0 24.0 24.0 24.0 24.0
Thermal dye, Kodak CMI1000, glossy professional photo
5 1.0 0.0 0.0 1.0 1.0 1.0 1.0
Black/White laser, HP 4050, plain 5 25.6 5.4 2.4 18.9 32.3 16.0 28.0
3+1 inkjet, Epson C68,. Epson glossy photo 5 10.0 0.0 0.0 10.0 10.0 10.0 10.0
Total 55 17.7 7.4 1.0 15.7 19.7 1.0 28.0
The plot of the data from Table 18 is shown in Figure 27, below:
63
63
Figure 27: Ordered plot of automated contrast uniformity measurements. Lower values indicate
higher uniformity, because the image needed less anti-aliased shrinking to achieve contrast
homogeneity. Statistically similar groups of print setups are grouped together in pink boxes.
The thermal dye printer produced prints that had markedly more uniform contrast than the other
printers. The inkjet printers produced the next best contrast uniformity, and the laser printers
produced the worst results in terms of contrast uniformity. There were significant differences
between the thermal dye printer, the inkjet printers on glossy paper, the inkjet printers on plain
paper, and the group of laser printers.
For both inkjet printers tested, prints on glossy inkjet paper produced better contrast uniformity
than prints on plain paper.
64
64
6.5 Higher contrast uniformity is associated with more popular images
When the most popular print setups were highlighted on the ordered chart of contrast uniformity
(Figure 28), they clustered at the end of the spectrum with the best contrast uniformity.
Figure 28: Observers’ favorite images superimposed over the contrast uniformity plot. Popular
images were associated with high contrast uniformity.
Therefore, it can be concluded that within the confines of this study design, greater contrast
uniformity is associated with images that observers tend to prefer. This metric is potentially
useful in predicting which types of images people would prefer.
Interestingly, for both of the inkjet printers tested, the contrast uniformity is statistically
improved for glossy prints compared to plain paper prints. Therefore, based on this metric, inkjet
prints on glossy paper are superior to those produced on plain paper.
65
65
6.6 Higher spatial resolution is not associated with more popular images
In contrast to the previous metric, the most popular print setups did not show a discernable
pattern when highlighted on the spatial resolution plot in Figures 29 and 30, below.
Figure 29: Print popularity ratings superimposed over ordered plotted results of the spatial
resolution observations. Popular images were not associated with high spatial resolutions.
66
66
Figure 30: Print popularity ratings superimposed over the ordered plotted results of the
automated analysis. Note the similarity of the ordering of the results of the automated analysis to
that of the observer-generated results.
These findings suggest that spatial resolution measurements are not a good predictor of observer
preference within the bounds of this study.
6.7 The automated and observed spatial resolution results are highly correlated
The means of the observed and automated spatial resolutions were compared for each of the 11
print setups, and were found to be highly correlated with a Pearson's correlation coefficient of
0.782 and a two-tailed significance of 0.004. The scatter plot and best fit line are shown in Figure
31, below.
67
67
Figure 31: Scatter plot and best fit line for correlated observed and automated mean spatial
resolutions. The R value corresponds to Pearson's correlation coefficient. The formula for
calculating the expected automated mean from the observed mean is shown in the inset. the p-
value represents the 2-tailed significance.
The two outliers in Figure 31 were the HP 4050 laser printer on plain paper (that the computer
rated as having a high spatial resolution similar to the other laser printer in the study, but
observers rated poorly), and the Epson C68 inkjet printer with glossy paper (that the automated
analysis rated similarly to the other inkjet prints on glossy paper, but that the observers rated as
highly as the colour laser prints).
automated = 1.06*observed + 4.61R² = 0.612 R=0.782 p=0.004
60
80
100
120
140
160
70 80 90 100 110 120 130 140
auto
mat
ed
me
an s
pat
ial r
eso
luti
on
(l
ine
pai
rs p
er
pri
nte
d in
ch)
observed mean spatial resolution (line pairs per printed inch)
68
68
Chapter 7
7 Discussion
7.1 Dental education levels produced no differences in spatial resolution observations
One result from this study that was distinct from previously published data was the absence of a
relationship between level of dental expertise and spatial resolution observations. In previous
studies (Otis et al., 2005), the diagnosis of caries on hard copy media was used as the method by
which modalities were compared, which superimposed the skill of caries diagnosis over the task
of determining which print medium produced a higher quality output. The independence of the
observations from the type of observer has implications for future studies that employ this
methodology. Not having to rely on the expertise of dental specialists or general dentists for
evaluating images is of great benefit, because less dentally qualified observers are capable of
producing results that are statistically indistinguishable from those produced by trained dentists.
7.2 General agreement of automated analysis with observations of spatial resolution
The algorithm produced for this study was capable of producing spatial resolution results that
correlated with those produced by observers. Thus, the need for observers to determine spatial
resolution could potentially be eliminated entirely, which would be of even greater benefit than
simply relying on non-dentally trained observers. Intra-automated-analysis reliability is high
once the data has been digitized. The same cannot be said for intra-observer reliability, which
generally has been poor to moderate in all the previous studies on this subject, and poor in this
study.
7.3 Observer preference for glossy prints
A marked preference for glossy prints was noted, regardless of the printer modality used.
Surprisingly, the second most popular print (produced by the Kodak CMI1000 thermal dye
69
69
printer) had poor spatial resolution compared to the other printers in the study, but was the
glossiest and produced the highest contrast uniformity by a wide margin.
7.4 Reconciliation of overall observer preference with measured results
The print that was chosen as an observer favorite overall (Kodak ESP9 on Kodak Everyday
photo paper) was glossy, had a moderately high contrast uniformity and a moderate spatial
resolution. It was not a top performer on any of the tests, but did not have any obvious
deficiencies either.
In contrast, the second most popular print (Kodak CMI1000 on Kodak professional glossy photo
paper) had exceptional contrast uniformity and was printed on the glossiest paper of all the
papers tested, but performed poorly on the spatial resolution test, which hindered its ranking.
The third most popular print was an inkjet print on glossy paper (Epson C68 on Epson glossy
photo paper) as well and performed similarly to the most popular image.
The laser printed images all scored low on the contrast uniformity test, which significantly
hindered their popularity rankings. A notable exception is the lone laser print that was made on
glossy paper (1200 by 1200dpi C543DN colour laser on HP presentation glossy laser paper),
which performed significantly better (4th most popular) in the popularity contest, despite having
the same test results as the print setups from the same printer that used non-glossy paper. This
implies that the glossiness of the paper is a significant factor in influencing an observer's
decision regarding print ratings.
7.5 Cost vs. quality
This study has demonstrated that the most expensive prints do not necessarily produce superior
results, from both objective and subjective points of view. The most favoured print (Kodak ESP9
4+1 inkjet with Kodak everyday glossy photo paper) cost 27.8 cents to produce, which was an
order of magnitude less expensive than the thermal dye prints that have historically been
suggested as the choice for dental radiographic printing. The cheapest print in the study was
produced by a black and white laser printer (HP 4050) on plain paper. This printer suffered from
very poor contrast uniformity, and was not selected by any observer as a favorite print.
70
70
7.6 Study limitations
This study was not large enough to draw sweeping conclusions about observer preference. It was
merely an attempt to determine whether the metrics proposed for evaluating printer output were
worth evaluating.
The metrics selected are not an exhaustive list of print properties. For example, one untested
property that emerged during testing was the production of non-uniform line pair widths on
prints. Informal interviews of observers revealed that they tended to dislike prints that produced
non-uniform line pair widths, so a future study should evaluate this property of prints as well.
The automated analysis system produced for this study is rudimentary, and although it performs
as reliably as all programs do in terms of reproducing results, much more work needs to be done
on more sensitive print outputs to determine whether the algorithm is capable of truly replacing
observers by mimicking them accurately. Note, however, that these assertions pertain to the
evaluation of print characteristics, rather than to diagnostic ability. The diagnostic significance of
these findings has not been determined.
For this type of study, the only real gold standard is not actually a printed hard copy, but a pixel-
to-pixel zoomed image on a computer screen that demonstrates the perfect line pairs that exist in
the source image. There is no way around this limitation, because there is no hard copy medium
of any sort that is capable of reproducing a high resolution digital image in its entirety.
7.7 Future directions
7.7.1 Technical considerations
The groundwork has been laid for future studies to simply employ and improve on the
methodologies established in this study. The software is open source and freely available to
everyone, as are the data sets examined in this study.
Standardizing the test methodology for evaluating hard copies of any type has benefits to the
research community in general, because it provides a foundation on which direct comparisons
71
71
can be made. At the very least, the current methods will be applied to a new crop of printers on a
regular basis, in order to help determine whether upgrades are necessary in dental offices, or
whether the touted improvements in newer printers are immaterial.
7.7.2 Clinical considerations
The basic parameters for evaluating print quality were explored in this study. However, the
diagnostic significance of these parameters is untested. For example, do prints produced by
thermal dye printers with excellent contrast uniformity at $2.22 per page have a greater
diagnostic value than those produced by a colour inkjet printer on glossy paper for $0.27 per
page? The next logical step, therefore, would be to perform a diagnostic experiment with prints
produced by these print setups in order to see whether differences that justify the price difference
exist.
An example of such an experiment would be the production of a series of prints from each print
setup of bitewing views with varying depths of interproximal decay, and subsequently having a
uniformly trained group of observers evaluate them. The results from such a study could be
compared to the results from the current study to ascertain whether the parameters evaluated had
any correlation with the diagnostic outcomes.
72
72
Chapter 8
8 Conclusion
This study introduced a novel method of evaluating the print results from various commodity
printers available to the general public at reasonable prices. The method does not rely on any
particular level of dental knowledge or expertise, which suggests that it may be useful for future
printer or hard copy evaluations that seek to eliminate the dental knowledge variable from an
already complicated task.
An automated analysis system that correlates with the results produced by observers was
introduced and tested. The general agreement of findings between the automated test and the
observed results is promising, and has the potential to replace observers entirely for the purposes
of print quality evaluations, once it has been evaluated and tested in greater detail.
The production of various quantitative metrics of print quality underpins this entire endeavor,
and of the print qualities tested, the contrast uniformity of a print and the use of glossy paper
appear to be associated with observer preference, whereas the production of prints with high
contrast resolution do not appear to be associated with increased operator preference.
The print that was selected by observers as a favorite print most often did not perform
spectacularly on any of the tests in this study, but did perform moderately well to very well on all
tests, without demonstrating any real deficiencies, and was an order of magnitude less expensive
than the thermal dye print. In contrast, the prints that tended to perform extremely well on one
test tended to perform poorly on others. For example, the laser printer that produced the highest
spatial resolution had poor contrast uniformity, and the thermal dye printer that had excellent
contrast uniformity had poor spatial resolution.
73
73
Bibliography
Benediktsdottir, I. S., Wenzel, A.. Accuracy of digital panoramic images displayed on monitor,
glossy paper, and film for assessment of mandibular third molars. Oral Surg Oral Med
Oral Pathol Oral Radiol Endod 98(2): 217-222; 2004.
Bley, T. A., Kotter E., Saueressig U., Springer O. S., Fisch D., Ghanem N. A., Langer M.. Using
receiver operating characteristic methodology to evaluate the diagnostic quality of
radiography on paper prints versus film. AJR Am J Roentgenol 181(6): 1487-90; 2003.
Brady, D. T. Digital radiography: a survey of dentists in Hawai'i. Hawaii Dent J 38(4): 10, 12-3;
2007.
Brian, J. N.,Williamson. G. F.. Digital radiography in dentistry: a survey of Indiana dentists.
Dentomaxillofac Radiol 36(1): 18-23; 2007.
Briggs, J. C., Klein, A. H., Tse, M. K.. Applications of ISO-13660, a new international standard
for objective print quality evaluation. Japan Hardcopy 1999: 281-285; 1999
Gijbels, F., G. Sanderink, Pauwels H., Jacobs R.. Subjective image quality of digital panoramic
radiographs displayed on monitor and printed on various hardcopy media. Clin Oral
Investig 8(1): 25-9; 2004.
Guerrant, G. H., W. S. Moore, Murchison D.F.. Diagnostic utility of thermal printed panographs
compared with corresponding computer monitor images." Gen Dent 49(2): 190-6; quiz
197-8; 2001.
Haak, R Wicht M.J., Nowak G., Hellmich M.. Influence of displayed image size on radiographic
detection of approximal caries. Dentomaxillofac Radiol 32(4): 242-6; 2003.
Lyttkens, K., Kirkhorn T., Kehler M., Andersson B., Ebbesen A., Hochbergs P., Jarlman O.,
Lindberg C.G., Holmer N.G.. Evaluation of the image quality of ink-jet printed paper
copies of digital chest radiographs as compared with film: a receiver operating
characteristic study. J Digit Imaging 7(2): 61-8; 1994.
McDowell, D. Q.. Standards Update. IS&T Reporter 18(1): 12-13; 2003
74
74
Mouyen, F., Benz C., Sonnabend E., Lodter J.P., Presentation and physical evaluation of
RadioVisioGraphy." Oral Surg Oral Med Oral Pathol 68(2): 238-42; 1989.
Otis, L. L.,Sherman R. G.. Assessing the accuracy of caries diagnosis via radiograph. Film
versus print. J Am Dent Assoc 136(3): 323-30; 2005.
Schulze, R. K., Schulze D., Voss K., Rottner M., Keller H.P., Dollmann K., Maager B., Wedel
M.. Quality of individually calibrated customary printers for assessment of typical dental
diagnoses on glossy paper prints: a multicenter pilot study. Oral Surg Oral Med Oral
Pathol Oral Radiol Endod 106(4): 578-86; 2008.