Letter to the Editor

Computer-automated caries detection tools: a big payoff is possiblewhen user judgment is combined with the tool

Dear Sir,As one of the inventors of the Logicon Caries Detector

(LCD) software tool (1) discussed in a recent paper byWenzel et al. (2) appearing in this journal, I have thefollowing comments about their conclusions as well asguidance on how to use such tools to gain maximumbenefit.

The authors conclude from a laboratory study thatLCD is less accurate than human observers. We have, infact, demonstrated in a clinical study that dentists usingour software as a diagnostic aid can find 20% moredentinal caries than they can find without it, while notmisdiagnosing any additional healthy teeth. Our clinicalstudy was conducted under the guidelines of the U.S.Food and Drug Administration (FDA). The study pro-tocol was planned with the help of the University ofCalifornia, Los Angeles (UCLA) School of Dentistry. Itinvolved 18 general practice dentists evaluating 175surfaces in a normal office environment. The studyenabled us to receive FDA premarket approval fordistribution of LCD in the USA in September 1998. TheFDA’s approval notice and their report of the safety andeffectiveness of LCD is available on-line (3). I have alsorecently published a journal article (4) that describes theresults of our clinical study.

There could be a number of reasons why the authorsfound poor performance with LCD in their laboratorystudy. One of those reasons is clear from Fig. 2(b, d) intheir paper, which shows that they did not follow all ofthe instructions in the LCD User Guide distributed withthe product. They also overlooked some importantinformation displayed by LCD in their figure. As aresult, they claim that LCD missed a lesion in that figure,but evidence to the contrary is there.

In particular, in the last paragraph on page 8 of theLCD User Guide, we state that �it is important to haveregions of good tooth on both sides of the suspiciousregion being analyzed� when the V-tool (also referred toas a fan in the Guide and a cursor tool or wedge by theauthors) is opened to start an analysis. This is importantbecause our software program compares good toothmaterial to bad (demineralized/decay) material.

It is clear from the LCD computer screen shots shownin Fig. 2 (b, d) of the Wenzel et al. paper (2) that thisrequirement was not met. In particular, referring to thetooth density plots in the screen shots, one can see asharp decline in the density curves as they move from theocclusal end of the tooth toward the apical end as one

would expect when going from a region of good toothmaterial into a region of decay. However, these curves donot recover as they continue in the apical direction asthey should if they are passing through a region of decayand re-entering good tooth material. Since this pattern isthe same along each density curve across the enamel (10green contours) and into the dentin (5 blue contours), itsuggests that tooth decay is present. However, LCD didnot predict it because the density pattern does not meetall of the criteria in our software for predicting caries witha high probability. In this instance, the density curvesmay not recover because (a) the decay extends beyond thearea where the V-tool was opened on the apical end,and/or (b) the decay is contiguous with an area of cervicalburnout. The authors claim that decay is present, andthey are presumably correct in this particular case.

The authors did follow the instructions on page 8 ofthe LCD User Guide for opening the V-tool on theproximal surface of interest, including being certain it didnot extend onto the occlusal surface at one end andbeyond the cemento-enamel junction (CEJ) on the otherend. However, they missed the instruction about havinggood tooth material on both sides of the suspiciousregion. When using LCD, it is very important for thedentist to examine the tooth density plot to see if it meetsour criteria of good tooth material on both sides of thesuspicious region. The probabilities in Fig. 2 (b, d) arebased on comparing the density patterns to similar oneswhich meet this criteria in a database of caries problemsdeveloped at the UCLA School of Dentistry. Thatdatabase does not include a pattern like the one shown inFig. 2 (b, d) of the paper so the probabilities were low.Sometimes cervical burnout or the proximity of thedecay region to the CEJ can cause such a density pattern.In this particular case, the dentist should attempt toreposition the V-tool to include good tooth material onboth sides of the decay region. If cervical burnout ordecay near the CEJ prevents having good tooth materialon both sides of the suspicious region, the dentist shouldrely on the density plot alone as a warning that decaymay be present.

When we give training with our software, we cautionusers about such an occurrence. We recommend thatthey rely on the tooth density plots which are a morebasic diagnostic than the probabilities. Since the authorsmake no mention of the density plots, it appears thatthey do not understand their importance. Software tools

Eur J Oral Sci 2003; 111: 179–181Printed in UK. All rights reserved

Copyright � Eur J Oral Sci 2003

European Journal ofOral SciencesISSN 0909-8836

like LCD require user thought and interpretation toextract the maximum benefit. This paper is a goodexample of where such thought and interpretation couldhave been applied and a different conclusion reached.For the record, none of the authors have taken a trainingclass from us and only one saw us demonstrate LCD (inJanuary 1998). In addition, the authors did not ask uswhy the probabilities were low when a carious lesion wasfound to be present, and they apparently have notreviewed our Product Update Bulletin Number 5 publishedin April 2000 which warns LCD users about these issues.

The authors also mention that they had trouble findingthe dentino-enamel junction (DEJ) with LCD in some ofthe premolar teeth. In later versions of LCD we haveadded a contrast filter to help the dentist find thisboundary. In addition, the dentist can trace the bound-ary manually if necessary, and our User Guide providesinstruction on how to do a manual trace. Correctlyfinding the DEJ is very important since LCD is pre-dicting whether or not the decay crosses the DEJ into thedentin.

Their paper further states that LCD has not beenevaluated in the laboratory. This is not true. We con-ducted an extensive laboratory study with LCD inassociation with the UCLA School of Dentistry prior toour clinical study. We collected 320 images of extractedteeth with a range of caries problems, and we evaluatedthe images with and without LCD. The evaluations werethen compared to histological data taken after cross-sectioning the teeth. The results are presented in theabove-referenced FDA report (3). The most importantfindings are that (a) LCD showed significant improve-ment in finding caries penetrating into the dentin, but (b)LCD showed no improvement in finding caries in theenamel alone. Thus, our clinical study focussed onevaluating the efficacy of LCD for finding dentinallesions. We make no claims for capability to find enamellesions (3). We do not understand why enamel cariesproblems were included in this study (2) with LCD.

In closing, it is interesting to note that the perform-ance of these researchers in finding dentinal carieswithout LCD is very conservative, and many patientswith dentinal caries would go untreated. Averaging overthe four researchers’ results without LCD in Table 1 oftheir paper (2), their sensitivities for finding dentinalcaries were 37% with the old RVG sensor and 41%with the new RVG sensor, while their specificities were96% with the old sensor and 94% with the new sensor.By comparison, in our clinical study (3, 4) the 18dentists who participated had an average sensitivity of70% with an average specificity of 89% for dentinalcaries detection before using LCD. and when they usedLCD their sensitivity went up to 90% while their spe-cificity stayed the same. This raises questions about thevalue of the authors’ laboratory study to practicingdentists.

References1. Yoon DC, Wilensky GD, Neuhaus JA, Manukian N,

Gakenheimer DC. Quantitative Dental Caries Detection Systemand Method. U.S. patent 5 742 700.: Washington: U.S. PatentOffice.

2. Wenzel A, Hintze H, Kold LM, Kold S. Accuracy of com-puter-automated caries detection in digital radiographs com-pared with human observers. Eur J Oral Sci 2002; 110: 199–203.

3. U. S. Food and Drug Administration. Center for Devices andRadiological Health, Summary of safety and effectiveness data:Logicon Caries Detector. PMA no. P980025 September, 1998.Available at: www.fda.gov/cdrh/pma/pmasep98.html�. AccessedFebruary 28, 2002.

4. Gakenheimer DC. The efficacy of a computerized cariesdetector in intraoral digital radiography. JADA 2002; 133: 883–890.

David C. Gakenheimer, Ph.D.Northrop Grumman Information Technology

PO Box 471San Pedro, CA 90733–0471, USA

E-mail: dgakenheimer@northropgrumman.com

Response

Computer-automated caries detection by Logicon Caries Detector:does it work?

Dear Sir,Thanks to David Gakenheimer, Ph.D., Logicon Inc.

(now Northrop Grumman Information Technology) forhis interest in our study on the accuracy of a computer-automated caries detection program, named LogiconCaries Detector (LCD). Dr Gakenheimer has made anumber of references to our study (1) and also to a previouspublication on reproducibility of the LCD program (2).

Before we initiated these studies, Dr Gakenheimerhimself instructed me in the use of the program while wewere both in Paris to visit Trophy Radiology. For no lessthan 2 h, we discussed the possibilities of the programand how to run it. During installation of the program atour dental school, the academic staff, including myself,were trained again by the instructor from Trophy who

180 Letters to the Editor

sells the program (which runs only with Trophy imagesas far as I am aware). We read the instruction manualthoroughly before initiating the study and before train-ing the other observers. I am sure that all dentists, whomight buy the program, get similar detailed and time-consuming instructions.

The observers were instructed to use the LCD severaltimes on each tooth surface using the V-tool in differentangulations and finally to record what was the mostfrequent outcome of the LCD for that surface. Unfor-tunately, LCD does not provide a very consistent out-come; if the V-tool is changed just a few pixels, theoutcome can change immensely (2). Thus, the instructionto the observers was made in the way Dr Gakenheimerrecommends: �slightly varying the selected region inorder to verify the consistency of the results�. If theobserver suspected a dentinal lesion, he/she would surelytry to make the program also detect a lesion. Unfortu-nately, this was not possible with the program in many ofthe surfaces, which truly had lesions, as seen from thelow sensitivities obtained with the LCD for dentinallesions. In the example shown in Figs. 1 and 2, a cariouslesion extending into dentine is seen in the histologicsection of the tooth. The lesion is not broad and there issound tissue on both sides of the lesion. The programshould have been able to detect this. The sensitivities andspecificities obtained by the observers without the use ofLCD were in the range known from other in vitro cariesdetection studies. The cervical burn-out was of coursenever included in the V-tool.

Dr Gakenheimer reports a clinical study where dentists�validated� the diagnostic outcome with LCD (3). Besidesthe difficulties that arise when 18 practising dentists areto be calibrated, the design of all such clinical studiesonly allows information on whether a suspected lesion isa true or a false positive observation (when drilling in thesurface), since it is not ethical to perform operativetreatment in a surface that is believed by the dentist to besound. Clinical inspection of the outer surface of thetooth is not a valid criterion for the true state of thesurface (4). In a clinical study therefore, sensitivities andspecificities for the performance of a diagnostic tool cannever be obtained, and information is reduced to the

positive predictive values. A study on accuracy shouldtherefore be performed in the laboratory on extractedteeth and with a solid histological validation for the truestate of disease.

Dr Gakenheimer further states that an in vitro studywas performed before releasing the LCD (results dis-played on the internet). However, adequate experimentaldetails for this study are not provided, such as the ver-sion number of the software, the type of sensor that wasused, how it was used, and the number of observersparticipating in the study. If, for example, a differentsensor was used (!), these results with the LCD softwaremay not be applicable to the Trophy sensor with which itis sold.

In my opinion, no research results can be accepteduntil they are published in peer-reviewed scientific jour-nals by independent parties. Indeed, it is interesting tonote that a recent study on LCD reports findings similarto ours (Manual vs. automated computer-aided proximaldental caries assessment. Kang, Scheetz & Farman, JDent Res 2001; 80: 136, abstract 807). I am lookingforward to seeing the full paper on these results andother university studies on the LCD program. That a �bigpayoff is possible when user judgement is combined withthe tool� is yet to be demonstrated.

References1. Wenzel A, Hintze H, Kold LM, Kold S. Accuracy of com-

puter-automated caries detection in digital radiographs com-pared with human observers. Eur J Oral Sci 2002; 110: 199–203.

2. Wenzel A. Computer-automated caries detection in digitalbitewings: Consistency of a program and its influence onobserver agreement. Caries Res 2001; 35: 12–20.

3. Gakenheimer D. The efficacy of a computerized caries detectorin intraoral digital radiography. JADA 2002; 133: 883–890.

4. Wenzel A, Hintze H. The choice of gold standard for evalu-ating tests for caries diagnosis. Dentomaxillofac Radiol 1999; 28:132–136.

Professor Ann Wenzel, DDS, Ph.D., Dr. Odont.University of Aarhus

E-mail: awenzel@odont.aau.dk

Response

Dear Sir,Thank you for the opportunity to comment on the

letter by Dr Gakenheimer in which he mentions UCLA. Iwill limit my comments to those aspects of his letter.UCLA indeed had contracts with Logicon RDA toparticipate in a laboratory and clinical study. In thelaboratory study, UCLA provided to Logicon RDA datadescribing the performance of dentists in identifyingcaries by conventional radiographic examination. ALogicon RDA employee operated Logicon CariesDetector on the same images and compared its per-

formance to that of the dentists. In the clinical studyLogicon RDA employees trained the participating den-tists and also collected and analyzed the data. The resultsof both studies can be found in the document preparedfor the FDA by Logicon RDA. Neither UCLA nor Iendorse any claims of Logicon RDA for their product.

Stuart C. White, DDS, Ph.D.UCLA School of Dentistry

E-mail: swhite@ucla.edu

Letters to the Editor 181