assessment of diagnostic accuracy using a digital camera for
DESCRIPTION
TRANSCRIPT
Assessment Of Diagnostic Accuracy Using A Digital
Camera For Teledermatology
Elizabeth Krupinski, PhDBen LeSueur, BS
Lansing Ellsworth, MDNorman Levine, MDRonald Hansen, MD
Nancy Silvis, MDPeter Sarantopoulos, BSPamela Hite, MD, MBA
James Wurzel, BSRonald Weinstein, MD
Ana Marie Lopez, MD, MPH
Patients’ Opinions*
• 24% satisfied by care from non-dermatologist
• 89% satisfied by care from a dermatologist
• 6% believe a generalist can treat their skin disease
• 87% say access to dermatologist very important to their health care
*Owen SA, Maeyens E, Weary PE. Patients’ opinions regarding direct access to dermatologic specialty care. JAAD 1997;36:250-256.
Goal The goal of this study was to compare
diagnostic accuracy of a dermatologic diagnosis based on in-person examination compared to diagnosis with
still photo images acquired
with a digital camera and
displayed on a CRT monitor.
Rationale Real-time video conferencing technologies
may not be available or may be too costly for some rural sites. Store-forward technologies may be more appropriate, and should be tested before they are used clinically.
Subjects & Readers
• 308 consecutive patients referred for specialty consultation by PCP or general dermatologist to the Dermatology Clinic at the University of Arizona Health Sciences Center
• 104 of these cases were ultimately biopsied
• 3 board-certified specialty dermatologists
Exams• Each dermatologist examined approximately 1/3 of the
patients in person at the AHSC clinic
• Rendered either a single diagnosis (75% of the cases, n = 230) or 2 or 3 differential diagnoses (25% of the cases, n = 78)
• Up to 5 photos of the lesion ROIs were taken with a digital camera by 4 medical students trained in the use of the camera
• Global and close-up shots
The Digital Camera
• Canon PowerShot600
• CCD image sensor
• 832 x 608 pixels
• 24-bit color resolution
• f/2.5 lens
• Built-in flash
• 150 kB file size
The Display
• Gateway 2000 computer
• Gateway CrystalScan color monitor: 1024 x 768
• PhotoImpact Album v 3.0 display software
• Brief patient history in each case folder
• Randomized case presentation
Procedure• Approximately 2 months after in-person exams, digital
images were examined• Same 3 dermatologists examined all 308 cases• Single most likely diagnosis rendered• Decision confidence: very definite, definite, probable,
possible• Rate image sharpness & color: excellent, good, fair, poor• Viewing time recorded
Analyses• In-person diagnosis = truth
• Correct match = digital diagnosis matches one of the differential possibilities listed during in-person
• Incorrect mismatch = digital diagnosis does not match any of the in-person differentials
• For biopsy analyses, biopsy = truth
Types of Cases Clinical Diagnosis Number of Cases
• Malignant or Premalignant 91• Benign Proliferations 74• Eczema/Dermatitis 36• Pigmented Lesions 32• Infections/Infestations 20• Papulosquamous Disorders 12• Urticarial & Allergic 5• Collagen/Vascular 1• Miscellaneous 37
Biopsied Lesions
Lesion Type Number Percent
• Infection/Infestation 3 3%• Pigmented Lesions 26 25%• Malignant/Premalignant 49 47%• Dermatitis/Eczema 4 4%
• Benign Proliferations 12 11%
• Miscellaneous 10 10%
Diagnostic Accuracy
F = 0.011, df = 2, p = 0.989
no differences in accuracy between dermatologists
0
10
20
30
40
50
60
70
80
90
Per
cent
Correct Incorrect
1
2
3
Reader
Decision Confidence
0
10
20
30
40
50
60P
erce
nt
VeryDefinite
Definite Probable Possible
1
2
3
Ave
Reader
Observer Variation• Intra-Observer Variation
– Reader 1: 90% agreement (n = 104 cases)– Reader 2: 85% agreement (n = 102 cases)– Reader 3: 76% agreement (n = 102 cases)
• Inter-Observer Variation– Reader 1 vs 2: kappa = 0.82– Reader 1 vs 3: kappa = 0.81– Reader 2 vs 3: kappa = 0.80
Correct Decisions Biopsy CasesIn-Person Photo vs Photo vs
Reader vs Biopsy Biopsy In-Person
1 80% 78% 87%
2 97% 76% 79%
3 90% 73% 85%
Mean 89% 76% 84%
Biopsy vs In-Person MismatchesBiopsy In-Person
Basal Cell Carcinoma Intradermal Nevus
Mycosis Fungoides Psoriasis/Parapsoriasis
Basal Cell Carcinoma Bowen's Disease
Folliculitis Vasculitis
Foreign Body Granuloma Basal Cell Carcinoma
Dysplastic Nevus Melanoma
Premalignant Keratosis Pyogenic Granuloma
Epidermoid Cyst Basal Cell Carcinoma
Basal Cell Carcinoma Foreign Body Granuloma/Dermatofibroma
Seborrheic Keratosis Squamous Cell Carcinoma/KeratoacanthomaBenign Lichenoid Keratosis Basal Cell Carcinoma
Image Quality
0
10
20
30
40
50
60
70
80
90
Per
cent
Excellent Good Fair Poor
1
2
3
Ave
Reader
Correlations
• Color & sharpness: r = 0.73
• Color & Decision Confidence: r = 0.48
• Sharpness & Decision Confidence: r = 0.47
• Decision confidence was not affected significantly by overall quality of the images
Viewing Time
0
10
20
30
40
50
60
70
80
90
100P
erce
nt
< 30 sec < 60 sec > 90 sec
1
2
3
Ave
Reader
Correlations
• View time vs color rating: r = 0.35
• View time vs sharpness rating: r = 0.24
• View time vs accuracy: r = 0.21
• View time vs confidence: r = 0.54
• View time vs single/multiple diagnoses: r = 0.15
Conclusions• Digital photography can be used to acquire
dermatology images for telediagnosis
• Image quality is overall excellent to good
• Diagnostic accuracy and confidence are high
• Digital image diagnoses compare favorably with in-person diagnoses and with biopsy results