![Page 1: Integration of Radiologists Feedback into Computer-Aided Diagnosis Systems Sarah A. Jabon a Daniela S. Raicu b Jacob D. Furst b a Rose-Hulman Institute](https://reader035.vdocuments.us/reader035/viewer/2022070306/551737f6550346fe558b5ff8/html5/thumbnails/1.jpg)
Integration of Radiologists’ Feedback into Computer-Aided Diagnosis Systems
Sarah A. Jabona
Daniela S. Raicub
Jacob D. Furstb
aRose-Hulman Institute of Technology, Terre Haute, IN 47803bSchool of Computing, CDM, DePaul Universtiy, Chicago, IL 60604
![Page 2: Integration of Radiologists Feedback into Computer-Aided Diagnosis Systems Sarah A. Jabon a Daniela S. Raicu b Jacob D. Furst b a Rose-Hulman Institute](https://reader035.vdocuments.us/reader035/viewer/2022070306/551737f6550346fe558b5ff8/html5/thumbnails/2.jpg)
Overview• Introduction• Related Work• The Data• Methodology
▫ Simple Distance Metrics▫ Linear Regression▫ Principle Component Analysis
• Results▫ Simple Distance Metrics▫ Linear Regression▫ Principle Component Analysis
• Conclusions• Future Work
![Page 3: Integration of Radiologists Feedback into Computer-Aided Diagnosis Systems Sarah A. Jabon a Daniela S. Raicu b Jacob D. Furst b a Rose-Hulman Institute](https://reader035.vdocuments.us/reader035/viewer/2022070306/551737f6550346fe558b5ff8/html5/thumbnails/3.jpg)
Introduction
•The 2008 official estimate▫ 215,020 cases diagnosed▫161,840 deaths will occur
•Five-year relative-survival rate (1996 – 2004): 15.2%
•Computer-aided diagnosis systems can help improve early detection
![Page 4: Integration of Radiologists Feedback into Computer-Aided Diagnosis Systems Sarah A. Jabon a Daniela S. Raicu b Jacob D. Furst b a Rose-Hulman Institute](https://reader035.vdocuments.us/reader035/viewer/2022070306/551737f6550346fe558b5ff8/html5/thumbnails/4.jpg)
Related Work• El-Naqa et al.
▫ mammography images▫neural networks and support vector machines
• Muramatsu et al.▫mammography images. ▫three-layered artificial neural network to
predict the semantic similarity rating between two nodules
• Park et al.▫linear distance-weighted K-nearest neighbor
algorithm to identify similar images
![Page 5: Integration of Radiologists Feedback into Computer-Aided Diagnosis Systems Sarah A. Jabon a Daniela S. Raicu b Jacob D. Furst b a Rose-Hulman Institute](https://reader035.vdocuments.us/reader035/viewer/2022070306/551737f6550346fe558b5ff8/html5/thumbnails/5.jpg)
Related Work
•ASSERT by Purdue University▫Content-based features: co-occurrence,
shape, Fourier Transforms, global gray level statistics
▫Radiologists also provide features•BiasMap by Zhou and Huang
▫Relevance feedback, content-based features
▫Analysis: biased-discriminant analysis (BDA)
![Page 6: Integration of Radiologists Feedback into Computer-Aided Diagnosis Systems Sarah A. Jabon a Daniela S. Raicu b Jacob D. Furst b a Rose-Hulman Institute](https://reader035.vdocuments.us/reader035/viewer/2022070306/551737f6550346fe558b5ff8/html5/thumbnails/6.jpg)
The Data
• Lung Image Database Consortium
• Reduced 1,989 images down to 149 (one for each nodule)
• Summarized the radiologists’ ratings (up to 4) into a single vector
• Each nodule has 7 semantic based characteristics and 64 content-based characteristics
![Page 7: Integration of Radiologists Feedback into Computer-Aided Diagnosis Systems Sarah A. Jabon a Daniela S. Raicu b Jacob D. Furst b a Rose-Hulman Institute](https://reader035.vdocuments.us/reader035/viewer/2022070306/551737f6550346fe558b5ff8/html5/thumbnails/7.jpg)
Overview• Introduction• Related Work• The Data• Methodology
▫ Simple Distance Metrics▫ Linear Regression▫ Principle Component Analysis
• Results▫ Simple Distance Metrics▫ Linear Regression▫ Principle Component Analysis
• Conclusions• Future Work
![Page 8: Integration of Radiologists Feedback into Computer-Aided Diagnosis Systems Sarah A. Jabon a Daniela S. Raicu b Jacob D. Furst b a Rose-Hulman Institute](https://reader035.vdocuments.us/reader035/viewer/2022070306/551737f6550346fe558b5ff8/html5/thumbnails/8.jpg)
Methodology
![Page 9: Integration of Radiologists Feedback into Computer-Aided Diagnosis Systems Sarah A. Jabon a Daniela S. Raicu b Jacob D. Furst b a Rose-Hulman Institute](https://reader035.vdocuments.us/reader035/viewer/2022070306/551737f6550346fe558b5ff8/html5/thumbnails/9.jpg)
Methodology: Simple Distance Metrics
Semantic-Based Similarity
Content-Based Similarity
![Page 10: Integration of Radiologists Feedback into Computer-Aided Diagnosis Systems Sarah A. Jabon a Daniela S. Raicu b Jacob D. Furst b a Rose-Hulman Institute](https://reader035.vdocuments.us/reader035/viewer/2022070306/551737f6550346fe558b5ff8/html5/thumbnails/10.jpg)
Simple Distance MetricsContent-Based Similarity Values
(Euclidean)Semantic-Based Similarity
Values (1 – Cosine)
1.0000000.8000000.6000000.4000000.2000000.000000
VAR00001
600
400
200
0
Fre
qu
ency
Mean =0.2840127Std. Dev. =0.154278896N =11,026
0.400.200.00
VAR00002
1,200
1,000
800
600
400
200
0
Fre
qu
ency
Mean =0.0766Std. Dev. =0.06374
N =11,026
![Page 11: Integration of Radiologists Feedback into Computer-Aided Diagnosis Systems Sarah A. Jabon a Daniela S. Raicu b Jacob D. Furst b a Rose-Hulman Institute](https://reader035.vdocuments.us/reader035/viewer/2022070306/551737f6550346fe558b5ff8/html5/thumbnails/11.jpg)
Methodology: Linear Regression
![Page 12: Integration of Radiologists Feedback into Computer-Aided Diagnosis Systems Sarah A. Jabon a Daniela S. Raicu b Jacob D. Furst b a Rose-Hulman Institute](https://reader035.vdocuments.us/reader035/viewer/2022070306/551737f6550346fe558b5ff8/html5/thumbnails/12.jpg)
Methodology: Principle Component Analysis
Lobulation Malignancy Margin Sphericity Spiculation Subtlety Texture
Lobulation 1.000 .199 .085 -.008 .815 .065 .101
Malignancy .199 1.000 .346 .187 .155 .594 .351
Margin .085 .346 1.000 .391 .109 .533 .717
Sphericity -.008 .187 .391 1.000 .078 .300 .230
Spiculation .815 .155 .109 .078 1.000 .156 .146
Subtlety .065 .594 .533 .300 .156 1.000 .523
Texture .101 .351 .717 .230 .146 .523 1.000
Content-Based Features:
• 77 pairs with a correlation > 0.9• 136 pairs with a correlation > 0.8 or < -0.8
![Page 13: Integration of Radiologists Feedback into Computer-Aided Diagnosis Systems Sarah A. Jabon a Daniela S. Raicu b Jacob D. Furst b a Rose-Hulman Institute](https://reader035.vdocuments.us/reader035/viewer/2022070306/551737f6550346fe558b5ff8/html5/thumbnails/13.jpg)
Scree Plots: 5 – 9 Matches
7654321
Component Number
3.0
2.5
2.0
1.5
1.0
0.5
0.0
Eig
env
alu
e
Scree Plot
63
61
59
57
55
53
51
49
47
45
43
41
39
37
35
33
31
29
27
25
23
21
19
17
15
13
11
97531
Component Number
20
15
10
5
0
Eig
env
alu
e
Scree Plot
![Page 14: Integration of Radiologists Feedback into Computer-Aided Diagnosis Systems Sarah A. Jabon a Daniela S. Raicu b Jacob D. Furst b a Rose-Hulman Institute](https://reader035.vdocuments.us/reader035/viewer/2022070306/551737f6550346fe558b5ff8/html5/thumbnails/14.jpg)
Methodology: Principle Component Analysis
•PCA on content-based features▫accounts for 99% of the variance▫23 components
•PCA on semantic-based characteristics▫Method 1
accounts for 92% of the variance 4 components
▫Method 2 accounts for 98% of the variance 6 components
![Page 15: Integration of Radiologists Feedback into Computer-Aided Diagnosis Systems Sarah A. Jabon a Daniela S. Raicu b Jacob D. Furst b a Rose-Hulman Institute](https://reader035.vdocuments.us/reader035/viewer/2022070306/551737f6550346fe558b5ff8/html5/thumbnails/15.jpg)
Overview• Introduction• Related Work• The Data• Methodology
▫ Simple Distance Metrics▫ Linear Regression▫ Principle Component Analysis
• Results▫ Simple Distance Metrics▫ Linear Regression▫ Principle Component Analysis
• Conclusions• Future Work
![Page 16: Integration of Radiologists Feedback into Computer-Aided Diagnosis Systems Sarah A. Jabon a Daniela S. Raicu b Jacob D. Furst b a Rose-Hulman Institute](https://reader035.vdocuments.us/reader035/viewer/2022070306/551737f6550346fe558b5ff8/html5/thumbnails/16.jpg)
Results: Simple Distance Metric
Matches
Gabor MarkovCo-
Occurrence
Gabor, Markov, and
Co-Occurrence
All Features
6 – 10 24 18 31 36 432 – 5 107 104 94 98 930 – 1 18 27 24 15 13
![Page 17: Integration of Radiologists Feedback into Computer-Aided Diagnosis Systems Sarah A. Jabon a Daniela S. Raicu b Jacob D. Furst b a Rose-Hulman Institute](https://reader035.vdocuments.us/reader035/viewer/2022070306/551737f6550346fe558b5ff8/html5/thumbnails/17.jpg)
Matches: Nodule 117
![Page 18: Integration of Radiologists Feedback into Computer-Aided Diagnosis Systems Sarah A. Jabon a Daniela S. Raicu b Jacob D. Furst b a Rose-Hulman Institute](https://reader035.vdocuments.us/reader035/viewer/2022070306/551737f6550346fe558b5ff8/html5/thumbnails/18.jpg)
Simple Distance Metrics
![Page 19: Integration of Radiologists Feedback into Computer-Aided Diagnosis Systems Sarah A. Jabon a Daniela S. Raicu b Jacob D. Furst b a Rose-Hulman Institute](https://reader035.vdocuments.us/reader035/viewer/2022070306/551737f6550346fe558b5ff8/html5/thumbnails/19.jpg)
5 – 9 Matches: PCA and Linear Regression
![Page 20: Integration of Radiologists Feedback into Computer-Aided Diagnosis Systems Sarah A. Jabon a Daniela S. Raicu b Jacob D. Furst b a Rose-Hulman Institute](https://reader035.vdocuments.us/reader035/viewer/2022070306/551737f6550346fe558b5ff8/html5/thumbnails/20.jpg)
Results: Linear Regression
Data Set
No. of Nodule
Pairs (≈ 2/3 Set)
Correlation: Euclidean
vs. Semantic
R2 Adj. R2 Feature Set
Distance
6 – 9 Matches
166 -0.016 0.948 0.871 2 -
6 – 9 Matches
166 -0.016 0.802 0.679 1 dist3
5 – 9 Matches
218 -0.006 0.927 0.850 2 -
5 – 9 Matches
218 -0.006 0.733 0.624 1 dist3
![Page 21: Integration of Radiologists Feedback into Computer-Aided Diagnosis Systems Sarah A. Jabon a Daniela S. Raicu b Jacob D. Furst b a Rose-Hulman Institute](https://reader035.vdocuments.us/reader035/viewer/2022070306/551737f6550346fe558b5ff8/html5/thumbnails/21.jpg)
Results: Linear Regression
Data Set
No. of Nodule
Pairs (≈1/3 Set)
Correlation: Euclidean
vs. Semantic
RMSD Euclidea
n
Correlation: Predicted
vs. Semantic
RMSD Predicted
Features
6 – 9 Matches
85 -0.023 0.2328 0.710 0.0242 128
6 – 9 Matches
85 -0.023 0.2328 0.748 0.0181 64
5 – 9 Matches
108 -0.039 0.1985 0.829 0.0136 128
5 – 9 Matches
108 -0.039 0.1985 0.733 0.0155 64
![Page 22: Integration of Radiologists Feedback into Computer-Aided Diagnosis Systems Sarah A. Jabon a Daniela S. Raicu b Jacob D. Furst b a Rose-Hulman Institute](https://reader035.vdocuments.us/reader035/viewer/2022070306/551737f6550346fe558b5ff8/html5/thumbnails/22.jpg)
Results: Linear RegressionLinear Regression versus Euclidean Distance
(5 to 9 Matches with 128 Features)
0
0.02
0.04
0.06
0.08
0.1
0.12
0.14
0.16
0.18
0.2
0 0.2 0.4 0.6 0.8
Predicted Similarity Value (Calculated with Content-Based Features)
Se
ma
nti
c S
imila
rity
Va
lue
Linear Regression
Euclidean Distance
![Page 23: Integration of Radiologists Feedback into Computer-Aided Diagnosis Systems Sarah A. Jabon a Daniela S. Raicu b Jacob D. Furst b a Rose-Hulman Institute](https://reader035.vdocuments.us/reader035/viewer/2022070306/551737f6550346fe558b5ff8/html5/thumbnails/23.jpg)
Results: Linear RegressionResidual Plot: Linear Regression versus Euclidean Distance
(5 to 9 Matches with 128 Features)
-0.7
-0.6
-0.5
-0.4
-0.3
-0.2
-0.1
0
0.1
0 0.02 0.04 0.06 0.08 0.1 0.12 0.14 0.16
Semantic Similarity Value
Err
or Linear Regression
Euclidean Distance
![Page 24: Integration of Radiologists Feedback into Computer-Aided Diagnosis Systems Sarah A. Jabon a Daniela S. Raicu b Jacob D. Furst b a Rose-Hulman Institute](https://reader035.vdocuments.us/reader035/viewer/2022070306/551737f6550346fe558b5ff8/html5/thumbnails/24.jpg)
Results: PCA
Data Set
No. of Nodule
Pairs (≈ 1/3 Set)
Correlation: Euclidean
vs. Semantic
RMSD Euclidea
n
Correlation: Predicted
vs. Semantic
RMSD Predicte
dFeatures
6 – 9 Matches
85 -0.115 0.3043 0.787 0.0061 128
6 – 9 Matches
85 -0.115 0.3043 0.393 0.0114 64
5 – 9 Matches
108 -0.094 0.2664 0.570 0.0096 128
5 – 9 Matches
108 -0.094 0.2664 0.136 0.0112 64
![Page 25: Integration of Radiologists Feedback into Computer-Aided Diagnosis Systems Sarah A. Jabon a Daniela S. Raicu b Jacob D. Furst b a Rose-Hulman Institute](https://reader035.vdocuments.us/reader035/viewer/2022070306/551737f6550346fe558b5ff8/html5/thumbnails/25.jpg)
Results: PCANo PCA versus PCA
(5 to 9 Matches with 128 Features)
0
0.02
0.04
0.06
0.08
0.1
0.12
0.14
0.16
0.18
0.2
0 0.05 0.1 0.15 0.2
Predicted Similarity Value (Calculated with Content-Based Features)
Se
ma
nti
c S
imila
rity
Va
lue
Linear Regression with No PCA
Linear Regression with PCA
![Page 26: Integration of Radiologists Feedback into Computer-Aided Diagnosis Systems Sarah A. Jabon a Daniela S. Raicu b Jacob D. Furst b a Rose-Hulman Institute](https://reader035.vdocuments.us/reader035/viewer/2022070306/551737f6550346fe558b5ff8/html5/thumbnails/26.jpg)
Results: PCAResidual Plot: No PCA versus PCA (5 to 9 Matches with 128 Features)
-0.05
-0.04
-0.03
-0.02
-0.01
0
0.01
0.02
0.03
0.04
0.05
0 0.05 0.1 0.15
Semantic Similarity Value
Err
or Linear Regression with No PCA
Linear Regression with PCA
![Page 27: Integration of Radiologists Feedback into Computer-Aided Diagnosis Systems Sarah A. Jabon a Daniela S. Raicu b Jacob D. Furst b a Rose-Hulman Institute](https://reader035.vdocuments.us/reader035/viewer/2022070306/551737f6550346fe558b5ff8/html5/thumbnails/27.jpg)
RMSD – Percent of RangeLinear Regression: No
PCALinear Regression: PCA
Data Set Features Euclidean Predicted Euclidean Predicted
6 – 9 Matches
128 23.3% 17.3% 30.4% 6.7%
6 – 9 Matches
64 23.3% 12.9% 30.4% 12.5%
5 – 9 Matches
128 19.9% 9.7% 26.6% 10.1%
5 – 9 Matches
64 19.9% 11.1% 26.6% 11.8%
![Page 28: Integration of Radiologists Feedback into Computer-Aided Diagnosis Systems Sarah A. Jabon a Daniela S. Raicu b Jacob D. Furst b a Rose-Hulman Institute](https://reader035.vdocuments.us/reader035/viewer/2022070306/551737f6550346fe558b5ff8/html5/thumbnails/28.jpg)
Example: Nodule 37 and Nodule 38
Nodule 38 Nodule 37Euclidean
Similarity ValuePCA Similarity
Value
0.549066 0.004379
Nodule Number
Lobulation Malignancy Margin Sphericity Spiculation Subtlety Texture
37 5 3 5 5 5 4 5
38 5 3 5 5 5 5 5
![Page 29: Integration of Radiologists Feedback into Computer-Aided Diagnosis Systems Sarah A. Jabon a Daniela S. Raicu b Jacob D. Furst b a Rose-Hulman Institute](https://reader035.vdocuments.us/reader035/viewer/2022070306/551737f6550346fe558b5ff8/html5/thumbnails/29.jpg)
Future Work
•Perform the analysis only nodules on which all three radiologists agree
•In order to address the small size of the data set, perform the analysis using a leave one out technique (instead of 2/3 training and 1/3 testing)
•Incorporate relevance feedback into the system
![Page 30: Integration of Radiologists Feedback into Computer-Aided Diagnosis Systems Sarah A. Jabon a Daniela S. Raicu b Jacob D. Furst b a Rose-Hulman Institute](https://reader035.vdocuments.us/reader035/viewer/2022070306/551737f6550346fe558b5ff8/html5/thumbnails/30.jpg)
Questions?