![Page 1: Hyperspectral Imaging to Discern Benign and Malignant Canine Mammary tumors](https://reader036.vdocuments.us/reader036/viewer/2022081512/568163a1550346895dd49f61/html5/thumbnails/1.jpg)
Hyperspectral Imaging to Discern Benign and Malignant Canine
Mammary tumors
Control Sensor Network and Perception LaboratoryElectrical and Computer Engineering DepartmentTemple UniversityPhiladelphia, PA 19122, U.S.A.
Amrita Sahu 9th May, 2013.
Dr. Chang-Hee Won (Advisory Chair)Dr. Nancy PleshkoDr. Joseph Picone
1
![Page 2: Hyperspectral Imaging to Discern Benign and Malignant Canine Mammary tumors](https://reader036.vdocuments.us/reader036/viewer/2022081512/568163a1550346895dd49f61/html5/thumbnails/2.jpg)
Outline• Motivation• Background and Literature Review• Objectives• Image Processing Methods• Experimental Setup• Characterization of System• Data Acquisition• Results• Conclusions• Future Work
2
![Page 3: Hyperspectral Imaging to Discern Benign and Malignant Canine Mammary tumors](https://reader036.vdocuments.us/reader036/viewer/2022081512/568163a1550346895dd49f61/html5/thumbnails/3.jpg)
Motivation
Using Hyperspectral Imaging (HSI) for tumor detection
• Non-invasive• Less time-consuming • Allows assessment of a large area of tissue. Applications of HSI tissue imaging:• Mammary tumors
Human Breast Cancer Canine Cancer Feline Cancer
http://www.thepetcenter.com/gen/can.html
3
![Page 4: Hyperspectral Imaging to Discern Benign and Malignant Canine Mammary tumors](https://reader036.vdocuments.us/reader036/viewer/2022081512/568163a1550346895dd49f61/html5/thumbnails/4.jpg)
Motivation• No good device to identify
malignant mammary tumors.• Doctors usually perform biopsy or
just ‘wait and watch’.• Biopsy is the gold standard for
cancer detection.• It is invasive and requires several
days for the results to be determined.
To avoid the above disadvantages, we propose to use a non-invasive hyperspectral imaging sensor for characterizing canine mammary tumors.
http://www.beliefnet.com/healthandhealing/getcontent.aspx?cid=14777
4
![Page 5: Hyperspectral Imaging to Discern Benign and Malignant Canine Mammary tumors](https://reader036.vdocuments.us/reader036/viewer/2022081512/568163a1550346895dd49f61/html5/thumbnails/5.jpg)
BackgroundHyperspectral imaging measures and collects reflectance intensity information of more than hundred spectral bands across the electromagnetic spectrum.
http://www.nature.com/nphoton/journal/v3/n11/
fig_tab/nphoton.2009.205_F3.html
5
![Page 6: Hyperspectral Imaging to Discern Benign and Malignant Canine Mammary tumors](https://reader036.vdocuments.us/reader036/viewer/2022081512/568163a1550346895dd49f61/html5/thumbnails/6.jpg)
Applications of Hyperspectral ImagingApplications of Hyperspectral Imaging are: • Agriculture• Mineralogy• Surveillance• Monitoring of Oil Drilling• Non-Invasive Tissue Analysis
http://www.markelowitz.com/Hyperspectral.html
6
![Page 7: Hyperspectral Imaging to Discern Benign and Malignant Canine Mammary tumors](https://reader036.vdocuments.us/reader036/viewer/2022081512/568163a1550346895dd49f61/html5/thumbnails/7.jpg)
Cancer Detection using Infrared Hyperspectral Imaging
• Breast Cancer• Canine Mammary Cancer• Tongue Cancer • Gastric Cancer• Skin Cancer• Other diseases: intestinal ischemia, lung emphysema
Liu, Z. et. al, “Tongue Tumor Detection in Medical Hyperspectral Images”, Sensors, 12(1), 162-174 (2012).
Balas, C., Themelis et. al “A Novel Hyper-Spectral Imaging System : Application on in-vivo Detection and Grading of Cervical Precancers and of Pigmented Skin Lesions”, In Proc. of "Computer Vision Beyond the Visible Spectrum" CVBVS'01 Workshop, Hawaii,
USA, (2001).
Lee, J., Won, C. H., “Characterization of Lung Tissues using Liquid-Crystal Tunable Filter and hyperspectral Imaging System,” Proc. IEEE EMBC 09, 1416-1419 (2009).
7
![Page 8: Hyperspectral Imaging to Discern Benign and Malignant Canine Mammary tumors](https://reader036.vdocuments.us/reader036/viewer/2022081512/568163a1550346895dd49f61/html5/thumbnails/8.jpg)
Breast Cancer• Results from 58 malignant breast tumors are reported.• A steady state spectrometer used (650-1100nm).• Six laser diodes used for illumination.• Fiber Optic cable delivers laser light to tissue.
Shah, N., A. E. Cerussi, D. Jakubowski, D. Hsiang, J. Butler, and B. J. Tromberg, The role of diffuse optical spectroscopy in the clinical
management of breast cancer. Dis. Markers 19:95–105, 2003.
8
![Page 9: Hyperspectral Imaging to Discern Benign and Malignant Canine Mammary tumors](https://reader036.vdocuments.us/reader036/viewer/2022081512/568163a1550346895dd49f61/html5/thumbnails/9.jpg)
Breast Cancer • Hemoglobin, water and lipid content is different in malignant and
benign tumors.• Tissue Optical Index (TOI) was developed.• Higher TOI indicates tumor malignancy.
Shah, N., A. E. Cerussi, D. Jakubowski, D. Hsiang, J. Butler, and B. J. Tromberg, The role of diffuse optical spectroscopy in the clinical management of breast cancer. Dis. Markers 19:95–105, 2003.
9
![Page 10: Hyperspectral Imaging to Discern Benign and Malignant Canine Mammary tumors](https://reader036.vdocuments.us/reader036/viewer/2022081512/568163a1550346895dd49f61/html5/thumbnails/10.jpg)
Canine Mammary Cancer
• Fluorescent dyes used.
• The dyes were administered in the vein of the canine patient.
• For illumination, a 660nm laser diode beam used.
• The uptake and release rates of the dye varied in the diseased
and normal tissue.
M. Gurfinkel et al, Pharmacokinetics of ICG and HPPH-car for the Detection of Normal and Tumor Tissue Using Fluorescence, Near-Infrared Reflectance Imaging: A Case Study, Photochemistry and Photobiology, 2000, 72(1), 94-102
10
![Page 11: Hyperspectral Imaging to Discern Benign and Malignant Canine Mammary tumors](https://reader036.vdocuments.us/reader036/viewer/2022081512/568163a1550346895dd49f61/html5/thumbnails/11.jpg)
Objectives• Characterize a hyperspectral imaging system and use it to
discern malignant and benign canine mammary tumors.
• Normalize and preprocess the spectral data.
• Develop algorithms to discern malignant tumors and benign canine mammary tumors.
• Design an experiment to acquire the clinical data and analyze the results.
11
![Page 12: Hyperspectral Imaging to Discern Benign and Malignant Canine Mammary tumors](https://reader036.vdocuments.us/reader036/viewer/2022081512/568163a1550346895dd49f61/html5/thumbnails/12.jpg)
Why is Near Infrared Spectral Range Used?• Near Infrared Hyperspectral Imaging has been used in
literature for the detection of various kinds of cancer.
• NIR light has good penetration depth into tissue, because tissue has low absorptivity in this region.
• NIR light is absorbed by certain chromophores in the tissue that are biochemically significant.
• In this thesis, we use the visible - NIR spectral range, 650 nm to 1100 nm.
12
![Page 13: Hyperspectral Imaging to Discern Benign and Malignant Canine Mammary tumors](https://reader036.vdocuments.us/reader036/viewer/2022081512/568163a1550346895dd49f61/html5/thumbnails/13.jpg)
Methods1. Image Preprocessing To improve the signal to noise ratio. Savitzky-Golay Smoothing process used. It performs local polynomial regression using method of least
squares.
2. Image Normalization Data should be normalized to treat spectral non-uniformity of
device. Raw data may change due to illumination, temperature and non-
uniform contour of the subject. Range normalization used. In range normalization, each spectral row is divided by its range
(max value - min value).13
![Page 14: Hyperspectral Imaging to Discern Benign and Malignant Canine Mammary tumors](https://reader036.vdocuments.us/reader036/viewer/2022081512/568163a1550346895dd49f61/html5/thumbnails/14.jpg)
Methods3. Identification of chromophore-specific wavelengths
Second derivative method applied to the reflectance spectra. Negative peaks in the second derivative spectra would give the wavelengths corresponding to the chromophores.
4. Algorithm to detect malignancy
Algorithm Literature Canine CancerSupport Vector Machine Most widely used in
detection of prostate, gastric cancer.
Does not work well
PCA-LDA Also used in some kinds of cancers.
Sensitivity and Specificity 86 % and 86 %
Tissue Optical Index (TOI) Used in breast cancer Sensitivity and Specificity 86 % and 95%
14
![Page 15: Hyperspectral Imaging to Discern Benign and Malignant Canine Mammary tumors](https://reader036.vdocuments.us/reader036/viewer/2022081512/568163a1550346895dd49f61/html5/thumbnails/15.jpg)
Tissue Optical Indices Method
2
2
[ ][ ][ ][ ]H O HbTTOILipid StO
2 210 10
10 10 10
log (1 )log (1 ( ))log (1 )(log (1 )) log (1 ( ))
H O H O Hb
lipid HbO HbO HbO
R R RTOI
R R R R
[ ]A c l
[ ]A c
101logAR
• A is the absorption of NIR light,
• ε is the molar extinction coefficient (mol/litre/cm)
• [c] is the concentration of chromophore (mol/litre)
• l is the photon path length (cm)
15
![Page 16: Hyperspectral Imaging to Discern Benign and Malignant Canine Mammary tumors](https://reader036.vdocuments.us/reader036/viewer/2022081512/568163a1550346895dd49f61/html5/thumbnails/16.jpg)
Tissue Optical Indices Method• Higher content of hemoglobin (HbT) suggests elevated blood
volume and angiogenesis.• Higher water content (H2O) suggest edema and increased
cellularity• Decreased StO2 (tissue oxygen saturation) indicated tissue
hypoxia driven by metabolically active tumor cells• Decreased lipid content reflects displacement of parenchymal
adipose• A higher TOI suggest that the tumor is malignant, because it
indicates higher metabolic activity of the cells.
16
2
2
[ ][ ][ ][ ]H O HbTTOILipid StO
![Page 17: Hyperspectral Imaging to Discern Benign and Malignant Canine Mammary tumors](https://reader036.vdocuments.us/reader036/viewer/2022081512/568163a1550346895dd49f61/html5/thumbnails/17.jpg)
PCA-LDA Method
• Converts a larger number of correlated variables into a smaller number of linearly uncorrelated variables called principal components (PC).
• The first principal component has the highest variance, the second principal component has the second highest variance and so on.
• Each PC is orthogonal to each other.
Principal Component Analysis
http://cnx.org/content/m11461/latest/
17
![Page 18: Hyperspectral Imaging to Discern Benign and Malignant Canine Mammary tumors](https://reader036.vdocuments.us/reader036/viewer/2022081512/568163a1550346895dd49f61/html5/thumbnails/18.jpg)
PCA-LDA MethodLinear Discriminant Analysis
• Linear Discriminant Analysis is widely used in statistics, machine learning and pattern recognition.
• It finds a linear combination of features which characterized two or more classes of objects.
• It used Bayes’ Formula, and we assume that the prior probabilities for groups are given.
18
![Page 19: Hyperspectral Imaging to Discern Benign and Malignant Canine Mammary tumors](https://reader036.vdocuments.us/reader036/viewer/2022081512/568163a1550346895dd49f61/html5/thumbnails/19.jpg)
PCA-LDA Method
19
![Page 20: Hyperspectral Imaging to Discern Benign and Malignant Canine Mammary tumors](https://reader036.vdocuments.us/reader036/viewer/2022081512/568163a1550346895dd49f61/html5/thumbnails/20.jpg)
Hyperspectral Imaging System Description
The imaging system consists of:
• A digital imager (CCD camera, 1.4 megapixel, 12 bit output).
• A Liquid Crystal Tunable Filter.
• LCTF Controller.
• 500W dual quartz tungsten halogen lamps (360-2500nm) were used for illumination.
20
![Page 21: Hyperspectral Imaging to Discern Benign and Malignant Canine Mammary tumors](https://reader036.vdocuments.us/reader036/viewer/2022081512/568163a1550346895dd49f61/html5/thumbnails/21.jpg)
Characterization of HSI System
• Experiment 1 To test the repeatability of the HSI system
• Experiment 2 Depth of penetration of NIR light into chicken breast tissue.
• Experiment 3 Effect of camera-to-sample distance on the reflectance intensity spectra.
21
![Page 22: Hyperspectral Imaging to Discern Benign and Malignant Canine Mammary tumors](https://reader036.vdocuments.us/reader036/viewer/2022081512/568163a1550346895dd49f61/html5/thumbnails/22.jpg)
Repeatability Experiment
• Hyperspectral image of a neoprene rubber sheet is captured for 5 consecutive days.
• The ambient temperature and humidity are recorded.
• External conditions such as lighting were kept as similar as possible.
• The reflectance spectra were compared.
22
![Page 23: Hyperspectral Imaging to Discern Benign and Malignant Canine Mammary tumors](https://reader036.vdocuments.us/reader036/viewer/2022081512/568163a1550346895dd49f61/html5/thumbnails/23.jpg)
Results
0
2
4
6
8
10
12
14
16
650
670
690
710
730
750
770
790
810
830
850
870
890
910
930
950
970
990
1010
1030
1050
1070
1090
Refle
ctan
ce In
tens
ity (a
.u.)
Wavelength in nm
Day1
Day2
Day3
Day4
Day5
Day Temperature Humidity Weather conditions
1 22.9 ˚C 28% Cloudy
2 22.7 ˚C 29% Cloudy
3 23.6 ˚C 26% Sunny
4 23.8 ˚C 25% Sunny
5 24.1˚C 25% Sunny
System found to be fairly repeatable.
23
![Page 24: Hyperspectral Imaging to Discern Benign and Malignant Canine Mammary tumors](https://reader036.vdocuments.us/reader036/viewer/2022081512/568163a1550346895dd49f61/html5/thumbnails/24.jpg)
Depth of Penetration Experiment
• Chicken breast tissue was cut into sections of varying thickness.
• Neoprene rubber sheet was placed under the chicken slice.
• Quantify at what minimum width of the chicken slice the spectral effect of neoprene rubber is obtained.
24
![Page 25: Hyperspectral Imaging to Discern Benign and Malignant Canine Mammary tumors](https://reader036.vdocuments.us/reader036/viewer/2022081512/568163a1550346895dd49f61/html5/thumbnails/25.jpg)
Results
0
20
40
60
80
100
120
140
650
670
690
710
730
750
770
790
810
830
850
870
890
910
930
950
970
990
1010
1030
1050
1070
1090
Refle
ctan
ce In
tens
ity (a
.u.)
Wavelength in nm
Neoprene and 1.23 mm chicken
Neoprene and 3.00 mm chicken
Neoprene and 5.00 mm chicken
Neoprene and 7.27 mm chicken
Neoprene and 10.7 mm chicken
Chicken Slice
Neoprene and 40 mm chicken
Neoprene and 60 mm chicken
Neoprene rubber sheet
The depth of penetration of NIR light into chicken breast to be between 3 mm to 5 mm
25
![Page 26: Hyperspectral Imaging to Discern Benign and Malignant Canine Mammary tumors](https://reader036.vdocuments.us/reader036/viewer/2022081512/568163a1550346895dd49f61/html5/thumbnails/26.jpg)
Effect of camera to sample distance on the output spectra.
• The distance between the sample and the camera is varied each time.• The target was a slice of chicken breast.
0
20
40
60
80
100
120
650
680
710
740
770
800
830
860
890
920
950
980
1010
1040
1070
1100
Refle
ctan
ce In
tens
ity (a
.u.)
Wavelength in nm
23 cm
26 cm
35 cm
39 cm
47 cm
The reflectance spectra is not affected by the camera to sample distance.
26
![Page 27: Hyperspectral Imaging to Discern Benign and Malignant Canine Mammary tumors](https://reader036.vdocuments.us/reader036/viewer/2022081512/568163a1550346895dd49f61/html5/thumbnails/27.jpg)
Data Acquisition From Canine Patients
The data were acquired in collaboration with the Veterinary Hospital of the University of Pennsylvania.
27
![Page 28: Hyperspectral Imaging to Discern Benign and Malignant Canine Mammary tumors](https://reader036.vdocuments.us/reader036/viewer/2022081512/568163a1550346895dd49f61/html5/thumbnails/28.jpg)
Data Acquisition From Canine Patients
Hyperpspectral image cube of a canine patient
Increasing wavelength
28
![Page 29: Hyperspectral Imaging to Discern Benign and Malignant Canine Mammary tumors](https://reader036.vdocuments.us/reader036/viewer/2022081512/568163a1550346895dd49f61/html5/thumbnails/29.jpg)
Results• Spectral data smoothed by Savitzky-Golay filtering
Smoothing applied on raw
data to minimize noise
29
![Page 30: Hyperspectral Imaging to Discern Benign and Malignant Canine Mammary tumors](https://reader036.vdocuments.us/reader036/viewer/2022081512/568163a1550346895dd49f61/html5/thumbnails/30.jpg)
Results • Smoothed spectral data from one of the canine patients shown.• Cancer tissue has relatively lower reflectance intensity compared to the
benign and the normal tissue.
0
10
20
30
40
50
60
650 700 750 800 850 900 950 1000 1050 1100
Refle
ctan
ce in
tens
ity
Wavelength in nm
R1 (benign)
L3 (malignant)
Normal tissue 1
Normal tissue 2
Normal tissue 3
Normal tissue 4
30
![Page 31: Hyperspectral Imaging to Discern Benign and Malignant Canine Mammary tumors](https://reader036.vdocuments.us/reader036/viewer/2022081512/568163a1550346895dd49f61/html5/thumbnails/31.jpg)
Results
0
0.2
0.4
0.6
0.8
1
1.2
680 690 700 710 720 730 740 750 760 770 780 790 800 810 820 830 840 850 860 870 880 890 900 910 920 930 940 950 960 970 980 990 1000 1010 1020 1030 1040 1050 1060 1070
Nor
mal
ized
refle
ctan
ce in
tens
ity (a
.u.)
Wavelength (nm)
After range normalization
Before range normalization
31
![Page 32: Hyperspectral Imaging to Discern Benign and Malignant Canine Mammary tumors](https://reader036.vdocuments.us/reader036/viewer/2022081512/568163a1550346895dd49f61/html5/thumbnails/32.jpg)
Results
Negative Peaks at 700, 840, 900 and 970 nm observed in the second derivative reflectance spectra, these peaks were attributed to deoxy-hemoglobin, oxy-hemoglobin, lipid and water respectively.
• Identifying chromophore-specific wavelength
-0.09
-0.06
-0.03
0
0.03
0.06
0.09
650 700 750 800 850 900 950 1000 1050 1100
Seco
nd D
eriv
ative
Wavelength (nm)
Malignant
Benign
Normal
Hb HbO2Lipid Water
32
![Page 33: Hyperspectral Imaging to Discern Benign and Malignant Canine Mammary tumors](https://reader036.vdocuments.us/reader036/viewer/2022081512/568163a1550346895dd49f61/html5/thumbnails/33.jpg)
Results (TOI Method)
Using a TOI threshold of 2.00 units,
• 6 out of 7 malignant tumors,
• 13 out of 15 benign tumors,
• All of 22 normal tissue ROIs were correctly identified. Sensitivity and specificity of the proposed method were 86% and 95%
respectively. 33
Sahu A., McGoverin C. et. al “Hyperspectral Imagimg to Discern Malignant and Benign Canine Mammary Tumors”, In Proc. of SPIE Defense Security Sensing 2013
![Page 34: Hyperspectral Imaging to Discern Benign and Malignant Canine Mammary tumors](https://reader036.vdocuments.us/reader036/viewer/2022081512/568163a1550346895dd49f61/html5/thumbnails/34.jpg)
Results (PCA-LDA Method)
• We have 44 Regions of Interest (ROI). Cannot construct separate training and testing dataset.
• 44-fold cross validation is used.
• Sensitivity and specificity is 86% and 86%.
34
![Page 35: Hyperspectral Imaging to Discern Benign and Malignant Canine Mammary tumors](https://reader036.vdocuments.us/reader036/viewer/2022081512/568163a1550346895dd49f61/html5/thumbnails/35.jpg)
DiscussionsType of cancer Sensitivity and specificity
Prostate 93% and 97%
Gastric 93% and 91%
Skin 90% and 75%
Tongue 93% and 91%
Canine Mammary Cancer 86% and 95% (TOI)86% and 86% (PCA-LDA)
Both TOI and PCA-LDA method works well. The TOI method has a slightly higher specificity for identifying benign tumors.
35
![Page 36: Hyperspectral Imaging to Discern Benign and Malignant Canine Mammary tumors](https://reader036.vdocuments.us/reader036/viewer/2022081512/568163a1550346895dd49f61/html5/thumbnails/36.jpg)
Dicussions• TOI Method Advantages:
Four wavelengths identified characteristic of the four chromophores. This could significantly reduce imaging time. Less-time consuming, easy to compute.
Disadvantage: Wavelength dependent
• PCA-LDA method Advantages:
More robust, takes into account all wavelength information. Cross validation applied.
Disadvantage: More time consuming than TOI method.
36
![Page 37: Hyperspectral Imaging to Discern Benign and Malignant Canine Mammary tumors](https://reader036.vdocuments.us/reader036/viewer/2022081512/568163a1550346895dd49f61/html5/thumbnails/37.jpg)
Conclusions• A hyperspectral imaging system was used to characterize malignant
and benign canine mammary tumors.
• Reflectance intensities of malignant tumors were lower than benign and normal tissue over the wavelength 650 – 1100 nm.
• Four negative peaks were observed at the wavelengths of 700, 840, 900 and 970 nm characteristic of deoxyhemoglobin, oxyhemoglobin, lipid and water.
• A ‘Tissue Optical Index’ was used to classify canine cancer.
• Preliminary results with 22 canine mammary tumors showed that the sensitivity and specificity of the TOI method was 86% and 96% respectively. 37
![Page 38: Hyperspectral Imaging to Discern Benign and Malignant Canine Mammary tumors](https://reader036.vdocuments.us/reader036/viewer/2022081512/568163a1550346895dd49f61/html5/thumbnails/38.jpg)
Conclusions• PCA-LDA method was developed to classify malignant tumors and
the model was cross validated.
• The sensitivity and specificity of the PCA-LDA method was 86% and 86% respectively.
38
![Page 39: Hyperspectral Imaging to Discern Benign and Malignant Canine Mammary tumors](https://reader036.vdocuments.us/reader036/viewer/2022081512/568163a1550346895dd49f61/html5/thumbnails/39.jpg)
Future Work
• Further work needs to be done to collect more canine spectral data to generalize an application for the predictions put forward by the current study.
• We can also conduct an in-vitro study of the canine tumor tissue after resection and compare the analysis with that of the in-vivo study.
• In future experiments, we should use the reflectance standards to normalize data.
39
![Page 40: Hyperspectral Imaging to Discern Benign and Malignant Canine Mammary tumors](https://reader036.vdocuments.us/reader036/viewer/2022081512/568163a1550346895dd49f61/html5/thumbnails/40.jpg)
Future Work• The lighting should be as uniform as possible.
• Non-uniform light will introduce variability in the data.
• The temperature of the tungsten halogen lights are very high.
• Uncomfortable for patients.
• Use of fiber optic cable can mitigate can the problem.
40
![Page 41: Hyperspectral Imaging to Discern Benign and Malignant Canine Mammary tumors](https://reader036.vdocuments.us/reader036/viewer/2022081512/568163a1550346895dd49f61/html5/thumbnails/41.jpg)
Acknowledgements
I would like to thank the following people:• Dr. Chang-Hee Won for providing me the opportunity to work on this
project and for guiding me through the project.• My Committee Members: Dr. Joseph Picone and Dr. Nancy Pleshko.• Dr. Karin Sorenmo, for providing the canine patient data.• Dr. Cushla McGoverin, for her constant help, support and
encouragement.• Dr. Won and Firdous Saleheen, for their help in canine data
acquisition.• The members of the CSNAP lab.• Amrita Sahu is supported by the University Fellowship from Temple
University Graduate School.• This work was supported in part by the Tobacco Formula Fund of
Pennsylvania Department of Health. 41
![Page 42: Hyperspectral Imaging to Discern Benign and Malignant Canine Mammary tumors](https://reader036.vdocuments.us/reader036/viewer/2022081512/568163a1550346895dd49f61/html5/thumbnails/42.jpg)
42