ece6780 a3 pillcam project demonstration
TRANSCRIPT
8/15/2019 ECE6780 A3 PillCam Project Demonstration
http://slidepdf.com/reader/full/ece6780-a3-pillcam-project-demonstration 1/42
PillCam Foundation and Image Retrieval
Project Report
Georgia Institute of Technology
Medical Imaging
Spring 2007
Point of Contact
Todd Stokes
Instructor
Dr. May Wang
David
McCann
Stephen
Conover
8/15/2019 ECE6780 A3 PillCam Project Demonstration
http://slidepdf.com/reader/full/ece6780-a3-pillcam-project-demonstration 2/42
Overview
Problem
Solution
Methodology Implementation
Results
8/15/2019 ECE6780 A3 PillCam Project Demonstration
http://slidepdf.com/reader/full/ece6780-a3-pillcam-project-demonstration 3/42
What is PillCam?
Wireless Capsule
Endoscopy (WCE)
Non-invasive
Low Complication Rate
Out Patient
Potentially End-to-End Good diagnosis and
first look
http://www.nch.org/about/tech.shtml
8/15/2019 ECE6780 A3 PillCam Project Demonstration
http://slidepdf.com/reader/full/ece6780-a3-pillcam-project-demonstration 4/42
Problem
Diagnosis Takes a Long Time
Where are we? What are we looking at?
Hours required to review video
10 ± 6Time for physician to write report
10 ± 3Time for nurse to begin download
30 ± 7Time for nurse to prepare patients
100 ± 14Time to read video by physician
Time (min) ± SDTask
* Bossa, et al. "Detection of Abnormal Lesions Recorded by Capsule Endoscopy: A Prospective Study
Comparing Endoscopist's and Nurse's Accuracy." Digestive and Liver Disease 38 (Aug 2006): 599-602.
8/15/2019 ECE6780 A3 PillCam Project Demonstration
http://slidepdf.com/reader/full/ece6780-a3-pillcam-project-demonstration 5/42
Solution
Provide a tool to retrieve “similar” images
from a database
Allow reviewer to find related frames for
comparison/reference
Provide general 3D model as an
alternative to frames
Reduce review time through search
8/15/2019 ECE6780 A3 PillCam Project Demonstration
http://slidepdf.com/reader/full/ece6780-a3-pillcam-project-demonstration 6/42
8/15/2019 ECE6780 A3 PillCam Project Demonstration
http://slidepdf.com/reader/full/ece6780-a3-pillcam-project-demonstration 7/42
Methodology Overview
Metrics Symmetry
Curvature
Color
3D Modeling
Ranking
8/15/2019 ECE6780 A3 PillCam Project Demonstration
http://slidepdf.com/reader/full/ece6780-a3-pillcam-project-demonstration 8/42
Methodology
Symmetry
Finding the “Center” of the Image
8/15/2019 ECE6780 A3 PillCam Project Demonstration
http://slidepdf.com/reader/full/ece6780-a3-pillcam-project-demonstration 9/42
8/15/2019 ECE6780 A3 PillCam Project Demonstration
http://slidepdf.com/reader/full/ece6780-a3-pillcam-project-demonstration 10/42
8/15/2019 ECE6780 A3 PillCam Project Demonstration
http://slidepdf.com/reader/full/ece6780-a3-pillcam-project-demonstration 11/42
Nominal Results
8/15/2019 ECE6780 A3 PillCam Project Demonstration
http://slidepdf.com/reader/full/ece6780-a3-pillcam-project-demonstration 12/42
Methodology
Curvature
Extracting the Shape of the Image
8/15/2019 ECE6780 A3 PillCam Project Demonstration
http://slidepdf.com/reader/full/ece6780-a3-pillcam-project-demonstration 13/42
Curvature Overview
Why curvature? Good metric for features
and lumen/wall views.
A B C D
8/15/2019 ECE6780 A3 PillCam Project Demonstration
http://slidepdf.com/reader/full/ece6780-a3-pillcam-project-demonstration 14/42
Process
1. Threshold
2. Extract Curvature
3. Histogram4. Compare
8/15/2019 ECE6780 A3 PillCam Project Demonstration
http://slidepdf.com/reader/full/ece6780-a3-pillcam-project-demonstration 15/42
Thresholding
Global average of intensity
Simple but effective
Sample results:( )
( )ij
ij
I Avg T
BG R I
=
++=3
1
8/15/2019 ECE6780 A3 PillCam Project Demonstration
http://slidepdf.com/reader/full/ece6780-a3-pillcam-project-demonstration 16/42
Curvature Extraction
Curvature Equations (Osher and Sethien)
Binary Image Gaussian Kappa 1/Kappa
kappaC
NormGrad NormGrad w kappa
normGrad normGrad w
NormGrad
Y Y
X X ij
ij
Y X
1
elsewhere0
)max(*75.0where1
22
22
=
∇
∇+
∇
∇=
>
=
∇+∇=
φ φ
φ φ
8/15/2019 ECE6780 A3 PillCam Project Demonstration
http://slidepdf.com/reader/full/ece6780-a3-pillcam-project-demonstration 17/42
Base Test Case
0 50 100 150 2000
50
100
150
Curvature
C o u n t
Curvature Histogram
50 100 150 200 250
50
100
150
200
250
50 100 150 200 250
50
100
150
200
250
30
40
50
60
70
80
90
100
Circle
Radius of 50
Extracted
Curvature
Histogram of
Curvature is Centered
Around 50!
8/15/2019 ECE6780 A3 PillCam Project Demonstration
http://slidepdf.com/reader/full/ece6780-a3-pillcam-project-demonstration 18/42
Process Development and Refinement I
Initial draft in MATLAB m-code
Ported to and refined in Java
Java Application for Testing:
8/15/2019 ECE6780 A3 PillCam Project Demonstration
http://slidepdf.com/reader/full/ece6780-a3-pillcam-project-demonstration 19/42
Process Development and Refinement II
Bad
Better
Best
Dilated
& Masked
Masked
8/15/2019 ECE6780 A3 PillCam Project Demonstration
http://slidepdf.com/reader/full/ece6780-a3-pillcam-project-demonstration 20/42
8/15/2019 ECE6780 A3 PillCam Project Demonstration
http://slidepdf.com/reader/full/ece6780-a3-pillcam-project-demonstration 21/42
Comparison
Sum of the squares difference in
histogramed values 0:10:1000:
Must be normalized afterwards:
( )∑= −= N
n
B A n Binn Bin1
2][][ScoreComparison
( ) )(Score
Min(Score)-ScoreScoreComparison Normalized
ScoreMinMax −=
8/15/2019 ECE6780 A3 PillCam Project Demonstration
http://slidepdf.com/reader/full/ece6780-a3-pillcam-project-demonstration 22/42
Methodology
Color
8/15/2019 ECE6780 A3 PillCam Project Demonstration
http://slidepdf.com/reader/full/ece6780-a3-pillcam-project-demonstration 23/42
Overview
HSI Histogram
Sum Power Compare
8/15/2019 ECE6780 A3 PillCam Project Demonstration
http://slidepdf.com/reader/full/ece6780-a3-pillcam-project-demonstration 24/42
Methodology
Final Scoring ∑=
=Metrics
k
ii sw#
1
ScoreFinal
8/15/2019 ECE6780 A3 PillCam Project Demonstration
http://slidepdf.com/reader/full/ece6780-a3-pillcam-project-demonstration 25/42
Methodology
3D Modeling
8/15/2019 ECE6780 A3 PillCam Project Demonstration
http://slidepdf.com/reader/full/ece6780-a3-pillcam-project-demonstration 26/42
Overview
3D OBJ Model interpolated from MRI slices
Basic lighting and flat color texturing
8/15/2019 ECE6780 A3 PillCam Project Demonstration
http://slidepdf.com/reader/full/ece6780-a3-pillcam-project-demonstration 27/42
Smoothed Surface Normals
The surface normal at
a given vertex is the
average of the
surface normals of thepolygons meeting at
the vertex.
8/15/2019 ECE6780 A3 PillCam Project Demonstration
http://slidepdf.com/reader/full/ece6780-a3-pillcam-project-demonstration 28/42
Methodology
Summary
Final Rationale: How does all this work together?
8/15/2019 ECE6780 A3 PillCam Project Demonstration
http://slidepdf.com/reader/full/ece6780-a3-pillcam-project-demonstration 29/42
Implementation
8/15/2019 ECE6780 A3 PillCam Project Demonstration
http://slidepdf.com/reader/full/ece6780-a3-pillcam-project-demonstration 30/42
Implementation Overview
Completely Java
OpenGL and JOGL
30 files 3,000 SLOC
8/15/2019 ECE6780 A3 PillCam Project Demonstration
http://slidepdf.com/reader/full/ece6780-a3-pillcam-project-demonstration 31/42
Class Diagram - Model
8/15/2019 ECE6780 A3 PillCam Project Demonstration
http://slidepdf.com/reader/full/ece6780-a3-pillcam-project-demonstration 32/42
Modules
Curvature
Symmetry
Color Sum of
Squares
Distance
Score AllPresent
Results
Single
Image Load
3D ModelNavigation
Tracks
Rendering
N or m al i z e
Source
Selection
Metric
Extraction
Comparison Results
8/15/2019 ECE6780 A3 PillCam Project Demonstration
http://slidepdf.com/reader/full/ece6780-a3-pillcam-project-demonstration 33/42
Features
Easy to use graphical user interface Navigate manually through model with AWSD and
mouse Record camera positions to a track
Save and load camera tracks Quickly scan through loaded camera track Add images and directories of images to database Compare database images to 3D model or image file
Quick link back to results Save/Load feature database functionality
8/15/2019 ECE6780 A3 PillCam Project Demonstration
http://slidepdf.com/reader/full/ece6780-a3-pillcam-project-demonstration 34/42
Weaknesses
Thresholding Simple
Simple Ranking
Need higher fidelity model
Slow feature extraction
8/15/2019 ECE6780 A3 PillCam Project Demonstration
http://slidepdf.com/reader/full/ece6780-a3-pillcam-project-demonstration 35/42
Results
8/15/2019 ECE6780 A3 PillCam Project Demonstration
http://slidepdf.com/reader/full/ece6780-a3-pillcam-project-demonstration 36/42
Interface
8/15/2019 ECE6780 A3 PillCam Project Demonstration
http://slidepdf.com/reader/full/ece6780-a3-pillcam-project-demonstration 37/42
Results
VideoFrame
Independent
Image
Model
Frame
Bad Frame
Bad Results
8/15/2019 ECE6780 A3 PillCam Project Demonstration
http://slidepdf.com/reader/full/ece6780-a3-pillcam-project-demonstration 38/42
Results I: Search for Image from Video
1 2
3 4
8/15/2019 ECE6780 A3 PillCam Project Demonstration
http://slidepdf.com/reader/full/ece6780-a3-pillcam-project-demonstration 39/42
Results II: Search for Image from Independent Frame
1 2
3 4
8/15/2019 ECE6780 A3 PillCam Project Demonstration
http://slidepdf.com/reader/full/ece6780-a3-pillcam-project-demonstration 40/42
Results III: Search for Image from 3D Model
1 2
3 4
5
8/15/2019 ECE6780 A3 PillCam Project Demonstration
http://slidepdf.com/reader/full/ece6780-a3-pillcam-project-demonstration 41/42
Results IV: Bad Results
1 2
3 4
8/15/2019 ECE6780 A3 PillCam Project Demonstration
http://slidepdf.com/reader/full/ece6780-a3-pillcam-project-demonstration 42/42
Summary
Metrics perform well
Need deformable model
Review time is improved
Questions?