feature based opacity function for liver...
TRANSCRIPT
Shape prior integration in discrete
optimization segmentation
algorithms M. Freiman
Computational Radiology Lab,
Children’s Hospital, Harvard Medical School, Boston, MA.
Email: [email protected]
Shape prior integration in discrete
optimization segmentation
algorithms This research was done at the:
Computer Aided Surgery and Medical Image Processing Lab. School of Eng. And Computer Science, The Hebrew University of Jerusalem,
Israel
Website: http://www.cs.huji.ac.il/~caslab/site
Outline:
Introduction
Local shape constraint graph min-cut for vascular
lumen segmentation
Latent parametric shape constraint graph min-cut for
Aortic Arch Aneurysm (AAA) thrombus
segmentation
Latent non-parametric shape constraint graph min-
cut for kidney segmentation
Related work
Introduction
Discrete segmentation
Segmentation: A labeling map that classify each
voxel to its class
The classification problem can treat each voxel
independently (thresholding etc.) or as a Markov
Random Field (MRF, dependencies between
neighboring voxels)
Relaxation: We will discuss only 2 classes MRF
problems, although the presented solution are
extendable to problems with more than 2 classes
Discrete segmentation
Maximum A Posteriori Estimation of Labeling map
(M) given an observed image (I) is defined as:
where
Discrete segmentation
: The likelihood term, represents the
likelihood of the observed information at voxel x
given its label m(x)
: The spatial regularization term,
penalize for assigning different labels to neighboring
voxels
Discrete segmentation
The solution can be found by minimizing the
negative log of this energy:
Discrete segmentation In case of binary problems:
Illustration from Boykov et al, 2001
The optimal solution can be obtained by the graph
min-cut technique in polynomial time, where edge
weights are representing the model probabilities,
(Boykov et al, 1999,2001).
Intensity based probabilities
• Boykov et al framework used only intensity information to
compute the MRF probabilities
× Not always sufficient to separate between objects in medical
images
× Does not include any object shape information
× Estimation of the prior intensity model is usually obtained by
having the user delineate foreground and background regions
× Energy function is biased to convex shapes, which is
inappropriate for segmenting elongated objects with
bifurcations such as vascular structures
Incorporation of fixed shape priors
into the graph min-cut framework
1. “Graph cut segmentation using an elliptical shape prior”, Slabaugh & Unal, ICIP
2005.
2. “Interactive Graph Cut Based Segmentation With Shape Priors”, Freedman &
Zhang, CVPR 2005.
Incorporation of shape priors into the
graph min-cut framework
3. “OBJ-CUT”, Kumar, Torr & Zisserman, CVPR 2005.
4. “Graph cut segmentation with non linear shape prior”,
Malcolm, Rathi & Tannenbaum, ICIP 2007.
Local shape constraint graph min-
cut for vascular lumen
segmentation (Freiman et al,
3DPH 2009)
Shape constrained graph-cut based
segmentation
Global minimization of a shape constrained discrete
energy model:
• Both the likelihood and the regularization terms depend on the
shape model.
• Shape prior is obtained using a local shape descriptor
Local tubular shape descriptor (Frangi 98)
Local tubular shape descriptor (Frangi 98)
Asymmetric adaptive regularization
weights
„boundary‟ based regularization
Encourage labeling map to include voxels nearby high
vesselness response to be included in the object class
Less sensitive to intensity variability inside the vessel
σ is linearly depend on the vesselness shape term
Energy sub-modularity
Energy must be sub-modular to allow polynomial
optimization with the graph-cut framework
is non-negative, therefore:
Effect of intensity and shape terms on
carotid bifurcation segmentation
Carotid arteries segmentation
results (3D)
Carotid arteries segmentation
results
(2D views)
(a) Severe stenosis (b) Dental implants
artifacts
Carotid arteriessegmentation
results
(2D views)
(c) Vertebral
arteries
(d) Coronal view
Interactive refinement
1. Given two seed points
2. Compute the shortest-path on
the image graph, based on
local and global edge weights
3. Estimate vessel radius near the seed
points and define the possible
region for vessel surface
5. Compute optimal cut – based on
smoothing and gradient terms
4. Estimate vessel intensity model,
based on the computed path
Final results
Latent parametric shape constraint
graph min-cut for Aortic Arch
Aneurysm (AAA) thrombus
segmentation
(Freiman et al, ISBI’10)
A close look at the anatomy…
1) Aortic lumen
2) Aortic thrombus
3) Inferior Vena Cava (IVC)
4) Right psoas muscle
5) Left psoas muscle
6) Vertebrae
7) The small bowel
1
2 3
4 5 6
7
Abdominal Aortic Aneurysm (AAA)
lumen segmentation
Lumen segmentation using our method:
“Nearly automatic vessels segmentation using
graph-based energy minimization”.
Proc. 3D Segmentation in the Clinic: A Grand
Challenge III, Carotid bifurcation evaluation,
MICCAI 2009 workshop.
Intensity information is not sufficient
for thrombus segmentation
Abdominal Aortic Aneurysm
thrombosis segmentation
Challenge: No explicit model for the thrombosis
Discrete energy minimization using the Expectation-
Maximization approach
Solution: Treat the shape constraint as a latent variable
Optimization scheme
Loop until convergence:
E-step:
Estimation of both intensity and shape
parametric models.
M-step:
Graph min-cut segmentation, using the
assumed shape and intensity models.
End loop.
Latent parametric shape model
Thrombosis can be modeled as a set of axial ellipsoids
First iteration: prior intensity model
without shape constraint
Fixed prior intensity
model
No shape constraint
Optimization is limited
to a predefined fixed
radius around the lumen
Robust ellipsoid fitting
1. Collect a set of points P on the
segmentation surface
4. Fit a 2D parametric ellipsoid to
the selected points using
Taubin‟s least-squares method
(IEEE TPAMI, 1991)
2. Compute the distance from each
point pi to the estimated ellipsoid
surface
3. Select the N closest points to
current estimated ellipsoid
EM optimization: E-step
For each slice – ellipsoid is fitted using the proposed
method
3D model is reconstructed by collecting the 2D
ellipsoids
Distance map is used to represent the shape model
EM optimization: M-step
Voxel to terminal nodes edges:
• Intensity term: based on the previous iteration thrombosis
region intensity PDF. Background probability is considered
as: 1-foreground.
• Shape term: voxel‟s probability to belong to the thrombosis,
based on the ellipsoids model
Voxel to neighbor voxels edges:
• Intensity term: based on voxels contrast
• Shape term: spatial probability of the thrombosis surface,
based on the ellipsoids model
Segmentation results
Green contour: ground truth
Red contour: our result (includes the lumen)
Segmentation results
Green contour: ground truth
Red contour: our result (includes the lumen)
Latent non-parametric shape
constraint graph min-cut for kidney
segmentation
(Freiman et al, MICCAI 2010)
Kidney anatomy
1) Left kidney
2) Right kidney
3) Liver
4) Vertebrae
1 2
3
4
Main challenge:
Separation between
the kidney
surrounding tissue
such as the liver,
muscles, and
spleen
Kidney segmentation: Intensity
based graph-cuts 1) Shim, H., Chang, S., Tao, C., Wang, J.H., Kaya, D. and Bae,
K.T. Semiautomated Segmentation of Kidney From High-
Resolution Multidetector Computed Tomography Images
Using a Graph-Cuts Technique. J Comput Assist Tomogr, 33:
893-901, 2009.
Non parametric latent shape prior
Non parametric shape prior:
Set of Kidney CT volumes, with annotated kidneys
A common coordinate system is not required
No parameterization of the inter-patient shape variability
Required multiple registrations during the segmentation
process
Required multiple registrations during the segmentation
process – accelerated using parallel computing
EM based energy minimization
(1)
EM based energy minimization
(2)
E-step: model estimation First iteration:
The new CT volume is registered using B-Spline registration to each
one of the atlas‟ CT volumes
The kidney region is a weighted average of the projected annotations
from the atlas‟ datasets, to the new volume. Weights represent the
fidelity between the grayscale images
Intensity model is computed based on weighted histogramming of the
assumed kidney region
Subsequent iterations:
The binary result from previous iteration is used for intensity model
computation
The kidney region is a weighted average of the projected annotations
from the atlas‟ datasets, to the new volume. The weights represent the
fidelity to current segmentation
M-step: Graph min-cut optimization
Voxel to terminal nodes edges:
Intensity term:
Foreground: based on the kidney region intensity PDF
(computed from the kidney region histogram)
Background probability is considered as: 1-foreground.
Shape term: Voxel‟s probability to belong to the kidney, based
on the atlas model:
M-step: Graph min-cut optimization
Voxel to neighbor voxels edges:
Intensity term: based on voxels contrast
Shape term: spatial probability of the kidney surface, based
on the atlas model.
More sensitive to contrast changes on the expected object
boundary
Examples
Results
Conclusions
1. A local shape constraint graph min-cut approach for
vascular lumen segmentation.
2. A global parametric shape constraint approach for
AAA thrombosis segmentation.
3. General non-parametric shape constraint graph min-
cut approach for organs segmentation with
application to kidney.
Shape constraints integration in
graph structure
1. S. Vicente, V. Kolmogorov, and C. Rother, “Graph cut based image
segmentation with connectivity priors”, in CVPR 2008.
2. A. Besbes, N. Paragios, N. Komodakis, and G. Langs, "Shape Priors and
Discrete MRFs for Knowledge-based Segmentation“, In CVPR 2009.
3. C. Wang, O. Teboul, F. Michel, S. Essafi and N. Paragios, “3D
Knowledge-Based Segmentation Using Pose-Invariant Higher-Order
Graphs”, In MICCAI 2010
4. D.R. Chittajallu, S.K. Shah, and I.A. Kakadiaris, “A shape-driven MRF
model for the segmentation of organs in medical images”, In CVPR 2010
5. I. Ben Ayed, K. Punithakumar, G. Garvin, W. Romano, and S. Li, “Graph
Cuts with Invariant Object-Interaction Priors: Application to Intervertebral
Disc Segmentation ”, in IPMI 2011
Shape constraints integration in
graph structure
NP hard problems - require complex optimization schemes to
achieve approximate solutions
Enforce Discretization of the shape models
• Prof. L. Joskowicz, M. Natanzon, N. Boride, J. Frank, L.
Weizman, A. Kronman (School of Eng. and Computer
Science, The Hebrew Univ.)
• Dr. J. Sosna, S.J. Esses, P. Berman (Dept. of Radiology,
Hadassah Medical Centre).
• O. Shilon, E. Nammer (Simbionix LTD).
• This research is supported in part by MAGNETON grant 38652
from the Israeli Ministry of Trade and Industry and by the
Hoffman Hebrew Univ. Responsibility and Leadership program.
Acknowledgements
Thank you!