geometric active contours
Post on 03-Jan-2016
29 Views
Preview:
DESCRIPTION
TRANSCRIPT
Geometric Active ContoursRon Kimmel
www.cs.technion.ac.il/~ron
Computer Science Department Technion-Israel Institute of Technology
Geometric Image Processing Lab
Edge Detection Edge Detection:
The process of labeling the locations in the image where the gray level’s “rate of change” is high. OUTPUT: “edgels” locations,
direction, strength
Edge Integration: The process of combining “local” and perhaps sparse and
non-contiguous “edgel”-data into meaningful, long edge curves (or closed contours) for segmentation OUTPUT: edges/curves consistent with the local data
The Classics Edge detection:
Sobel, Prewitt, Other gradient estimators Marr Hildreth
zero crossings of Haralick/Canny/Deriche et al.
“optimal” directional local max of derivative
Edge Integration: tensor voting (Rom, Medioni, Williams, …) dynamic programming (Shashua & Ullman) generalized “grouping” processes (Lindenbaum et al.)
IG *
The “New-Wave” Snakes Geodesic Active Contours Model Driven Edge Detection
Edge Curves
“nice” curves that optimize a functional of g( ), i.e.
nice: “regularized”,
smooth, fit some prior information
curve
dsg ) (
Image
),( yxg
Edge Indicator Function
2
1
1 | ( * ) |G I
Geodesic Active Contours
Snakes Terzopoulos-Witkin-Kass 88 Linear functional efficient implementation non-geometric depends on parameterization
Open geometric scaling invariant, Fua-Leclerc 90 Non-variational geometric flow
Caselles et al. 93, Malladi et al. 93 Geometric, yet does not minimize any functional
Geodesic active contours Caselles-Kimmel-Sapiro 95 derived from geometric functional non-linear inefficient implementations:
Explicit Euler schemes limit numerical step for stability Level set method Ohta-Jansow-Karasaki 82, Osher-Sethian 88
automatically handles contour topology Fast geodesic active contours Goldenberg-Kimmel-Rivlin-
Rudzsky 99 no limitation on the time step efficient computations in a narrow band
Laplacian Active Contours Closed contours on vector fields
Non-variational models Xu-Prince 98, Paragios et al. 01 A variational model Vasilevskiy-Siddiqi 01
Laplacian active contours open/closed/robust Kimmel-Bruckstein 01
Most recent:variational measures
for good old operatorsKimmel-Bruckstein 03
Segmentation
Segmentation
Ultrasound images
Caselles,Kimmel, Sapiro ICCV’95
Segmentation
Pintos
Woodland Encounter Bev Doolittle 1985
With a good prior who needs the data…
Segmentation
Caselles,Kimmel, Sapiro ICCV’95
Prior knowledge…
Prior knowledge…
Segmentation
Segmentation
Segmentation
Caselles,Kimmel, Sapiro ICCV’95
Segmentation
With a good prior who needs the data…
Wrong Prior???
Wrong Prior???
Wrong Prior???
Curves in the Plane C(p)={x(p),y(p)}, p [0,1]
y
x
C(0)
C(0.1) C(0.2)
C(0.4)
C(0.7)
C(0.95)
C(0.9)
C(0.8)
pC =tangent
Arc-length and Curvature
s(p)= | |dp 0
p
| | 1,sC pC | |p
sp
CC
C
ssC N
1
ssC N
C
Calculus of Variations
Find C for which is an extremum
Euler-Lagrange:
1
0
( , )pE L C C dp( ) ( , , , )p p pE L x y x y dp
0
( )px x x
E dE L L dp
dp
0
0
p
p
x x
y y
dL L
dp
dL L
dp
Calculus of Variations
Important Example Euler-Lagrange: , setting Curvature flow
( )t ss ssC C C N
1
0
| |pE C dp| | 0pN C
QQQQQQQQQQQQQQ| |pds C dp
0N
Potential Functions (g)
x
I(x,y) I(x)
x
g(x)
xx
g(x,y)
Image
Edges2
1
1 | ( * ) |G I
Snakes & Geodesic Active Contours
Snake modelTerzopoulos-Witkin-Kass 88
Euler Lagrange as a gradient descent
Geodesic active contour modelCaselles-Kimmel-Sapiro 95
Euler Lagrange gradient descent
CgCCdt
dCpppppp
0arg min
L C
Cg C ds
NΝg(C),κCgdt
dC
1
0
22 )(2minarg dpCgCC ppp
C
Maupertuis Principle of Least Action
Snake = Geodesic active contourup to some , i.e Snakes depend on parameterization. Different initial parameterizations yield solutions for different geometric functionals
0g g E 0E
12
0
0
0
arg min ( ( ( )) | | )
arg min ( ( ))
pC
L
C
g C p C dp
g C s E ds
x
y
p
1
0
Caselles Kimmel Sapiro, IJCV 97
Geodesic Active Contours in 1D
Geodesic active contours are reparameterization invariant
I(x)
x
g(x)
x
NΝ
),(CgCgdt
dC
Geodesic Active Contours in 2D
2
1
1 | ( * ) |G I g(x)=
G *I
NNCgCgdt
dC ),(
Controlling -max
0
( )Lg C ds
I g
Smoothness
Cohen Kimmel, IJCV 97
Fermat’s PrincipleIn an isotropic medium, the paths taken by
light rays are extremal geodesics w.r.t.
i.e.,
Cohen Kimmel, IJCV 97
),(),( yxIyxg
CC
dssCg ))((minarg
Experiments - Color Segmentation
Goldenberg, Kimmel, Rivlin, Rudzsky,
IEEE T-IP 2001
NNCgCgdt
dC
dsCg
),(
)(
Tumor in 3D MRI
Caselles,Kimmel, Sapiro, Sbert, IEEE T-PAMI 97
NNSgHSgdt
dS
daSg
),(
)(
Segmentation in 4D
NNMgHMgdt
dM ),(
Malladi, Kimmel, Adalsteinsson, Caselles, Sapiro, Sethian
SIAM Biomedical workshop 96
NNMgHMgdt
dM
dvMg
),(
)(
Tracking in Color Movies
Goldenberg, Kimmel, Rivlin, Rudzsky,
IEEE T-IP 2001
Tracking in Color Movies
Goldenberg, Kimmel, Rivlin, Rudzsky,
IEEE T-IP 2001
Edge Gradient Estimators
)],(),,([),( yxvyxuIyxI
Xu-Prince 98, Paragios et al. 01, Vasilevskiy-Siddiqi 01, Kimmel-Bruckstein 01
Edge Gradient Estimators
We want a curve with large points and small ‘s so:
Consider the functional
Where is a scalar function, e.g. .
C
dsINCE ),()(
)],(),,([ yxvyxuI
)(sC
)(sN
cos|)(|)(, CICIN
)(s
|| I
) ( ) (
The Classic ConnectionSuppose and we consider a closed contour for C(s).We have
and by Green’s Theorem we have
dsxyvudsNICEsC
ss
sC
)()(
),(),,(,)(
dxdyyxI
dxdyII
Ivu
dxdyuvCE
sC
sC
yyxx
sC
xy
)( nArea withi
)( nArea withi
)( nArea withi
),( of Laplacian
yields this,),(for But,
)(
)(
Therefore:
Hence curves that maximize are curves that enclose all regions where is positive!
We have that the optimal curves in this case are The Zero Crossings of the Laplacian
isn’t this familiar?
The Classic Connection
dxdyIGCEC
)*()(
),(* yxIG
C )(sC
IG *)(CE
IG *
It is pedagogically nice, but the MARR-HILDRETH edge detector is a bit too sensitive.
So we do not propose a grand return to MH but a rethinking of the functionals used in active contours in view of this.
INDEED, why should we ignore the gradient directions (estimates) and have every edge integrator controlled by the local gradient intensity alone?
The Classic Connection
Our Proposal
Consider functional of the form
These functionals yield “regularized” curves that combine the good properties of LZC’s where precise border following is needed, with the good properties of the GAC over noisy regions!
s...other term ||1
1)( where
))((),()(
ICg
dssCgdsINCECC
Implementation Details
We implement curve evolution that do gradient descent w.r.t. the functional
Here the Euler Lagrange Equations provide the explicit formulae.
For closed contours we compute the evolved curve via the Osher-Sethian “miracle” numeric level set formulation.
C
tCE
dt
dC
))((
)(CE
Closed contours
EL eq.
0,)div(),sign( NgNgVVN
GAC
GAC
LZC
LZC
Kimmel-Bruckstein IVCNZ01
gLL
Closed contours
EL eq.
0,)div(),sign( NgNgVVN
GAC
LZC
LZC+GAC
Kimmel-Bruckstein IVCNZ01
gLL
Along the curveb.c. at C(0) and C(L)
Open contours 0)div(),sign( NVVN
0,),sign(|),|1( NVTVNTVN
Kimmel-Bruckstein IVCNZ01
LL
Open contours
Kimmel-Bruckstein IVCNZ01
LL
Geometric Measures
Weighted arc-length
Weighted area
Alignment
Robustalignment e.g.
RR 2:, yx
0),(
Ngdayxg
0),())(( NNggdssCgC
0)div(, NVdsVNC
0)div(),sign(|,| NVVNdsVNC
0, NIdsINC
Variational meaning for Marr-Hildreth edge detector Kimmel-Bruckstein IVCNZ01
Geometric Measures
Minimal varianceChan-Vese, Mumford-Shah, Max-Lloyd, Threshold,…
0))((
)()(
221
\
22
21
21
NIcc
dacIdacI
cc
CC
C
0))(( 212t
2
1
21
\
\
NIccC
c
c
cc
da
Ida
da
Ida
C
C
C
C
C
C \
Geometric Measures
Robust minimal deviation
0- 21
\
21
NcIcI
dacIdacICC
C
NcIcIC
yxIc
yxIc
C
C
12t
\2
1
),(median
),(median
C
C \
Haralick/Canny-like Edge Detector
Haralick suggested as edge detector0I
IIIII yyxx
III
Laplace
Alignment Topological Homogeneity
Haralick/Canny Edge Detector
III
II I
div
III
dIdsdIdxdyIdxdyICCC
II 2
0I
Haralick
co-area
h hdxdyIC
2
x
y
I
Thus, indicates optimal alignment + topological homogeneity
0I
Closed Contours & Level Set Method
implicit representation of CThen,
Geodesic active contour level set formulation
Including weighted (by g) area minimization
RR 2:, yx 0),(:, yxyxC
dC d
VN Vdt dt
yxgdt
d,div
divd
g gdt
y
x
C(t)
C(t) level set 0
, ,x y t
x
y
Operator Splitting Schemes
Additive operator splitting (AOS) Lu et al. 90, Weickert, et al. 98 unconditionally stable for non-linear diffusion
Given the evolution write
Consider the operator Explicit scheme
, the time step, is upper bounded for stability
gt div 00 uu
ll xxl yxgA ,
k
ll
k A
2
1
1 Ι
2
1
divl
xx llgg
LOD:
Operator Splitting Schemes
Implicit scheme
inverting large bandwidth matrix First order, semi-implicit, additive operator splitting (AOS),
or locally one-dimensional (LOD) multiplicative schemes are
stable and efficient given by linear tridiagonal systems of equations
that can be solved for by Thomas algorithm
k
ll
k A 12
1
1
I
k
ll
k A
2
1
11 22
1I
1k
kl
l
k A 12
1
1
I AOS:
Operator Splitting Schemes
We used the following relation (AOS)
Locally One-Dimensional scheme (LOD)
Decoupling the axes and the implicit formulation leads to computational efficiencyThe 1st order `splitting’ idea is based on the operator expansion
)(21212
1 )1( 21
21
11
211 OAAAA kkkk
)(11 )1( 212
11
121
1 OAAAA kkkk
)(11 21 OAA
The geodesic active contour model
Where I is the image and the implicit representation of the curve
If is a distance, then , and the short time evolution is
Note that and thus can be computed once for the whole image
1
Igt div
IAl IAlI
, ,x y t
Example: Geodesic Active Contour
y
x
C(t)
x
y
Goldenberg, Kimmel, Rivlin, Rudzsky,
IEEE T-IP 2001
),(div yxIgt
Example: Geodesic Active Contour
is restricted to be a distance map:Re-initialization by Sethian’s fast marching method every iteration in O(n).
Computations are performed in a narrow band around the zero set
Multi-scale approach:process a Gaussian pyramid of the image
NNO log
tyxz ,,
y
x
C(t)
x
y
Tracking Objects in Movies
Movie volume as a spatial-temporal 3D hybrid space The AOS scheme is
Edge function derived by the Beltrami framework Sochen Kimmel Malladi 98
Contour in frame n is the initial condition for frame n+1.
k
ll
k IA 11 33
1 I
222
222
222
1
1
1
BGRBBGGRRBBGGRR
BBGGRRBGRBBGGRR
BBGGRRBBGGRRBGR
g
yyyxxx
yyyyyyyxyxyx
xxxyxyxyxxxx
ij
x
x
y
y
t
t
Experiments - Curvature Flow
20 50
Experiments - Curvature Flow CPU Time
400
600
800
1000
1200
0 1 2 3 4 5 6 7 8 9 10 11
time step
CP
U t
ime
(sec
) [1
]
0
50
100
150
200
250
300
CP
U t
ime
(sec
) [2
,3]
1. explicit (w hole image) 2. explicit (narrow ) 3. AOS (narrow )
Tracking
Goldenberg, Kimmel, Rivlin, Rudzsky, ECCV 2002
Tracking
Goldenberg, Kimmel, Rivlin, Rudzsky, ECCV 2002
Tracking
Goldenberg, Kimmel, Rivlin, Rudzsky, ECCV 2002
Information extraction
Goldenberg, Kimmel, Rivlin, Rudzsky, ECCV 2002
Holzman-Gazit, Goldshier, Kimmel 2003
=
I I I I
I H I
Thin Structures
Segmentation in 3D
Change in topology
Caselles,Kimmel, Sapiro, Sbert, IEEE T-PAMI 97
Gray Matter Segmentation
Goldenberg Kimmel Rivlin Rudzsky, VLSM 2001
Coupled surfaces
EL equations
dxdydzhg
dxdydzhg
||)(
||)(
212
121
),(||
div
),(||
div
1222
222
2111
111
Fg
Fg
t
t
Gray Matter Segmentation
Goldenberg Kimmel Rivlin Rudzsky, VLSM 2001
Gray Matter Segmentation
Goldenberg Kimmel Rivlin Rudzsky, VLSM 2001
Classification (dogs & cats)
walk run gallop cat...
Goldenberg, Kimmel, Rivlin, Rudzsky, ECCV 2002
Classification (people)
walk run run45
Goldenberg, Kimmel, Rivlin, Rudzsky, ECCV 2002
Conclusions Geometric-Variational method for segmentation
and tracking in finite dimensions based on prior knowledge (more accurately, good initial conditions).
Using the directional information for edge integration.
Geometric-variational meaning for the Marr-Hildreth and the Haralick (Canny) edge detectors, leads to ways to design improved ones.
Efficient numerical implementation for active contours.
Various medical and more general applications.www.cs.technion.ac.il/
~ron
Edge Indicator Function for Color
Beltrami framework: Color image = 2D surface in space The induced metric tensor for the image surface
Edge indicator = largest eigenvalue of the structure tensor metric. It represents the direction of maximal change in
, , , , , , ,GRx y x y y Bx x y , , , ,Rx Gy B
i i j
ji
i
ii uuuu2
222
2
1
2
1
2
11
2 2 2
2 2 2
1
1x x x x y x y x y
ijx y x y x y y y y
R R R
R R
G G GB B B
B B BG GRGg
2 2 2Gd d BR d
X
I
Y
AOS
Proof:
The whole low order splitting idea is based on the operator expansion
)(1 )()(11
)()(221
11
)(221
)()(1
)(221
)(1
)(221
)(22
2
1
)(221
2121
2
1
21
1
21
1
2
1
221
221
22
2121
221
221
221
21
221
212
21
21
21
OAAOOAA
OOAA
AA
OAA
OAA
OAA
AA
OAA
AA
OAA
AA
AA
)(21
1
21
1
2
11 2
2121
O
AAAA
)(11
1 2
OAA
top related