vector valued image regularization with pde’s: a common framework for different applications
DESCRIPTION
Vector Valued Image Regularization with PDE’s: A Common Framework for Different Applications. CVPR 2003 Best Student Paper Award. Definitions. Scalar images: Intensity images Vector valued images: RGB, HSV, YIQ… Regularization: Finding approximate solution for ill-posed problems. - PowerPoint PPT PresentationTRANSCRIPT
Vector Valued Image Regularization with PDE’s: A Common Framework for Different Applications
CVPR 2003 Best Student Paper Award
Definitions
Scalar images: Intensity images Vector valued images: RGB, HSV, YIQ… Regularization: Finding approximate solution
for ill-posed problems.
Let G be 2x2 matrix
dc
baG
2
1,)(
iiigtrace G
Divergence and Curl
Vectors are obtained from image gradients I
y
A
x
AAAdiv yx
)(
Divergence defines expansion or contraction per unit volume
),( yxA vector function
yyxxy
xII
I
IdivIdivI
)(
Curl of a vector field is defined by cross product between vectors
Structure Tensor
n
iyx
y
xn
i
Tii II
I
III
11
G
Structure Tensor
n
iyx
y
xn
i
Tii II
I
III
11
G
One dimensional image (gray level)
yyyx
yxxxgray IIII
IIIIG
)1,(),(
),1(),(
yxIyxII
yxIyxII
y
x
Structure Tensor
yyyx
yxxx
yyyx
yxxx
yyyx
yxxxcolor
TTTcolor
BBBB
BBBB
GGGG
GGGG
RRRR
RRRR
BBGGRR
G
G
reigenvecto
eigenvalue
color
2
1
2
1
φ
φλ
φ
φG
Eigenvalue and eigenvectors of G (spectral elements)
Can be computed using Matlab functions
x derivative y derivative
More Insight (hessian&tensor)
Hessian of image I
Laplacian of I
Tensor: Matrix of matrices. They abuse the notation: Symmetric semi-positive definite matrix
yyyx
xyxx
II
IIH
yyxx IItraceII )()div( H
xx derivative yy derivative
tracexx derivative yy derivative
Image Regularization Functional minimization: Euler-Lagrange equations
Divergence expression: Diffusion of pixel values from high to low concentration
Oriented Laplacians: Image smoothing in eigenvector directions weighted by corresponding eigenvalue.
dINnRI
))((min:
)( itI IDdiv
vvuutI IcIc 21
Variational Problem
Define variational problem:
dIIIIFIE yxyx ),,,()(
0
yx I
F
dy
d
I
F
dx
d
I
FEuler-Lagrange equation
Solution using classic iterative method:
)1()1()1()1()()(
00
tI
F
dy
dt
I
F
dx
dt
I
FtItI
t
tI
II
yx
t
Variational Problem Horn&Schunck Example
0 tyx fvfuf dtdy
dtdx vu
dxdyvvuufvfufvuvuE yxyxtyx22222),,,(
022 avguu
yyxxxtyx uuffvfuf
022
avgvv
yyxxytyx vvffvfuf
tyxxt
avgt fvfuf
fuu
1
yx u
E
yu
E
xu
E
yx v
E
yv
E
xv
E
tyxyt
avgt fvfuf
fvv
1
Defining Energy Functional
Functional: (minimization based)
dIERI
),(min)(3:
Euler-Lagrange is given by (relation to divergence based methods)
)div(div i
i
ii ID
I
I
t
I
y
x
TTD
22
increasing function)()(, afbfab
D Matrix
TTD
22 is 2x2.
Eigenvalues and eigenvectors of D are
22 21
21 uu
iTTi I
t
I
22div
Old School Laplacian Approach
TT ccT 222111 vvuu
vvuu IcIct
I21
Second order image derivatives in directions of eigenvectors of G at that point (structure tensor)(Edge preserving smoothing)
1 tt II
)( ii Ttracet
IH
Solution of this PDE can be given by:
),(
)0()(
tii GIItt
T
)4
exp(4
1)(
1),(
ttG t xxT
xT
Laplacian Example (constant T)
Constant T, such that direction (eigenvectors are same)
Laplacian Example
Laplacian Example (varying T)
T is not constant and but independent of image content. There are similarities with bilateral filtering.
Laplacian Example (varying T)
)( ii IDdivt
I
Relation Between T and D
cb
baD
)()()( DdivIDtraceIDdiv Tiii H
Homework Due 19 January 2005
Show the following: (Page 3 of the paper):
Relation Between T and D
)( ii Ttracet
IH
)()( DdivIDtracet
I Tii
i
H
Ti
Ti
Ti
Ti
Ti
Itrace
IGtrace
IGdivtraceIDdivtraceDdivI
)()()(
2
2
Relation Between T and D
3
1
3
1
3
1
3
12
2
2
2
)(
jjjjj
j jjjj
jjjj
jjj
jjj
j jjjjjj
jjjjjj
j jjj
jjj
III
IIII
IIII
III
III
IIIIII
IIIIII
III
IIIdivGdiv
yyyxyx
xyyxxx
yyxxy
yyxxx
xyxyxxyyy
xyyyyxxxx
yyx
yxx
H
Let’s just solve for div(G). Rest can be solved similarly:
yyxxy
xII
I
IdivIdivI
)(
Relation Between T and D
Using trace property:
B
AAAB
2)(
T
tracetrace
3
1
)()(j
jij
iji QDtraceIDdiv H
Kronecker func.
2
2
2
Tijijij
Tij
TT
Tij
TT
jTi
ij
II
II
II
PPQ
G
P
Hessian
TT ffD
2
2
2
1 ),(),(
Rewriting the Formula
3
1
)()(j
jij
iji QDtraceIDdiv H
)()(
)(3
1
AHtraceIDdiv
jtraceIDdivj
iji
HA
333231
232221
131211
AAA
AAA
AAA
A
3
2
1
H
H
H
H
super matrix notation
Unified Expression
)(AHtracet
I
iti
ti
iti
ti
TT
traceII
traceII
H
TH
**
**
****
1
1
1
1
1
1
)(
),(1 f
),(2 f
Numerical Implementation
Conventional approach Compute image Hessians and Gradients
Proposed method Use local filtering by Gaussians (2nd page)
),(
)0()()( t
iiii GIItracet
Itt
TTH
)4
exp(4
1)(
1),(
ttG t xxT
xT
Similar to bilateral filtering
Numerical Implementation
1. Compute local convolution mask defining local geometry by T.
2. Estimate trace(THi) in local neighborhood of x.
3. Apply filtering for each trace(AijHj) in the vector trace(AH)
),( tG T
)1,1(),1()1,1(
)1,(),()1,(
)1,1(),1()1,1(
)1,1(),1()1,1(
)1,(),()1,(
)1,1(),1()1,1(
),()(
yxGyxGyxG
yxGyxGyxG
yxGyxGyxG
yxiyxiyxi
yxiyxiyxi
yxiyxiyxi
yxtrace iTH
Comparison Between Two Implementations
Noise Removal and Image impainting
Reconstruction, VisualizationSuperscale Images