Partial Differential Equations and Level-Set Methods in Image Sciences

www.ece.umn.edu/users/guille

What is a discretecomputer image?

Consequences of discrete image representations

! "

#

The PDE’s approach

$ %

$&'()*

("

"

Why? Why Now? Who?

+"!&,,,

'

+"&-!

."

/"01

1&"

What is it?

V=F(curvatures, etc)

Heart segmentation from MRI data(Malladi et al.)

Click each figure for a movie

The dataReconstruction

Reconstructedheart beating

Heartandmeasurements

Introduction to Differential Geometry

/!"23

Planar Curves !"!#$!%&

$

$% $'

$(

$)

$*+

$*

$,

∈

Arc-length and Curvature

--0

p

| | 1,sC =pC

1ρκ

=

nCss

ρκ=

nCss

ρκ=

==

|| p

ps C

CCt

ρ

Surface

.!

/ !01

2

.

3

2 M: 2 ≥→⊂Ω nS nR

),(),,(),,(),( vuzvuyvuxvuS =

vu

vu

SS

SSN

××=

ρ

Nρ

uS

vS

vu SSdA ×=

×= dudvSSA vu

Curves on Surfaces: The Geodesic Curvature

NNCCN ssssg

ρρ,ˆ −=κ

ssC

Nρ

Curves on Surfaces: The Geodesic Curvature

ssC

Nρ

nκ)(min

)(max

2

1

κκκκ

θ

θ

==

4

221 κκ +=H

21κκ=K

NCssn

ρ,=κ

2

Geometric measures

!!!56

7

8

21,κκ

gκ

pT

tρ

nρ

nκ

tρ

nρκ

ssCnκ

Planar Curve Evolution

Curvature flow

96:

nCCC sssst

ρκ== where

nCt

ρκ=

;

9

Curvature flow

3<

6

6

9==

<66

nCt

ρκ=

57

/=

>6

Important property

36

nnVCVC tt

ρρρρ , =⇔=

Vρ

nnVρρρ

,

Constant flow

?

@

9:566

6

9<

:

6

7

nCt

ρ=

Constant flow

?

nCt

ρ=

6

6<

Introduction to Calculus of Variations

Calculus of Variations

A 6<66!!!!;66/ 6 B 6

C6 695@19:!

dxuuF x ),(

0),( =

∂∂−

∂∂

xx

uuFudx

d

u

Calculus of Variations

9 B/666 ! " ;666B

! $! %

B:6 !

+1

0

21x

x

x dxy

( ) baxxyayy

yx

x

xx +===+

)( 01

2/32

0x 1x

)( 1xy)( 0xy

Extrema points in calculus

0x

)(xf

0)(0)(:0)(

lim:0

=⇔=∀⇔=

+∀→

xfxfd

xdfxx ηη

εεηη

ε

xt fx −=

x)( 001 xdtfxx x−=

Calculus of variations

0xx

1x

0)~,~(lim:)(

)()()(~

),())((

?

0=

∀

+=

=

→dxuuF

d

dx

xxuxu

dxuuFxuE

x

x

εη

εη

ε )(xη

)(xu)(~ xu

),()(

xx

uuFudx

d

uu

uE

∂∂−

∂∂=

δδ

u

uEut δ

δ )(−=

Euler Lagrange Equation

−=

−+=

+=+=

dxFdx

dF

dxFdx

dFdxF

dxFFdxuFuFdxuuFd

d

x

xx

xx

uu

u

x

xuu

xuuxuux

η

ηηη

ηηε εε

)(

)(

)()~~()~,~(

~~

~~~

~~~~

1

0

0),( =

∂∂−

∂∂

xx

uuFudx

d

u

3669@ :

4 $! %B

Conclusions

xt fx −=

u

uEut δ

δ )(−=

0x

x

1x

)(xu

0x

)(xf

x

dxuuF xxu

),(minarg)(

)(minarg xfx

95@

)(uE

Level Set Formulationfor Curve Evolution

366=

Implicit representation21:)( RS →pC

0),(|),( == yxyxC φ

0>φ0<φ

1=φ

1−=φ

Properties of level sets

36

4 .6;6A6!

6!==66

!

|| φφ

∇∇−=N

ρNρ

0=sφ

Tyxyx sysxs

ρ,),( φφφφ ∇=+=

Tρ

||||0,

|| φφ

φφ

φφ

∇∇−=⊥

∇∇

=∇∇

NTTρρρ

∇∇=

|| φφ

Tρ

Properties of level sets

36

4 A6 6!!

!

∇∇=

||div

φφκ

0=ssφ

NTds

dT

ds

dyx

ds

dyx sysxss

ρρρκφφφφφφ ,,,)(),( ∇+∇=∇=+=

[ ] ...||

,||

,,||

,||

,,,,

||],,[||

||,

φφ

φφφ

φφφ

φφφφ

φφφφφφφκ

φφφκ

∇∇

∇∇∇

∇∇∇−=

∇∇∇∇−=

∇∇++−=∇=

∇∇∇

yxyx

syysxysxysxx

TT

yxyx

ρρ

κρ 1=

Level Set Formulation(Osher-Sethian)

36!

4D66

36!

E6!

( ) RR →2:, yxφ( ) 0),(:, == yxyxC φ

dC d

VN Vdt dt

φ φ= ⇔ = ∇r

ttytx yxt

tyx φφφφ ++=∂

∂= );,(0

0=φ

( ), ,x y tφ

NVNVCyx ttytxt

ρρ,,, φφφφφφ ∇=∇=∇=+=−

|| φφ

∇∇−=N

ρ||

||,, φ

φφφφ ∇=

∇∇∇=∇− VVNV

ρ

|| φφ ∇= Vt

Level Set Formulation

76

2

266;6

Numerical Considerations

/

? !!

1;

96

!!;6

/@=

Numerical Considerations

/;

D<;

)(ihuui =)(xu

x

h

1 1 +− iii

h

uuuD ii

x 211 −+ −≡

h

uuuD ii

x

−≡ ++ 1

h

uuuD ii

x1−− −≡

Truncation Error

3 = 6

)()('

)()('

)()('

)()('')(')()(

)()('')(')()(

2

32!2

11

32!2

11

hOihuuD

hOihuuD

hOihuuD

hOihuhihhuihuhihuu

hOihuhihhuihuhihuu

ix

ix

ix

i

i

+=

+=

+=

++−=−=

+++=+=

−

+

−

+

21

21 −

1 1−

11 −

Numerical Approximations

2

1,11,11,11,1

4h

uuuuuD jijijiji

xy−−−++−++ +−−

≡

211 2

h

uuuuD iii

xx−+ +−≡

1 2- 1

41

41

41−

41−

Hamilton-Jacobi

C%1B7F7=;

C'1BG6HIJ

>6!

36HI 6H;<I

66H6I <;

HI

( ) xxxt uuHu ε=+ )(0

lim→ε

N

ρ+NCt

ρ=

General GBM FrameworkFor Object Segmentation

GBM

Introduction

&4%

.&5

"6

$"

Notation

(&

$&210 RpC ],[ : )( →→→→

) ( ],[],[ : NRRI 21010 →→→→××××

Basic active contours approach

6 22

E C C p dp C p dp I C dp( ) '( ) ''( ) ( ) = + − ∇λ γ2 2

Geodesic active contours (Caselles-Kimmel-Sapiro, ICCV ‘95)

6"

)

)10

/""

E C g I C s ds( ) [| ( ( ))|] = ∇

E C C p dp C p dp I C dp( ) '( ) ''( ) ( ) = + − ∇λ γ2 2

855!KAA !4516!

Geodesic computation

4

E C dsC

tN

E C g I C s dsC

tg N g N

( )

( ) [| ( ( ))|]

= =

= ∇ = − ∇ ⋅

∂∂

κ

∂∂

κ

ρ

ρ ρ

∂∂

β ∂∂

β

C

tN

t= = ∇

Φ Φ| |

/;?656B

Further geometric interpretation

∂∂

κ

Ct

g N g N g N= − ∇ ⋅ +

( )

Model correctness &

& 1!"!

& %!

Automatic skin lesion segmentation via GBM

D. H. Chung and G. Sapiro (IEEE-TMI 2000)

A non-invasive test to aid in the diagnosis of cystic fibrosis: Automatic chloride patch/sensor analysis

E=;;66!6/

=;646 ! 64LL;<!=D6 M

Automatic

Example

Morphing active contours

D55E!C99954. C

Show it to me!!!

The main problem and our goal(Bertalmio-Sapiro-Randall, IEEE-PAMI)

'& %7(

4&

8%"

.

"

'($9:;<5*;;9':;;

/ /=>

Basic Idea '()1

)1>&)1

)1?& )1

The Equations

11t

1

1

,0

track.oboundary t theisset level-zero its:U

)difference absolute (e.g., functiony discrepanc :

edges) value,-gray (e.g., i frame in features :

))(,(

),(

++∞→

+

+

→→ →∆

∆

∇•∆=

∇∆=

nnnn

i

UFnn

nnnn

CCFF

F

UNNFFt

U

FFFt

F

n

ρρ

∂∂∂

∂12

12

-"

'%

Tracking

Tracking