1 markov random field: a brief introduction (2) tzu-cheng jen institute of electronics, nctu...

1

Markov random field: A brief introduction (2)

Tzu-Cheng Jen

Institute of Electronics, NCTU

2007-07-25

2

Outline

Markov random field: Review

Edge-preserving regularization in image processing

3

Markov random field: Review

4

Prior knowledge

In order to explain the concept of the MRF, we first introduce following definition:

1. i: Site (Pixel) 2. fi: The value at site i (Intensity)

3. S: Set of sites (Image)

4. Ni: The neighboring site of i (1st order neighborhood of i: f2, f4, f5, f7 )

5. Ci: Clique, the subset of S and the element in this subset must be neighboring

f1 f2 f3

f4 fi f5

f6 f7 f8

A 3x3 imagined image

5

Markov random field (MRF)

View the 2D image f as the collection of the random variables (Random field)

A random field is said to be Markov random field if it satisfies following properties

Red: Neighboring site

1 2 8

{ }

1 2 8 2 4 5 7

(1) ( ) 0, (Positivity)

( ) ( , ,.... , ) 0

(2) ( | ) ( | ) (Markovianity)

( | , ,.... ) ( | , , , )

i

i S i i Ni

i i

P f f

P f P f f f f

P f f P f f

P f f f f P f f f f f

Ff1 f2 f3

f4 fi f5

f6 f7 f8

6

Gibbs random field (GRF) and Gibbs distribution

A random field is said to be a Gibbs random field if and only if its configuration f obeys Gibbs distribution, that is:

f1 f2 f3

f4 fi f5

f6 f7 f8

A 3x3 imagined image

1 2

1 2 '{ } { , '}

( ) ( ) ( ) ( , ) .....c i i ic C i C i i C

U f V f V f V f f

1( )1

1 2 8( ) ( , ,.... , )U f

TiP f P f f f f Z e

U(f): Energy function; T: Temperature Vi(f): Clique potential

7

Markov-Gibbs equivalence

Hammersley-Clifford theorem: A random field F is an MRF if and only if F is a GRF

Red: Neighboring site

f1 f2 f3

f4 fi f5

f6 f7 f8

{ }

{ }

1( )1

1 2 8

( | ) ( | )

( | ) ( | )

=> ( ) ( , ,.... , )

f is Gibbs field

i S i i Ni

i S i i Ni

U fT

i

P f f P f f

P f f P f f

P f P f f f f Z e

8

Edge-preserving regularization in image processing

9

MAP formulation for denoising problem

Noisy signal d denoised signal f

d = f + N(0, σ)

10


A signal denoising problem could be modeled as the MAP estimation problem, that is,

arg max{ ( | )}

By Baye's rule:

arg max{ ( | ) ( )}

:

:

f

f

f p f d

f p d f p f

f Unknown data

d Observed data

(Prior model)

(Observation

model)

11


Assume the observation is the true signal plus the independent Gaussian noise, that is

Assume the unknown data f is MRF, the prior model is:

2 2

1

( ) / 2( | )

2 2

1 1( | )

2 2

m

i i ii

f dU d f

m m

i ii m i m

p d f e e

21

11 ( )( )1 1( )i i

i

f fU f TTP f Z e Z e

12


Substitute above information into the MAP estimator, we could get:

22

121 1

arg max{ ( | )} arg min{ ( | ) ( )}

( )arg min{ ( ) }

2

f f

m mi i

f i ii i

f p f d U d f U f

f df f

Observation model (Similarity measure)

Prior model (Reconstruction constrain, Regularization)

13

The solver of the optimization problem: Gradient descent algorithm

14

Simulation results for denoising problem

Simulation resultSimulation result

Processed profiles are blurred !

15

Discussion for the phenomenon of blur (1)

From the potential function point of view:

,

^2

, ,, , ( '

')

''

,,

,arg min ( ) arg min{(1 )* ( ) }( )*i j

i j i jf fi j i j i j N

i j i jf E f w d gf fw f

Quadratic function is used as potential function g=x2

Simulation result

1st derivative

Energy

16

Discussion for the phenomenon of blur (2)

From the optimization process point of view (gradient descent algorithm):

,

,

( 1) ( ) ( ) ( ) ( ), , , , , ', '

( ', ')

( ) ( ) ( ) ( ), , , , ', '

( ', ')

*(2(1 )*( ) * '( ))

= *(2(1 )*( ) * 2 ( ))

i j

i j

t t t t ti j i j i j i j i j i j

i j N

t t t ti j i j i j i j i j

i j N

f f step w f d w g f f

f step w f d w f f

Update equation of gradient descent:

17

Edge-preserving regularization

S. Geman and D. Geman, "Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images," IEEE Trans. Pattern Anal. Mach. Intell, 6, 721-741, 1984.

S.Z. Li, “On Discontinuity-adaptive smoothness priors in computer vision,” IEEE Trans. Pattern Anal. Mach. Intell, June, 1995.

Pierre Charbonnier et al, “Deterministic edge preserving regularization in computed imaging,” IEEE Trans. Image Processing, Feb, 1997.

S.Z. Li, “Markov random field modeling in computer vision,“ Springer, 1995

18

MRF with pixel process and line process (Geman and Geman)

Lattice of pixel site: SP Labeling value: fi

p (real value)

Lattice of line site: SE Labeling value: fii’

E (only 0 or 1)

19


Based on the concept of line process, we could modify the original restoration problem as follows:

, ,

1( , )( | )

, ,

2

2,1

, arg max{ ( , | )} arg max{ ( | , )* ( , )}

arg max{ ( | )* ( , )} arg max{ * }

( )arg min{ ( , )}

2

P E P E

P EP

P E P E

P E

P E P E P E P E

f f f f

U f fP P E U d f Tf f f f

mP Ei i

f fi

f f p f f d P d f f P f f

P d f P f f e e

f dU f f

Goal: Find realizations fp and fE such that edge-preserving regularization could be achieved

?

20


Define the prior:

Substitute above information into the MAP estimator, we could get:

2' ' '

'

( , ) ( ) (1 )P

P E P P E Ei i ii ii

i Nii S

U f f f f f f

, ,

22

' ' '2,1 '

, arg max{ ( | )} arg min{ ( | ) ( )}

( )arg min{ ( ) (1 ) }

2

P E P E

P E

P

P E

f f f f

mP P E Ei ii i ii iif f

i i Nii S

f f p f d U d f U f

f df f f f

The above optimization problem is a combination of real and combinatorial problem

21


Blake and Zisserman covert previous restoration problem into real minimization problem by introducing truncated quadratic function as potential function

'

^

2

( ')

arg min ( )

arg min{(1 )* ( })) (*i

f

i if

ii i

i

P

i

P

N

g

f E f

w d f fw f

Truncated quadratic function

1st derivative alphaalpha

Energy

22


Simulation results

Original image Degraded image Restoration result (1000 iterations)

23


Simulation results

Original image

Degraded image

Restoration result (1000 iterations)

Restoration result (100 iterations)

24

Discontinuity-adaptive regularization (S. Z. Li)

Revisit the gradient descent algorithm

,

( 1) ( ) ( ) ( ) ( ), , , , , ', '

( ', ')

( 1) ( ) ( ) ( ) ( ) ( ) ( ), , , , , ', ' , ', '

( ', ')

*(2(1 )*( ) * '( ))

g'( )=2 h ( )

*(2(1 )*( ) * 2*( )* ( )

i j

t t t t ti j i j i j i j i j i j

i j N

t t t t t t ti j i j i j i j i j i j i j i j

i j

f f step w f d w g f f

Set

f f step w f d w f f h f f

,

)i jN

Adjust it adaptively !

Derivative or compensator

Weight or interaction function

25


For edge-preserving regularization, interaction function hr should satisfy following property:

26


Possible choices for interaction function hr

27


Simulation results (1D)

28


Simulation results (2D)

Original image Edge-preserving restoration

Restoration without preserving edge

29

Discontinuity-adaptive regularization (Pierre Charbonnier et al )

Pierre Charbonnier et al impose following conditions on potential function φfor edge preserving regularization

30

Discontinuity-adaptive regularization (Pierre Charbonnier et al )

Possible choices for potential function φ

31

Other related techniques for edge-preserving regularization

P. Perona and J. Malik, “Scale-space and edge detection using anisotropic diffusion,” IEEE Trans. Pattern Anal. Mach. Intell, July, 1990.

,

^2

, , , ', ', , ( ', ')

(1 )arg min ( ) arg min{ * ( ) * ( )}i j

i j i j i j i jf fi j i j i j N

f E f d f g f fw w

Dropping observation model (w=1) when evaluating f

32

Other related techniques for edge-preserving regularization

L.I. Rudin, S. Osher, E. Fatemi (1992): Nonlinear Total Variation Based Noise Removal Algorithms, Physica D, 60(1-4), 259-268.

,

^2

, ', ,, ,

, '( ', ')

arg min ( ) arg min{(1 | |)* ( ) * }i j

i j i jf fi

i j ij j i j N

ji

f E f w fd f w f

Replace the quadratic potential function with absolute value function

1st derivative

Quadratic function versus Absolute value function

Energy

1 markov random field: a brief introduction (2) tzu-cheng jen institute of electronics, nctu...

Documents

image slide

random field f

regularization slide

random variables random

review slide

image processing slide

markov random field

gibbs random field grf