Factor Graphs

2005. 5. 20Young Ki Baik

Computer Vision Lab.

Seoul National University

Factor Graph

Computer Vision Lab. SNU

Contents

Introduction

Sum product algorithm

Computing a single marginal function

Computing all marginal function

Probabilistic modeling

Conclusion

Factor Graph

Computer Vision Lab. SNU

Introduction

What is a Factor Graph?

A factor graph shows how a function of several variables can be

factored into a product of “smaller” function.

xxyyxg ),(

,),( 21 yfxfyxg 1 , 21 yyfxxf

x y1f 2f

variables

functions

:Example

Factor Graph

Computer Vision Lab. SNU

Introduction

What is a Factor Graph?

zxfzyfyxfzyxg ,,,),,( 321

variables

functions

:Example

1fx y z2f

3f

Factor Graph

Computer Vision Lab. SNU

Introduction

Why are factor graphs useful?

Factor graphs simplify problems.

Many efficient algorithms can be applied to factor graphs.

Special Feature

Factor graph represent not only variables or constant, but

also functions.

Factor Graph

Computer Vision Lab. SNU

Marginal function

“~{x}” notation

Marginal function

Sum product algorithm

ypyxpxpyxpxy

),(,),(

}{~

1 , ,ix

nii xxgxg variablesofnumber :

index:

n

i

1 32

321}{~

321 , ,:, ,x xx

xxxhxxxh

Object : Get marginal function g(x) using factor graph

Factor Graph

Computer Vision Lab. SNU

Example (Simple Factor Graph)

Let be a function of four variables.

Sum product algorithm

4321 ,,, xxxxg

4321 ,,, xxxxg

423221 ,,, xxfxxfxxf CBA

AfBf

Cf1x 2x 3x

4x

Factor Graph

Computer Vision Lab. SNU

Computing a single marginal function

Example (Simple Factor Graph)

Marginal function for

}{~

432111

1

,,,x

xxxxgxg

}{~ }{~42

}{~3221

423221

423221

1 22

2 43

2 3 4

,,,

,,,

,,,

x xC

xBA

x xC

xBA

x x xCBA

xxfxxfxxf

xxfxxfxxf

xxfxxfxxf

11 xg

Factor Graph

Computer Vision Lab. SNU

Computing a single marginal function

Example (Simple Factor Graph)

Marginal function for

}{~

432122

2

,,,x

xxxxgxg

}{~42

}{~32

}{~21

423221

423221

222

431

1 3 4

,,,

,,,

,,,

xC

xB

xA

xC

xB

xA

x x xCBA

xxfxxfxxf

xxfxxfxxf

xxfxxfxxf

22 xg

Factor Graph

Computer Vision Lab. SNU

Computing a single marginal function

Example (Simple Factor Graph)

Marginal function for and

}{~ 2x

11 xg

1x

}{~ 2x

}{~ 1x

2x

3x 4x

Af

Bf Cf

22 xg

1x

2x

3x 4x

Af Bf Cf

11 xg 22 xg

Bottom-up procedure

Factor Graph

Computer Vision Lab. SNU

Computing all marginal functions

Computing all marginal functions problem

In order to compute all marginal functions ,

we need to calculate single marginal function as much as

n times.

Message passing algorithm

Solution for redundancy problem

nn xgxg 11

AfBf

Cf1x 2x

Afxu 2

2xf Au

messageu :

Factor Graph

Computer Vision Lab. SNU

Message passing algorithm

Let denote the message sent from node x to node

f in the operation.

Let denote the message sent from node f to node

x.

Computing all marginal functions

xu fx

xu xf

Factor Graph

Computer Vision Lab. SNU

Message passing algorithm

Variable to local function

Local function to variable

Computing all marginal functions

}\{ fxnhxhfx xuxu

}{~ }\{

)(x xfny

fyxf yufnfxu

Factor Graph

Computer Vision Lab. SNU

A Detail Example (Message passing algorithm)

The message may be generated in 4 steps.

➀ ~ ➃ are step of message passing algorithm

Algorithms start from each leaves.

Computing all marginal functions

AfBf

Cf

1x 2x3x

4x

➀➀

➀

➁➁

➁

➂➃

➃➂➃ ➂

Factor Graph

Computer Vision Lab. SNU

A Detail Example (Message passing algorithm)

Step 1:

Computing all marginal functions

AfBf

Cf

1x 2x3x

4x

➀ ➀

➀

111 xu

Afx

133 xu

Bfx

144 xu

Cfx

11xu

Afx 33xu

Bfx

44xu

Cfx

}\{ fxnhxhfx xuxu

Variable to local function

Factor Graph

Computer Vision Lab. SNU

A Detail Example (Message passing algorithm)

Step 2:

Computing all marginal functions 4321 ,,, xxxxg

AfBf

Cf

1x 2x3x

4x

➁➁

➁

21}{~

12 ,2

12xxfxuxu A

xfxxf AA

32}{~

32 ,2

32xxfxuxu B

xfxxf BB

42}{~

42 ,2

42xxfxuxu C

xfxxf CC

22xu xf A

22xu xfB

22xu xfC

}{~ }\{

)(x xfny

fyxf yufnfxu

Local function to variable

Factor Graph

Computer Vision Lab. SNU

A Detail Example (Message passing algorithm)

Step 3:

Computing all marginal functions

AfBf

Cf

1x 2x3x

4x

➂

➂

222 222xuxuxu xfxffx CBA

➂

222 222xuxuxu xfxffx CAB

222 222xuxuxu xfxffx BAC

22xu

Afx

22xu

Bfx

22xu

Cfx

}\{ fxnhxhfx xuxu

Variable to local function

Factor Graph

Computer Vision Lab. SNU

A Detail Example (Message passing algorithm)

Step 4:

Computing all marginal functions

AfBf

Cf

1x 2x3x

4x

➃

➃➃

21}{~

21 ,1

21xxfxuxu A

xfxxf AA

32}{~

23 ,3

23xxfxuxu B

xfxxf BB

42}{~

24 ,4

24xxfxuxu B

xfxxf CC

11xu xf A

33xu xfB

44xu xfC

}{~ }\{

)(x xfny

fyxf yufnfxu

Local function to variable

Factor Graph

Computer Vision Lab. SNU

A Detail Example (Message passing algorithm)

Termination

Computing all marginal functions

AfBf

Cf

1x 2x3x

4x

➀➀

➀

➁➁

➁

➂➃

➃➂➃ ➂

111 1xuxg xf A

22222 222xuxuxuxg xffxxf CBA

333 3xuxg xfB

444 4xuxg xfC

Factor Graph

Computer Vision Lab. SNU

Probabilistic Modeling

Markov chain

n

iii xxfxxf

1121 |,,

1xf

1x 2x 3x 4x

12 | xxf 23 | xxf 34 | xxf

4n

Factor Graph

Computer Vision Lab. SNU

Probabilistic Modeling

Hidden markov model

ii

n

iii xyfxxfyyxxf ||,,,,,

112121

1xf 12 | xxf 23 | xxf 34 | xxf

4n

11 | xyf 22 | xyf 33 | xyf 44 | xyf

1x 2x 3x 4x

1y 2y 3y 4y

Factor Graph

Computer Vision Lab. SNU

Conclusion

More information

The closed factor graph

It can be computed by iterative method.

Conclusion

Factor graph can apply to many efficient algorithms.

Factor graph is only a simplifying tool to solve the problems.