CS774. Markov Random Field : Theory and Application

Lecture 20

Kyomin JungKAIST

Nov 17 2009

Remind: X is a positive binary MRF if

GCCCC xZ

xXP )(exp1

][

for some , where is the set of the cliques of G.

RCC ||}1,0{:

Learning positive binary MRF

Note that is dependent on at most |C| many variables.

Now we consider the problem of learning under the condition that we can check P[X=x] values for polynomially many x’s.

C

sC '

GC

Note that

ZxxXPGCC

CC log)(exp][

Learning positive binary MRF

Given P[X=x], the expression of is not unique : one can add a constant to .

Hence we want to learn one such set of (or want to learn up to constant addition.)

CC

sC '

Pseudo Boolean function

A function f is called a Pseudo-Boolean function if f is defined on and its value is in R.

A pseudo-Boolean function is of order k if f can be expressed as

If one can learn a pseudo-Boolean function of order k from function queries, one can learn the MRF of order k from the probability queries.

)...,()( 211

ikiii

m

i

xxxfxf

n}1,0{

Relation with Fourier transform

f can be expressed by the Fourier coefficients (with Walsh functions):

.)1()(

Hiix

H x

),()(ˆ)(][

xHfxf HnH

)...,()( 211

ikiii

m

i

xxxfxf

nx

nH

xgxfgfwherefHf

}1,0{ 2

)()(,,,)(ˆ

The underlying graph G

is said to have linkage If there is correlation among the variables of H. (for any expression , there is j so that

H belongs to the support set of )

The hyper-graph consisting of all such H’s is called the linkage graph of f.

Linkage graph corresponds to the underlying graph G of the MRF.

43221 2)( xxxxxxf 1x 4x2x

3x

][nH

fG

i

iff

jf

Learning the Linkage Graph

)1()1(),,(||

A

A

HA

xfxHfL

The following linkage test function tests whether there is a linkage among

Linkage among H?

H

].[nH

Property of the Linkage Test

)1()1(),,(||

A

A

HA

xfxHfL

A subset H of [n] is a hyper-edge of if and only if for some string x.

For an order k function f, and a hyperedge H of order j in , the probability that for x chosen uniformly at random from is at least

Linkage among H?

H

fG

0),,( xHfL

fG0),,( xHfL

n}1,0{ .2

1jk

Learning the Fourier coefficients

If and for all , is called a maximal non-zero Fourier coef-ficient of f.

For any H, is a maximal non-zero Fourier coefficient of f if and only if H is a maximal hy-peredge of .

For a maximal hyperedge H,

For any subset H,

From these relations, we can learn all the non-zero , which enables us to learn f.

0)(ˆ Hf 0)(ˆ Hf HHHH ','

)(ˆ Hf

)(ˆ Hf

fG

||2

)0,,()(ˆ

H

nHfLHf

','

||)'(ˆ

2

)0,,()(ˆ

HHHHH

n

HfHfL

Hf

)(ˆ Hf