markov entropy approximation scheme -...

Markov Entropy Approximation Scheme

David Poulin

Département de PhysiqueUniversité de Sherbrooke

Joint work with: Matt Leifer, Ersen Bilgin, and Matt Hastings

Quantum Computation and Quantum Spin SystemsVienna, August 2009

David Poulin (Sherbrooke) Markov Entropy Approximation Scheme Vienna’09 1 / 33

TaskCompute thermal properties of spin systems on lattices.

Origin of the method

Refinement of quantum generalized belief propagation.

ResultAlgorithm to produce lower bounds to free energy by solving finite-sizeconvex optimization problem with linear constraints.

Why is this interesting?

Variational methods (PEPS, MPS, MERA) provide UPPER bounds tofree energy.

Outline

1 Belief propagation

2 Quantum belief propagation

3 Markov Entropy Approximation Scheme

4 Closing

Belief propagation

Outline

4 Closing

Belief propagation

Description of the algorithm

PurposeApproximate the solution to statistical inference problems involvinga large number of random variables v (spins) located at thevertices v ∈ V of a graph G = (V ,E).The graph’s edges (u, v) ∈ E encode some king of directdependency relation of the random variables.

Algorithm architectureHighly parallelizable: one processor per vertex.Messages exchanged between processors related by an edge.Outgoing messages at v depend on local "fields" and incommingmessages.Exact when G is a tree and complexity = depth(G).Good heuristic on loopy graphs.

Belief propagation

Transfer matrix

Consider the 1d classical system with hamiltonianH =

∑i h(vi) +

∑〈ij〉 J(vi , vj).

Its Gibbs distribution is (µ(v) = e−βh(v) and ν(u : v) = e−βJ(u,v))

ρ(v1, v2, . . .) =1Z

e−βH(v1,v2,...)

=1Zµ(v1)ν(v1, v2)µ(v2)ν(v2, v3)µ(v3) . . .

So the reduced state of spin N can be evaluated step by step:

m1→2(v2) =∑v1

µ(v1)ν(v1 : v2)

m2→3(v3) =∑v2

mv1→v2(v2)µ(v2)ν(v2 : v3)

m3→4(v4) =∑v3

mv2→v3(v3)µ(v3)ν(v3 : v4)...

ρ(vN) = mvN−1→vN (vN)µ(vN)

Belief propagation

Transfer matrix

∑i h(vi) +

ρ(v1, v2, . . .) =1Z

e−βH(v1,v2,...)

=1Zµ(v1)ν(v1, v2)µ(v2)ν(v2, v3)µ(v3) . . .

m1→2(v2) =∑v1

µ(v1)ν(v1 : v2)

m2→3(v3) =∑v2

mv1→v2(v2)µ(v2)ν(v2 : v3)

m3→4(v4) =∑v3

mv2→v3(v3)µ(v3)ν(v3 : v4)...

Belief propagation

Transfer matrix

∑i h(vi) +

ρ(v1, v2, . . .) =1Z

e−βH(v1,v2,...)

=1Zµ(v1)ν(v1, v2)µ(v2)ν(v2, v3)µ(v3) . . .

m1→2(v2) =∑v1

µ(v1)ν(v1 : v2)

m2→3(v3) =∑v2

mv1→v2(v2)µ(v2)ν(v2 : v3)

m3→4(v4) =∑v3

mv2→v3(v3)µ(v3)ν(v3 : v4)...

Belief propagation

Transfer matrix

∑i h(vi) +

ρ(v1, v2, . . .) =1Z

e−βH(v1,v2,...)

=1Zµ(v1)ν(v1, v2)µ(v2)ν(v2, v3)µ(v3) . . .

m1→2(v2) =∑v1

µ(v1)ν(v1 : v2)

m2→3(v3) =∑v2

mv1→v2(v2)µ(v2)ν(v2 : v3)

m3→4(v4) =∑v3

mv2→v3(v3)µ(v3)ν(v3 : v4)...

Belief propagation

Transfer matrix

∑i h(vi) +

ρ(v1, v2, . . .) =1Z

e−βH(v1,v2,...)

=1Zµ(v1)ν(v1, v2)µ(v2)ν(v2, v3)µ(v3) . . .

m1→2(v2) =∑v1

µ(v1)ν(v1 : v2)

m2→3(v3) =∑v2

mv1→v2(v2)µ(v2)ν(v2 : v3)

m3→4(v4) =∑v3

mv2→v3(v3)µ(v3)ν(v3 : v4)...

Belief propagation

Transfer matrix

∑i h(vi) +

ρ(v1, v2, . . .) =1Z

e−βH(v1,v2,...)

=1Zµ(v1)ν(v1, v2)µ(v2)ν(v2, v3)µ(v3) . . .

m1→2(v2) =∑v1

µ(v1)ν(v1 : v2)

m2→3(v3) =∑v2

mv1→v2(v2)µ(v2)ν(v2 : v3)

m3→4(v4) =∑v3

mv2→v3(v3)µ(v3)ν(v3 : v4)...

Belief propagation

Transfer matrix

∑i h(vi) +

ρ(v1, v2, . . .) =1Z

e−βH(v1,v2,...)

=1Zµ(v1)ν(v1, v2)µ(v2)ν(v2, v3)µ(v3) . . .

m1→2(v2) =∑v1

µ(v1)ν(v1 : v2)

m2→3(v3) =∑v2

mv1→v2(v2)µ(v2)ν(v2 : v3)

m3→4(v4) =∑v3

mv2→v3(v3)µ(v3)ν(v3 : v4)...

Belief propagation

General setting

Graph G = (V ,E).Random variables vi for i ∈ V .Functions µi(vi) for i ∈ V and νij(vi : vj) for (i , j) ∈ E .

You can think of them as exponentiels of local fields µ(v) = e−βh(v)

and couplings ν(u, v) = e−βJ(u,v).

Task (basic case)Evaluate

ρk (vk ) =∑{vi}−vk

∏i∈V

µi(vi)∏ij∈E

νij(vi : vj)

=∑{vi}−vk

e−β(P

i∈V hi (vi )+P

i,j∈E Jij (vi ,vj ))

Belief propagation

General setting

∏i∈V

µi(vi)∏ij∈E

νij(vi : vj)

=∑{vi}−vk

e−β(P

i∈V hi (vi )+P

Belief propagation

General setting

∏i∈V

µi(vi)∏ij∈E

νij(vi : vj)

=∑{vi}−vk

e−β(P

i∈V hi (vi )+P

Belief propagation

General setting

∏i∈V

µi(vi)∏ij∈E

νij(vi : vj)

=∑{vi}−vk

e−β(P

i∈V hi (vi )+P

Belief propagation

General setting

∏i∈V

µi(vi)∏ij∈E

νij(vi : vj)

=∑{vi}−vk

e−β(P

i∈V hi (vi )+P

Belief propagation

Belief propagation algorithm

Algorithm

Initialization mu→v (v) = cte.Iterations mu→v (v) ∝∑u µ(u)ν(u : v)

∏v ′∈n(u)−v mv ′→u(u).

Beliefs b(u) ∝ µ(u)∏

v∈n(u) mv→u(u).b(u, v) ∝ µ(u)µ(v)ν(u : v)

∏w∈n(u)−v mw→u(u)

∏w∈n(v)−u mw→v (v).

Belief propagation

Algorithm

∏v ′∈n(u)−v mv ′→u(u).

... u v

µu!(u : v)mu!v

Belief propagation

Algorithm

∏v ′∈n(u)−v mv ′→u(u).

... u v

µu!(u : v)mu!v

Belief propagation

Algorithm

∏v ′∈n(u)−v mv ′→u(u).

... u v

µu!(u : v)mu!v

Belief propagation

Bethe free energy

Let H =∑

v hv +∑〈u,v〉 Juv be a local Hamiltonian on G.

Given a probability P(V ), the Gibbs free energy G = E − TSwhere

E =∑

P(V )H(V ) and S = −∑

P(V ) log P(V )

The Gibbs distribution P(V ) = 1Z e−βH is the stationary point of G.

E only depends on the two-body probabilities

E =∑〈u,v〉

P(u, v)Juv +∑

P(u)hu

So does S if P is the product of local functions on a tree (whichincludes Gibbs distributions)

S = −∑〈u,v〉

P(u, v) log P(u, v)−∑

(1− dv )P(u) log P(u)

Belief propagation

Bethe free energy

Let H =∑

E =∑

P(V )H(V ) and S = −∑

P(V ) log P(V )

E =∑〈u,v〉

P(u, v)Juv +∑

P(u)hu

S = −∑〈u,v〉

Belief propagation

Bethe free energy

Let H =∑

E =∑

P(V )H(V ) and S = −∑

P(V ) log P(V )

E =∑〈u,v〉

P(u, v)Juv +∑

P(u)hu

S = −∑〈u,v〉

Belief propagation

Bethe free energy

Let H =∑

E =∑

P(V )H(V ) and S = −∑

P(V ) log P(V )

E =∑〈u,v〉

P(u, v)Juv +∑

P(u)hu

S = −∑〈u,v〉

Belief propagation

Bethe free energy

Let H =∑

E =∑

P(V )H(V ) and S = −∑

P(V ) log P(V )

E =∑〈u,v〉

P(u, v)Juv +∑

P(u)hu

S = −∑〈u,v〉

Belief propagation

Bethe free energy

The Bethe free energy is extending this expression to arbitrarygraphs

GBethe =∑〈u,v〉

P(u, v)(T log P(u, v)+Juv )+∑

(1−du)P(u)(T log P(u)+hu).

This is seemingly easier to handle because it involves onlytwo-body distributions.

Belief propagation

Bethe free energy

The Bethe free energy is extending this expression to arbitrarygraphs

GBethe =∑〈u,v〉

P(u, v)(T log P(u, v)+Juv )+∑

(1−du)P(u)(T log P(u)+hu).

This is seemingly easier to handle because it involves onlytwo-body distributions.

Belief propagation

Bethe free energy and BP

Theorem (Yedidia, Freeman, and Weiss)

The fixed point of the belief propagation algorithms b(u, v) and b(u)are stationary points of the Bethe free energy.

Use BP to solve Bethe free energy minimization.Exact on tree, complexity = depth(G).Most successful on trees with only large loops:

LDPC and Turbo codes.Spin glasses on Bethe lattices.Random k -SAT assignments.

Belief propagation

Generalized belief propagation

When the graph contains small loops, “collapse" them into asingle super-site.Messages exchanged between (possibly overlapping) regions.

If clustered graph is a tree, method is exact.

Belief propagation

Kikuchi free energy

Fix a set of regions on the graph.Approximate the entropy by the sum of the entropy of the regions.Correct for the over counting of the sub-regions.Correct for the sub-sub-regions ...

Corresponds to Bethe’s free energy when regions are nearestneighbor pairs.

Belief propagation

Kikuchi free energy

S = S1245 + S2356 + S478 + S5689

Belief propagation

Kikuchi free energy

S = S1245 + S2356 + S478 + S5689

!S4 ! S8 ! S25 ! S56

Belief propagation

Kikuchi free energy

S = S1245 + S2356 + S478 + S5689

!S4 ! S8 ! S25 ! S56

Belief propagation

Kikuchi free energy

S = S1245 + S2356 + S478 + S5689

!S4 ! S8 ! S25 ! S56

Belief propagation

Kikuchi free energy and generalized BP

Theorem (Yedidia, Freeman, and Weiss)The fixed point of the generalized belief propagation algorithms arestationary points of the Kikushi free energy computed using the sameregions.a

aTechnical detail: the counting number of each sub-region must be non-positive.

Greatly improves numerical precision for graphs containing smallloops.Can use GBP instead of minimizing Kikuchi free energy.

Belief propagation

Quantum belief propagation

Outline

4 Closing

The algorithm

Defining the log-exp product A� B = elog A+log B, the quantum andclassical setting become very similar.

Let G = (V ,E) be a graph with Hamiltonian H =∑

hi +∑

Jij .Defining µi = e−βhi , νij = e−βJij , the Gibbs state is

ρV =1Z

(⊗i∈V

)�( ⊙

(i,j)∈E

)We can execute BP by substituting ordinary products for �.

TheoremIf G is a tree and (G, ρV ) is a quantum Markov random field, then thebeliefs bu converge to the correct marginals ρu = TrV−u{ρV} in a timeproportional to depth(G).

This doesn’t happen very often, but similar conditions forgeneralized BP are more natural.

The algorithm

hi +∑

ρV =1Z

(⊗i∈V

)�( ⊙

(i,j)∈E

The algorithm

hi +∑

ρV =1Z

(⊗i∈V

)�( ⊙

(i,j)∈E

The algorithm

hi +∑

ρV =1Z

(⊗i∈V

)�( ⊙

(i,j)∈E

The algorithm

hi +∑

ρV =1Z

(⊗i∈V

)�( ⊙

(i,j)∈E

The algorithm

hi +∑

ρV =1Z

(⊗i∈V

)�( ⊙

(i,j)∈E

Effective thermal hamiltonian

-5 -4 -3 -2 -1 0 1 2 3 4 5... ...

exp{!"H}H =!!

i="!hi + Ji,i+1

Effective Thermal Hamiltonian =∑∞