stereo matching using loopy belief propagation

Stereo Matching Using Loopy Belief Propagation

Li Zhang and Lin Liao

April 23, 2004

Outline

Problem setup Matching on benchmark stereo

images Matching on structure light stereo

images Conclusion

Matching Costs

Birchfield-Tomasi matching cost

Left image Right image

Matching Costs

Birchfield-Tomasi matching cost

}Piece-wise linear segment

Left image Right image

Separation Cost (Potts model)

Let xi, xj be the labels of two adjacent nodes i, j.

The separation cost V(xi,xj) is

and

where ∆I is the image gradient between i and

j; T, s, and P are the parameters.

Accelerated Belief Propagation Propagate message in one direction

and update each node immediately Advantages:

Messages propagate much faster Do not need to buffer the messages

from previous iteration; it’s easier to implement

We implemented both the MAP estimator and the MMSE estimator

Tsukuba image

Tsukuba image—MAP result

Iteration 1

Tsukuba image—MAP result

Iteration 2

Tsukuba image—MAP result

Iteration 3

Tsukuba image—MAP result

Iteration 5

Tsukuba image—MMSE result

Iteration 1

Tsukuba image—MMSE result

Iteration 3

Tsukuba image—MMSE result

Iteration 10

Tsukuba image—MMSE result

Iteration 20

Tsukuba image—MMSE result

Iteration 30

Tsukuba image—MMSE result

Iteration 40

Tsukuba image—MMSE result

Iteration 50

Tsukuba image—MAP vs. MMSE

Algorithm

Iterations before converge

Time (sec)

MAP 5 43

MMSE 50 328

Tsukuba image—Parameters

S = 50Best result

S = 500Over-smoothed

S = 5Under-smoothed

Change the separation cost parameter (S) in the Potts model

Sawtooth image—MAP result

Map Image—MAP result

Venus Image—MAP result

Structure Light Stereo

Richer texture Larger disparity range ~[0-

100]

Bust—MAP result

Iteration 1

Bust—MAP result

Iteration 2

Bust—MAP result

Iteration 3

Bust—MAP result

Iteration 5

Bust—MAP result

Iteration 10

Bust—MAP result

Iteration 20

Bust—MAP result

Iteration 30

Bust—MAP result

Iteration 40

Bust—MAP result

Iteration 50

Bust—BP vs. DP

Belief Propagation Dynamic Programming

320x240, 60 labels, 30 sec per iteration

640x480, 120 labels, ~30 sec one pass

Conclusion

BP-MAP works pretty well BP-MMSE doesn’t work great BP-MAP doesn’t show dramatic

improvement over DP on stereo with dense texture