stereo matching using loopy belief propagation li zhang and lin liao april 23, 2004
TRANSCRIPT
![Page 1: Stereo Matching Using Loopy Belief Propagation Li Zhang and Lin Liao April 23, 2004](https://reader031.vdocuments.us/reader031/viewer/2022020417/56649f1f5503460f94c37b1a/html5/thumbnails/1.jpg)
Stereo Matching Using Loopy Belief Propagation
Li Zhang and Lin Liao
April 23, 2004
![Page 2: Stereo Matching Using Loopy Belief Propagation Li Zhang and Lin Liao April 23, 2004](https://reader031.vdocuments.us/reader031/viewer/2022020417/56649f1f5503460f94c37b1a/html5/thumbnails/2.jpg)
Outline
Problem setup Matching on benchmark stereo
images Matching on structure light stereo
images Conclusion
![Page 3: Stereo Matching Using Loopy Belief Propagation Li Zhang and Lin Liao April 23, 2004](https://reader031.vdocuments.us/reader031/viewer/2022020417/56649f1f5503460f94c37b1a/html5/thumbnails/3.jpg)
Matching Costs
Birchfield-Tomasi matching cost
Left image Right image
![Page 4: Stereo Matching Using Loopy Belief Propagation Li Zhang and Lin Liao April 23, 2004](https://reader031.vdocuments.us/reader031/viewer/2022020417/56649f1f5503460f94c37b1a/html5/thumbnails/4.jpg)
Matching Costs
Birchfield-Tomasi matching cost
}Piece-wise linear segment
Left image Right image
![Page 5: Stereo Matching Using Loopy Belief Propagation Li Zhang and Lin Liao April 23, 2004](https://reader031.vdocuments.us/reader031/viewer/2022020417/56649f1f5503460f94c37b1a/html5/thumbnails/5.jpg)
Separation Cost (Potts model)
Let xi, xj be the labels of two adjacent nodes i, j.
The separation cost V(xi,xj) is
and
where ∆I is the image gradient between i and
j; T, s, and P are the parameters.
![Page 6: Stereo Matching Using Loopy Belief Propagation Li Zhang and Lin Liao April 23, 2004](https://reader031.vdocuments.us/reader031/viewer/2022020417/56649f1f5503460f94c37b1a/html5/thumbnails/6.jpg)
Accelerated Belief Propagation Propagate message in one direction
and update each node immediately Advantages:
Messages propagate much faster Do not need to buffer the messages
from previous iteration; it’s easier to implement
We implemented both the MAP estimator and the MMSE estimator
![Page 7: Stereo Matching Using Loopy Belief Propagation Li Zhang and Lin Liao April 23, 2004](https://reader031.vdocuments.us/reader031/viewer/2022020417/56649f1f5503460f94c37b1a/html5/thumbnails/7.jpg)
Tsukuba image
![Page 8: Stereo Matching Using Loopy Belief Propagation Li Zhang and Lin Liao April 23, 2004](https://reader031.vdocuments.us/reader031/viewer/2022020417/56649f1f5503460f94c37b1a/html5/thumbnails/8.jpg)
Tsukuba image—MAP result
Iteration 1
![Page 9: Stereo Matching Using Loopy Belief Propagation Li Zhang and Lin Liao April 23, 2004](https://reader031.vdocuments.us/reader031/viewer/2022020417/56649f1f5503460f94c37b1a/html5/thumbnails/9.jpg)
Tsukuba image—MAP result
Iteration 2
![Page 10: Stereo Matching Using Loopy Belief Propagation Li Zhang and Lin Liao April 23, 2004](https://reader031.vdocuments.us/reader031/viewer/2022020417/56649f1f5503460f94c37b1a/html5/thumbnails/10.jpg)
Tsukuba image—MAP result
Iteration 3
![Page 11: Stereo Matching Using Loopy Belief Propagation Li Zhang and Lin Liao April 23, 2004](https://reader031.vdocuments.us/reader031/viewer/2022020417/56649f1f5503460f94c37b1a/html5/thumbnails/11.jpg)
Tsukuba image—MAP result
Iteration 5
![Page 12: Stereo Matching Using Loopy Belief Propagation Li Zhang and Lin Liao April 23, 2004](https://reader031.vdocuments.us/reader031/viewer/2022020417/56649f1f5503460f94c37b1a/html5/thumbnails/12.jpg)
Tsukuba image—MMSE result
Iteration 1
![Page 13: Stereo Matching Using Loopy Belief Propagation Li Zhang and Lin Liao April 23, 2004](https://reader031.vdocuments.us/reader031/viewer/2022020417/56649f1f5503460f94c37b1a/html5/thumbnails/13.jpg)
Tsukuba image—MMSE result
Iteration 3
![Page 14: Stereo Matching Using Loopy Belief Propagation Li Zhang and Lin Liao April 23, 2004](https://reader031.vdocuments.us/reader031/viewer/2022020417/56649f1f5503460f94c37b1a/html5/thumbnails/14.jpg)
Tsukuba image—MMSE result
Iteration 10
![Page 15: Stereo Matching Using Loopy Belief Propagation Li Zhang and Lin Liao April 23, 2004](https://reader031.vdocuments.us/reader031/viewer/2022020417/56649f1f5503460f94c37b1a/html5/thumbnails/15.jpg)
Tsukuba image—MMSE result
Iteration 20
![Page 16: Stereo Matching Using Loopy Belief Propagation Li Zhang and Lin Liao April 23, 2004](https://reader031.vdocuments.us/reader031/viewer/2022020417/56649f1f5503460f94c37b1a/html5/thumbnails/16.jpg)
Tsukuba image—MMSE result
Iteration 30
![Page 17: Stereo Matching Using Loopy Belief Propagation Li Zhang and Lin Liao April 23, 2004](https://reader031.vdocuments.us/reader031/viewer/2022020417/56649f1f5503460f94c37b1a/html5/thumbnails/17.jpg)
Tsukuba image—MMSE result
Iteration 40
![Page 18: Stereo Matching Using Loopy Belief Propagation Li Zhang and Lin Liao April 23, 2004](https://reader031.vdocuments.us/reader031/viewer/2022020417/56649f1f5503460f94c37b1a/html5/thumbnails/18.jpg)
Tsukuba image—MMSE result
Iteration 50
![Page 19: Stereo Matching Using Loopy Belief Propagation Li Zhang and Lin Liao April 23, 2004](https://reader031.vdocuments.us/reader031/viewer/2022020417/56649f1f5503460f94c37b1a/html5/thumbnails/19.jpg)
Tsukuba image—MAP vs. MMSE
Algorithm
Iterations before converge
Time (sec)
MAP 5 43
MMSE 50 328
![Page 20: Stereo Matching Using Loopy Belief Propagation Li Zhang and Lin Liao April 23, 2004](https://reader031.vdocuments.us/reader031/viewer/2022020417/56649f1f5503460f94c37b1a/html5/thumbnails/20.jpg)
Tsukuba image—Parameters
S = 50Best result
S = 500Over-smoothed
S = 5Under-smoothed
Change the separation cost parameter (S) in the Potts model
![Page 21: Stereo Matching Using Loopy Belief Propagation Li Zhang and Lin Liao April 23, 2004](https://reader031.vdocuments.us/reader031/viewer/2022020417/56649f1f5503460f94c37b1a/html5/thumbnails/21.jpg)
Sawtooth image—MAP result
![Page 22: Stereo Matching Using Loopy Belief Propagation Li Zhang and Lin Liao April 23, 2004](https://reader031.vdocuments.us/reader031/viewer/2022020417/56649f1f5503460f94c37b1a/html5/thumbnails/22.jpg)
Map Image—MAP result
![Page 23: Stereo Matching Using Loopy Belief Propagation Li Zhang and Lin Liao April 23, 2004](https://reader031.vdocuments.us/reader031/viewer/2022020417/56649f1f5503460f94c37b1a/html5/thumbnails/23.jpg)
Venus Image—MAP result
![Page 24: Stereo Matching Using Loopy Belief Propagation Li Zhang and Lin Liao April 23, 2004](https://reader031.vdocuments.us/reader031/viewer/2022020417/56649f1f5503460f94c37b1a/html5/thumbnails/24.jpg)
Structure Light Stereo
Richer texture Larger disparity range ~[0-
100]
![Page 25: Stereo Matching Using Loopy Belief Propagation Li Zhang and Lin Liao April 23, 2004](https://reader031.vdocuments.us/reader031/viewer/2022020417/56649f1f5503460f94c37b1a/html5/thumbnails/25.jpg)
Bust—MAP result
Iteration 1
![Page 26: Stereo Matching Using Loopy Belief Propagation Li Zhang and Lin Liao April 23, 2004](https://reader031.vdocuments.us/reader031/viewer/2022020417/56649f1f5503460f94c37b1a/html5/thumbnails/26.jpg)
Bust—MAP result
Iteration 2
![Page 27: Stereo Matching Using Loopy Belief Propagation Li Zhang and Lin Liao April 23, 2004](https://reader031.vdocuments.us/reader031/viewer/2022020417/56649f1f5503460f94c37b1a/html5/thumbnails/27.jpg)
Bust—MAP result
Iteration 3
![Page 28: Stereo Matching Using Loopy Belief Propagation Li Zhang and Lin Liao April 23, 2004](https://reader031.vdocuments.us/reader031/viewer/2022020417/56649f1f5503460f94c37b1a/html5/thumbnails/28.jpg)
Bust—MAP result
Iteration 5
![Page 29: Stereo Matching Using Loopy Belief Propagation Li Zhang and Lin Liao April 23, 2004](https://reader031.vdocuments.us/reader031/viewer/2022020417/56649f1f5503460f94c37b1a/html5/thumbnails/29.jpg)
Bust—MAP result
Iteration 10
![Page 30: Stereo Matching Using Loopy Belief Propagation Li Zhang and Lin Liao April 23, 2004](https://reader031.vdocuments.us/reader031/viewer/2022020417/56649f1f5503460f94c37b1a/html5/thumbnails/30.jpg)
Bust—MAP result
Iteration 20
![Page 31: Stereo Matching Using Loopy Belief Propagation Li Zhang and Lin Liao April 23, 2004](https://reader031.vdocuments.us/reader031/viewer/2022020417/56649f1f5503460f94c37b1a/html5/thumbnails/31.jpg)
Bust—MAP result
Iteration 30
![Page 32: Stereo Matching Using Loopy Belief Propagation Li Zhang and Lin Liao April 23, 2004](https://reader031.vdocuments.us/reader031/viewer/2022020417/56649f1f5503460f94c37b1a/html5/thumbnails/32.jpg)
Bust—MAP result
Iteration 40
![Page 33: Stereo Matching Using Loopy Belief Propagation Li Zhang and Lin Liao April 23, 2004](https://reader031.vdocuments.us/reader031/viewer/2022020417/56649f1f5503460f94c37b1a/html5/thumbnails/33.jpg)
Bust—MAP result
Iteration 50
![Page 34: Stereo Matching Using Loopy Belief Propagation Li Zhang and Lin Liao April 23, 2004](https://reader031.vdocuments.us/reader031/viewer/2022020417/56649f1f5503460f94c37b1a/html5/thumbnails/34.jpg)
Bust—BP vs. DP
Belief Propagation Dynamic Programming
320x240, 60 labels, 30 sec per iteration
640x480, 120 labels, ~30 sec one pass
![Page 35: Stereo Matching Using Loopy Belief Propagation Li Zhang and Lin Liao April 23, 2004](https://reader031.vdocuments.us/reader031/viewer/2022020417/56649f1f5503460f94c37b1a/html5/thumbnails/35.jpg)
Conclusion
BP-MAP works pretty well BP-MMSE doesn’t work great BP-MAP doesn’t show dramatic
improvement over DP on stereo with dense texture