1

Stereo Matching

• Information Permeability For Stereo Matching– Cevahir Cigla and A.Aydın Alatan – Signal Processing: Image Communication, 2013

• Radiometric Invariant Stereo Matching Based On Relative Gradients – Xiaozhou Zhou and Pierre Boulanger– International Conference on Image Processing (ICIP), IEEE 2012

2

Outline

• Introduction• Related Works• Methods• Conclusion

3

Introduction

• Goal – Get accurate disaprity maps effectively.– Find more robust algorithm, especially refinement

technique.

• Foucus : Refinement step and Comparison

4

Related Works

• Stereo Matching– The same object, the same disparity• Segmentation• Calculate correspond pixels similarity

(color and geographic distance)

– Occlusion handling• Refinement

5

Related Works

• Global Methods– Energy minimization

process

(GC,BP,DP,Cooperative)– Per-processing– Accurate but slow

• Local Methods– A local support region

with winner take all– Fast but inaccurate.– Adaptive Support Weight

6

Related Works

Disparity Refinement

Disparity Optimization

Cost Aggregation

Matching Cost Computation

• Local methods algorithm

[1] D. Scharstein and R. Szeliski. A taxonomy and evaluation of dense two-frame stereo correspondence algorithms.International Journal of Computer Vision (IJCV), 47:7–42, 2002.

7

• Edge Preserving filter : Remove noise and preserve structure/edge, like object consideration. Adaptive Support Weight [3] Bilateral filter(BF) [34] Guided filter(GF) [5] Geodesic diffusion [33] Arbitrary Support Region [39]

Related Works

8

Reference Papers

[3] Kuk-JinYoon, InSoKweon, Adaptive support weight approach for correspondence search, IEEE Transactions on Pattern Analysis and Machine Intelligence, 2006.

[5] C. Rhemann, A. Hosni, M. Bleyer, C. Rother, M. Gelautz, Fast cost-volume filtering for visual correspondence and beyond, CVPR 2011.

[33] L. De-Maetzu, A. Villanueva nad, R. Cabeza, Near real-time stereo matching using geodesic diffusion, IEEE Transactions on Pattern Analysis and Machine Intelligence, 2012.

[34] A. Ansar, A. Castano, L. Matthies, Enhanced real time stereo using bilateral filtering, in: Proceedings of the International Symposium on 3D Data Processing Visualization and Transmission, 2004.

[39] X. Mei, X Sun, M Zhou, S. Jiao, H. Wang, Z. Zhang, On building an accurate stereo matching system on graphics hardware, in: Proceed- ings of GPUCV 2011.

9

Information Permeability For Stereo Matching

Method A.

10

Methods A.

• Goal : Get high quality but low complexity

Save memory

Real-time application

• Successive Weighted Summation (SWS)– Constant time filtering + Weighted aggregation

◎Qingqing Yang, Dongxiao Li, Lianghao Wang, and Ming Zhang, “Full-Image Guided Filtering for Fast Stereo Matching”, Signal Processing Letters, IEEE March 2013 http://www.camdemy.com/media/7110

11

Methods A.

• Cost Computation

Census Transform

1 1 0 0 0

1 1 0 0 0

1 1 X 0 0

0 0 0 1 1

1 1 1 1 1

121 130 26 31 39

109 115 33 40 30

98 102 78 67 45

47 67 32 170 198

39 86 99 159 210

1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 0 0 1 1 1 1 1 1 1

Census transform window :

Census Hamming Distance

• Left image

• Right image

Hamming Distance = 3

1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 0 0 1 1 1 1 1 1 1

1 1 1 0 0 1 1 0 0 1 0 1 0 0 0 0 0 1 1 1 1 1 1 1

XOR

0 0 1 0 0 0 0 0 0 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0

14

Methods A.

• Cost Computation

15

Methods A.

• Cost Aggregation

16

Methods A.

• Cost Aggregation

17

Methods A.

(b)Horizontal effective weights (c)Vertical effective weights (d)2D effective weights

18(a) AW [3]

(b) Geodesic support [12]

(c) Arbitrary support region [4]

(d) Proposed

ComparisonWith

Other Methods

19

Methods A.

• Refinement– Using cross-check to detect reliable and occluded

region detection ф is a constant (set to 0.1 throughout experiments)

20

Methods A.

(a) Linear mapping function for reliable pixels based on disparities

(b)The resultant map for the left image

21

Disparity Variation

BeforeAfter

0 <=> 1.151 <=> 1.302 <=> 1.453 <=> 1.604 <=> 1.755 <=> 1.906 <=> 2.057 <=> 2.208 <=> 2.359 <=> 2.50

10 <=> 2.6511 <=> 2.8012 <=> 2.9513 <=> 3.1014 <=> 3.2515 <=> 3.4016 <=> 3.5517 <=> 3.7018 <=> 3.8519 <=> 4

20 <=> 4.1521 <=> 4.3022 <=> 4.4523 <=> 4.6024 <=> 4.7525 <=> 4.9026 <=> 5.0527 <=> 5.2028 <=> 5.3529 <=> 5.50

30 <=> 5.6531 <=> 5.8032 <=> 5.9533 <=> 6.1034 <=> 6.2535 <=> 6.4036 <=> 6.5537 <=> 6.7038 <=> 6.8539 <=> 7

40 <=> 7.1541 <=> 7.3042 <=> 7.4543 <=> 7.6044 <=> 7.7545 <=> 7.9046 <=> 8.0547 <=> 8.2048 <=> 8.3549 <=> 8.50

50 <=> 8.6551 <=> 8.8052 <=> 8.9553 <=> 9.1054 <=> 9.2555 <=> 9.4056 <=> 9.5557 <=> 9.7058 <=> 9.8559 <=> 10

22

•

(b) Without occlusion handling, bright regions correspond to small disparities

(c) Detection of occluded and un-reliable regions

23

Methods A.

(b) occlusion handling with no background favoring (c) the proposed occlusion handling

24

Experimental Results A.

• Device : Core Duo 1.80 GHz 2G Ram CPU • Implemented in C++ • Parameter : (T, α, )=(15, 0.2, 8)

25

26

Parameter of Method A.

27

28

Experimental Results A.

29

Experimental Results A.

6D + 4D *V.S.

129D + 21D *

10~15X

30

Experimental Results A.

• Proposed method is the fastest method without any special hardware implementation among Top-10 local methods of the Middlebury test bench, as of February 2013.

31Proposed

O(1) AW

Guided filter

Geodesic support

Arbitrary shaped cross filter

32

Experimental Results A.

33

Computational times A.

Cost InitializationCost AggregationRefinementOthers

≈70~75%

≈20~25%

≈5%

≈84%Cost InitializationCost AggregationMinimizationRefinement

≈45%≈44%

34

Error Analysis A.

35

Comparison with Full-Image◎

Full-Image Proposed

Initialization AD + Gradient SAD + Census

Aggregation

Refinement 1.Cross checking (lowest disparity)2.Weighted median filter

1. Cross checking (normalized disparity)2. Median filter (background handling)

◎Qingqing Yang, Dongxiao Li, Lianghao Wang, and Ming Zhang, “Full-Image Guided Filtering for Fast Stereo Matching”, Signal Processing Letters, IEEE

36

Comparison with Full-Image

37

Full-Image Results

38

Full-Image Results

Proposed Results

Ground Truth

39

Comparison with Full-Image

• My Experimental Results (SAD+Gradient)

• Lowest V.S. Normalized disparity

40

Radiometric Invariant Stereo Matching Based On Relative

Gradients

Method B.

41

Methods B.

• Goal : Adapt different environmental factors.(Illumination condition)

Effective and robust algorithm

• Relative gradient algorithm + Gaussian weighted function

42

Background

• Lighting Model : – View independent, body reflection

•

•

43

Background

•

•

• Lighting Model :

•

ANCC

44

Method B.

• Cost Computation–

–

–

(i,j)

45

Method B.

• Cost Aggregation–

• Refinement–

– Avoid White and black noises

46

Experimental Results B.

47

48

Experimental Results B.

49

Experimental Results B.

50

Experimental Results B.

• My Experimental Results (SAD+Gradient)• Original V.S.Rerange disparity

51

Experimental Results B.

• Using related gradient intialization

52

Conclusion

Initialization

ADc/SADc

ADg

C-Census

G-Census

???

Aggregation

Weighted-Window

Permeability

Cost-Filter

Arbitrary Support Region

???

Refinement

Lowest Neighbor

Normalizes

Re-Range

Scan-line

???