A Rapid Stereo Matching Algorithm Based on Disparity Interpolation

Gang YaoYong Liu

Bangjun LeiDong Ren

Institute of Intelligent Vision and Image Information

China Three Gorges University

Outline

• Introduction•Method•Disparity Interpolation•Experiments•Conclusion

Introduction

•The disparity and the matching cost are calculated based on stereo matching▫left image as the base image.▫right image as the base image.

•The disparity in this two kind of situations are also optimized.

Introduction

•Matching area and un-matching area can be further marked.

•The disparity interpolation process is done ▫started from the edge of un-matching area.▫stop until all pixels in un-matching area are finished.

Introduction

•The proposed algorithm in this paper on the basis of the adaptive cost function in [13].

• Just to interpolation the results of local matching algorithm ▫maintain the disparity continuity.▫improve the stereo matching speed.

[13]Segment-Based Stereo Matching Using Belief Propagation and a Self-Adapting Dissimilarity Measure

Segment-Based Stereo Matching Using Belief Propagation and a Self-Adapting Dissimilarity Measure

Andreas KlausMario SormannKonrad Karner

VRVis Research Center, Graz, Austria

Introduction

•Color segmentation and Self-adapting matching▫maximize the number of reliable correspondences.

•Self-adapting conception : ▫The scene structure is modeled by a set of planar

surface patches.▫Instead of assigning a disparity value to each pixel, a

disparity plane is assigned to each segment.▫The optimal disparity plane labeling is approximated

by applying belief propagation.

Work-flow Diagram

Local Matching in Pixel Domain

•.

• .

• .

Disparity Plane Estimation

• .

▫step 1 : calculate matching cost.▫step 2 : the disparity plane with the minimum matching cost is assigned to each segment.▫step 3 : segments that are assigned to the disparity plane are grouped.

•Repeating the plane fitting for grouped regions.▫increase the accuracy

Disparity Plane Assignment

•.

• .

• .

Experiments

A Rapid Stereo Matching Algorithm Based on Disparity Interpolation

Back to …

The Proposed Algorithm

Outline

• Introduction•Method

▫Cost function▫Robust matching▫Disparity optimization

•Disparity Interpolation•Experiments•Conclusion

Cost Function

•The cost function includes two parts : ▫C(x,y,d) = (1-ω) × Cdata(x,y,d) + ω × Csmooth(x,y,d)

•Data term▫.

•Smooth term▫.

Cost Function

•Can ensure the gray consistency between the correspondent pixels.

•Can guarantee the smooth consistency in the pixel neighbourhood▫find matching pixel accurately.▫improve the smoothness of disparity.

Robust Matching

•The similarity cost function including ▫the gray consistency.▫the smoothness consistency in pixel neighbourhood.

•The cost function is actually a quadratic function▫like p(x) = x2, ▫its derivatives is p'(x) = 2x.

Robust matching

•The noise affects is linear increase in stereo matching.

•The noise is not control by the function, so the quantizing error and noise is very sensitive in both the data item and smooth item in cost function.

• In order to reduce the affection of noise▫robustness matching function Lorentzian [16].

• the noise is control by Lorentzian function, and the influence is nearly zero

Robust Matching

Robust Matching

•New cost function▫.

• .

Disparity Optimization

•No matter what kind of method, the corresponding points can not be found for a part of the pixels in the image.

•Left occluded and right occluded cannot appear at the same time.

Disparity Optimization

•The process of optimization as follows :

•Step 1 : ▫After robust matching method

right-to-left disparity dR-L(x,y) and left-to-right disparity dL-R(x,y).

value of cost function is calculated respectively as CR-

L(x,y) and CL-R(x,y).

Disparity Optimization

•Step 2 : ▫dR-L(x,y)、 dL-R(x,y)、 CR-L(x,y)、 CL-R will be

optimized.

occluded

Outline

• Introduction•Method•Disparity Interpolation•Experiments•Conclusion

Disparity Interpolation

•Many pixels can not find the corresponding points because of occluded in the image.

•The corresponding disparity can not be calculated in the disparity after optimized.

• the disparity can be divided into▫matching area▫un-matching area

Disparity Interpolation

•The disparity in the un-matching area should be continuity with the disparity in matching area surrounding the un-matching area.

•Disparity interpolation for un-matching area to get the smooth disparity on basis of the optimized disparity.

Disparity Interpolation

•Step 1 : Mark the matching area and un-matching▫1 stands for un-matching area.▫0 stands for matching area.

•Step 2 : Find the edge points in the un-matching area▫scanning the 8 neighbourhood of pixels.▫calculate the disparity of edge pixel.▫mark the point as 0.

•Step 3 : Stop scanning until all the points in un-matching area have been matched, or then return to Step2.

Disparity Interpolation

Outline

• Introduction•Method•Disparity Interpolation•Experiments•Conclusion

Experiments

•The results of each steps

Experiments

•The results of other methods

Experiments

•Compare the efficiency

Outline

• Introduction•Method•Disparity Interpolation•Proposed Algorithm•Experiments•Conclusion

Conclusion

•The dense and smooth disparity can be required.

•The effect of the new algorithm is better than traditional stereo matching algorithm.▫Time complexity can be reduced.▫Efficiency is closer to segment based algorithm.

•The main problem of this algorithm : ▫un-matching area is very big.

Conclusion

•The disparity error of pixels in centre of un-matching area will be bigger because of interpolating operation is start from the edge of un-matching area.