weighted joint bilateral filter with slope depth compensation filter for depth map refinement
DESCRIPTION
Weighted Joint Bilateral Filter with Slope Depth Compensation Filter for Depth Map Refinement. Takuya Matsuo, Norishige Fukushima and Yutaka Ishibashi VISAPP 2013 International Conference on Computer Vision Theory and Application. Outline. Introduction Related Works Proposed Method - PowerPoint PPT PresentationTRANSCRIPT
1
Weighted Joint Bilateral Filter with Slope Depth Compensation Filter
for Depth Map Refinement
Takuya Matsuo, Norishige Fukushima and Yutaka Ishibashi
VISAPP 2013 International Conference on Computer Vision Theory and
Application
2
Outline
• Introduction• Related Works• Proposed Method–Weighted Joint Bilateral Filter – Slope Depth Compensation Filter
• Experimental Results• Conclusion
3
INTRODUCTION
4
Introduction
• Goal : Using two filters to get more accurate disparity map in real-time.
• Consideration– Noise reduction – Correct edges – Blurring control
Goal
5
RELATED WORKS
6
Left Image Right Image
Related Works
• Stereo Matching
Related Works
Local Global
Estimate accuracy Low High
Calculation cost Low High
Methods Pixel matching Block matching
(Optimization methods) Graph cuts
Belief propagation
Example
Related Works
• Flow Chart (Local)
1• Matching Cost Computation
2• Cost Aggregation
3• Disparity Map Computation/Optimization
4• Disparity Map Refinement• Disparity Map Refinement
9
Related Works
• Depth map refinement with filter– Median filter– Bilateral filter
Input depth map Output depth map Filter
10
Related Works
• Bilateral filter – Space weight:Near pixels has large weight – Color weight:Similar color pixels has large weight
• Smoothing – Keep edges –Weak in spike noise
•
11
Related Works
• Joint bilateral filter – Add in the reference image – Color weight is calculated by the reference – Keep object edges of the reference
Reference : Low noise Target : High noise Filtered image
12
Related Works
• Joint bilateral filter– Noise reduction O– Correct edge O– Blurring X • Mixed depth values • Spreading error regions
• Multilateral filter– Space + Color + Depth– Boundary recovering X
13
PROPOSED METHOD
14
Proposed Method
• Weighted joint bilateral filter – Noise reduction – Edge correction
• Slope depth compensation filter – Blurring control
15
Weighted Joint Bilateral Filter
•
– 𝐷: Depth value – 𝑝: Coordinate of current pixel – 𝑠: Coordinate of support pixel – 𝑁: Aggregation set of support pixel – 𝑤(), (): Space/color weight 𝑐– 𝜎𝑠,𝜎𝑐: Space/color Gaussian distribution – 𝑅𝑠: Weight map
16
Weighted Joint Bilateral Filter
• Add in the weight map – Controlling amount of influence on a pixel –Weight of the edge and error is small
Joint bilateral filter 𝜎- Mixed depth values 𝜎- Spreading error regions
17
Weighted Joint Bilateral Filter
• Making weight map – Space/color/disparity weight – Sum of nearness of space,
color, and disparity between center pixel and surrounding pixels.
•
18
Weighted Joint Bilateral Filter
• Mask image is made by Speckle Filter– Detecting speckle noise –Weight of speckle region is 0
Red region: speckle noise Weight = 0
19
Weighted Joint Bilateral Filter
20
Slope Depth Compensation Filter
• Weighted joint bilateral filter– Remaining small blurring – Difference between foreground and background
color is small
• Slope depth compensation filter – Reason of blurring is mixed depth value – Convert mixed value to non-blurred candidate
using initial depth map
Removing remaining blur
21
Slope Depth Compensation Filter
•
– X in Dx {INITIAL;WJBF;SDCF}∈
22
Slope Depth Compensation Filter
23
Proposed Method
Initial disparityStereo matching
Noise reduction/ edge correction Weighted Joint Bilateral F.
Blurring control Slope Depth Compensention F.
24
EXPERIMENTAL RESULTS
25
Experimental Results
• Evaluating accuracy improvement for various types of depth maps – Block Matching (BM) – Semi-Global Matching (SGM) – Efficient Large-Scale (ELAS) – Dynamic Programing (DP) – Double Belief Propagation (DBP)
26
Experimental Results
27
Experimental Results
28
29
30
31
Experimental Results
• Comparing proposed method with cost volume refinement(Teddy).
Yang, Q., Wang, L., and Ahuja, N. A constantspace belief propagation algorithm for stereo matching.In Computer Vision and Pattern Recognition(2010).
32 times slower
32
Experimental Results
33
Experimental Results
• Device : Intel Core i7-920 2.93GHz• Comparing running time (ms) of BM plus proposed
filter with selected stereo methods.
34
Experimental Results
35
Experimental Results
• Use the proposed filter for depth maps from Microsoft Kinect.
36
37
CONCLUSION
38
Conclusion
Contribution• The proposed methods can reduce depth noise and
correct object boundary edge without blurring. • Amount of improvement is large when an input
depth map is not accurate.
Future Works• Investigating dependencies of input natural images
and depth maps.