100+ Times Faster Weighted Median Filter[cvpr ‘14]

Presenter: Chang-Ryeol Lee

Introduction

Typical problems in stereo matching Wrong disparity estimates in depth discontinuity regions

Disparity image estimated by our active stereo system [samsung project ’14]

Introduction

One of the ways to get better disparity image Post-processing of estimated disparity image

Initial disparity image Improved disparity map

Introduction

Post processing by using color image is popular and powerful Weighted Median Filtering (WMF) Weighted Bilateral Filtering (WBF) Weighted Mode Filtering (WDF)

Initial disparity image Color image Improved disparity map

Introduction

Post processing by using color image is popular and powerful Weighted Median Filtering (WMF)

Effective to salt and pepper noises Edge-preserving filter

Initial disparity image Color image Improved disparity map

What is WMF?

Based on sliding window strategies

Disparity image

What is WMF?

Based on sliding window strategies Median filtering

Select median value within a window

5 1 3 4 1

1 5 3 8 7

3 6 2 1 9

5 8 3 9 3

1 3 7 4 6

disparity

The number of disparity

Median value

Disparity image

Disparity image

What is WMF?

Based on sliding window strategies Weighted median filtering

Select color weighted median value within a window Weight

Color similarity between center point pixel and neighborhood pixel

5

,

| ( ) ( ) |exp( )p q

r

I Iw

p q

Disparity image Color imagedisparity

Color weight

,p qw

Weighted 히스토그램

I(p)

I(q)D(q)

What is WMF?

Based on sliding window strategies Weighted median filtering

Select color weighted median value within a window Median value computation by cumulative histogram

, ,1 1

1min( )

2

k n

p q p qq qkw w

k n

histogram Cumulative histogram

disparity

Color weight

disparity

Color weight

Limitation of WMF

Nonlinearity

⇒ histogram and cumulative histogram have to be computed within every window

Improvement strategies1) Joint histogram to use box filtering technique

2) Median tracking to cut cumulative histogram procedure

3) Necklace table for fast access in sparse data structure

median{ window1 + window2 } median{ window1 } + median{ window2 }

Fast WMF

Joint histogram 2D histogram composed of disparity and color intensity

Color weighted histogram can be computed by multiplication of joint histogram and color similarity

disparity

Color weight

,p qw

Color weighted histogram

disparity

Col

or in

tens

ity

0

255

100d

i

Joint histogram

H(d,i) #{q | I( ) , D( ) }i d q q

,

| ( ) |w H(d,i)exp( ), i I(q)p q

I i

p

Fast WMF

Joint histogram 2D histogram composed of disparity and color intensity

Only count the number of disparities and corresponding color intensity It means that we can use box filtering technique

Color intensity

disparity

Col

or in

tens

ity

0

255

100d

i

Joint histogram

0

I(q) D(q)

H(d,i) #{q | I( ) , D( ) }i d q q

Disparity image

Fast WMF

Median tracking Insight

Colors in a window are similar to those in neighborhood window So median value is not largely changed as a window moves

Tracking current median value by shifting of median value in previous window

1 0

1 0prev

newprev

k if balancek

k if balance

disparity

Color weight

disparity

Color weight

* k is median value

prevknewk

Previous window Current window

Fast WMF

Median tracking Balance

Difference between left sum and right sum of histogram

l rbalance w w

lw rwk disparity

Color weight

Left sum Right sum

Fast WMF

Median tracking Fast balance computation

To compute balance, all values in joint histogram and corresponding color similarities should be multiplied

Joint histogram

Disparity

Co

lor

inte

nsi

ty

0

255

100

k

i + -

100 255

0 0

| ( ) i |H(d, i)exp( )

d i

Ibalance

p

| ( ) i |exp( )

I

p

.*

Fast WMF

Median tracking Fast balance computation

To compute balance, all values in joint histogram and corresponding color similarities should be multiplied

Joint histogram

Disparity

Co

lor

inte

nsi

ty

0

255

100

k

i + -

100 255

0 0

| ( ) i |H(d, i)exp( )

d i

Ibalance

p

| ( ) i |exp( )

I

p

.*

Fast WMF

Median tracking Fast balance computation

Balance Counting Box (BCB)

Row by Row computation of balance in joint histogram

Joint Histogram

Disparity

Co

lor

inte

nsi

ty

0

255

100

k

i + -

Balance Counting Box (BCB)

(i) #{q | I(q) k} #{q | I(q) k}B

(i)B

Fast WMF

Median tracking Fast balance computation

Balance Counting Box (BCB)

Multiplication of only BCB and corresponding color similarity

255

0

| ( ) i |(i) exp( )

i

Ibalance B

p

| ( ) i |exp( )

I

p.*

Balance Counting Box (BCB)

(i)B

Fast WMF

Experiments Resolution: 640 x 480 Environments: single core, C code Comparison

About 6 times faster

Color image

time (s) speed (fps)

WMF 0.574 1.74

Fast WMF 0.091 10.98

WMF Fast WMFInitial disparity image

Conclusion

Practical issues show up in computer vision community The main idea of this paper can be applied to other histogram-

based application

Box filtering