Telefónica I+D

FAST NON-UNIFORM FILTERING WITH SYMMETRIC WEIGHTED INTEGRAL IMAGES

David Marimon

Telefónica Research and Development Barcelona, Spain marimon@tid.es

Telefónica I+D

Non-uniform filtering is costly

 However, non-uniform filtering is a computationally complex task.  For a generic filtering with a kernel k(x):

This requires 2N memory accesses and N multiplications per sample in the input function.

 Non-uniform filters are frequently used in many image processing applications to describe regions or to detect specific features.

Telefónica I+D

 The filtered output of for a box kernel of size NxM can be computed with the following addition:

Note that R implies only 4 memory accesses and 3 additions.   The limitation of box filtering is the uniform shape of the filter.

Related Work

 Fast box filtering on images was introduced by Crow [1] and later used for Haar wavelets by Viola and Jones [2].  The first step consists in pre-computing the integral image:

[1] F.Crow. Summed-area tables for texture mapping. In Proc. Computer Graphics (SIGGRAPH), volume 18, pages 207-212,1984. [2] P. Viola and M. Jones. Rapid object detection using a boosted cascade of simple features. In Proc. CVPR, vol 1, pages 511-518, 2001.

Telefónica I+D

Related Work (II)  Heckbert [1] proposed to approximate a kernel function by the n times

repeated convolution of a box filter. • Filtering can then be computed by only accessing n + 1 samples of the n

times integral of f(x).

 Hussein et al.[2] proposed the Kernel Integral Images (KII). • Find the linear combination of simple functions and the corresponding

weighting factors that generate, or approximate, the desired kernel shape.

 Porikli [3] proposed Reshuffling which exploits the redundancy in the weights of the kernel. • Prior to filtering, two structures must be built from a kernel: the links and

the corresponding weights. [1] P. Heckbert. Filtering by repeated integration. In Proc. Computer Graphics (SIGGRAPH), volume 20, pages 315-321, 1986. [2] M. Hussein, F. Porikli, and L. Davis. Kernel integral images: A framework for fast non uniform filtering. In Proc. CVPR, June 2008. [3] F. Porikli. Reshuffling: a fast algorithm for filtering with arbitrary kernels. In SPIE Electronic Imagining Conference on Real-Time Image Processing, volume 6811, 2008

Telefónica I+D

Symmetric Weighted Integral Images (SWII)

 Integral images for which the contribution of each sample of the input function is weighted. • Weighting is a slope of increasing or decreasing value. • Designed for filtering with non-uniform kernel shapes defined with

slopes of increasing or of decreasing weight.   5 SWII defined for the 2D case:

Telefónica I+D

Filtering with SWII

 Let us define computational complexity per output sample is indicated as C(a; b; c) where • a is the number of memory accesses, • b the additions, and • c, the multiplications.

 Kernel with increasing slope in x: • C(6,5,1)

 Decreasing slope in x:

Telefónica I+D

Filtering with SWII (II)

 Triangle-shaped kernel: C(10,9,0)

 Pyramid-shaped kernel built by adding two triangle-shaped kernels: C(20,19,0)

 Other kernel shapes can be built by translating, overlapping, and adding increasing or decreasing slopes

Telefónica I+D

Experiments

 Pyramid-shaped kernel

 Standard filtering: C ( 2·N·M+1, N·M -1, N·M)  Reshuffling

• Number of redundant coefficients: • C ( 2·N·M+1, N·M, U)

 Kernel Integral Images (KII) • Three KII needed:

• Pre-computation: C(13, 6, 2) • Filtering: C(21, 28, 16)

 SWII • Pre-computation: C(17, 10, 6) • Filtering: C(21, 19, 0)

Telefónica I+D

 Different kernel sizes (N=M)

Results  According to [1], we fix a relative cost of 9 for each memory access

(including array indexing and one addition per access) on a 2D array, 4 for an integer multiplication, 1 for an integer addition.  Reshuffling is 1.1 to 1.5x faster than conventional filtering, both for

different kernels sizes and for multiple scales.

 Performance for multi-scale filtering

[1] F. Porikli, “Reshuffling: A fast algorithm for filtering with arbitrary kernels,” in SPIE EI Conf. on Real-Time Image Processing, 2008,

Telefónica I+D

Application to keypoint detection  Keypoint detection at extrema of Determinant of Hessian in scale space.

 Mikolajczyk (VGG @ Oxford Univeristy) dataset and Repeatibility measure

Telefónica I+D

Conclusions

 Contributions: • Symmetric Weighted Integral Images (SWII) can be used to build a

variety of kernel shapes. • A novel technique to perform non-uniform filtering.

 The results show the speed improvement over Kernel Integral Images (especially relevant for multi-scale filtering) and Reshuffling.

 Successful application to keypoint detection.

Telefónica I+D