An eXtended Center-Symmetric Local Binary Pattern for Background

Modeling and Subtraction in Videos

Caroline Silva, Thierry Bouwmans, Carl Frelicot

March 14, 2015 - Berlin

Summary

2/17

1 Introduction: Background Subtraction

2 Brief overview of LBP and its variants

3 The XCS-LBP Descriptor

4 Experimental Results

5 Conclusion and Future Research

Introduction

3/17

Background Subtraction

Figure: Block diagram of the background subtraction process.

Challenging situations

llumination changes, dynamic backgrounds, camera jitter, noise and shadows.

Common features

Color features, Edge features, Stereo features, Motion features, Texture features

A variety of local texture descriptors recently have attracted great attention for background modeling,especially the Local Binary Pattern (LBP) because it’s simplicity and speed to computation.

Summary

4/17

1 Introduction: Background Subtraction

2 Brief overview of LBP and its variants

3 The XCS-LBP Descriptor

4 Experimental Results

5 Conclusion and Future Research

Brief overview of LBP and its variants

5/17

LBP descriptor

Figure: An example of LBP computation.

The LBP works in a 3×3 pixel block of an image. Thepixels in this block are thresholded by its center pixelvalue, multiplied by powers of two and then summedto obtain a label for the center pixel. The value LBPcode of a pixel (xc ,yc) is given by:

LBPP ,R =P−1

∑p=0

s (gp −gc)2p , (1)

where gc and gp denote the gray value of the centralpixel and its neighbors, R is the radius of theneighborhood and P is the radius of the neighborhood.The function s(x) is defined as follows:

s(x) =

{1 x ≥ 0

0 otherwise.(2)

Brief overview of LBP and its variants

6/17

LBP’s variants

Figure: Comparison of LBP and variants.

Brief overview of LBP and its variants

7/17

CS-LBP descriptor (Heikkila et al., 2009)

Figure: The CS-LBP descriptor.

The underlying idea of CS-LBP is to compare the graylevels of pairs of pixels in centered symmetricdirections instead of comparing the central pixel to itsneighbors. The CS-LBP operator is given by:

CS−LBPP,R(c) =(P/2)−1

∑i=0

s(gi −gi+(P/2))2i (3)

where gi and gi+(P/2) are the gray values ofcenter-symmetric pairs of pixels, and s is thethresholding function defined as:

s(x) =

{1 if x > T

0 otherwise(4)

where T is a user-defined threshold.

Summary

8/17

1 Introduction: Background Subtraction

2 Brief overview of LBP and its variants

3 The XCS-LBP Descriptor

4 Experimental Results

5 Conclusion and Future Research

Proposed XCS-LBP

9/17

Figure: The XCS-LBP descriptor.

The XCS-LBP (eXtended CS-LBP), expresses as:

XCS−LBPP,R(c) =(P/2)−1

∑i=0

s (g1(i,c)+g2(i,c))2i

(5)

The function s(x1 + x2) is defined as follows:

s(x1 + x2) =

{1 if (x1 + x2)≥ 0

0 otherwise.(6)

and where g1(i,c) and g2(i,c) are defined by:{g1(i,c) = (gi −gi+(P/2))+gc

g2(i,c) = (gi −gc) (gi+(P/2)−gc)(7)

The main advantage

Produces a small histogram as CS-LBP,but it extracts more image details.

Summary

10/17

1 Introduction: Background Subtraction

2 Brief overview of LBP and its variants

3 The XCS-LBP Descriptor

4 Experimental Results

5 Conclusion and Future Research

Experimental Results

11/17

We’ve compared XCS-LBP with three other texture descriptors among the reviewed ones,namely :

Original LBP Ojala et al. (2002),

CS-LBP Heikkila et al. (2009) and

CS-LDP Xue et al. (2011).

and we evaluate the performance with two popular background subtraction methods:

Adaptive Background Learning (ABL) (also know as Running Average) and

Gaussian Mixture Models (GMM).

The BMC (Background Models Challenge) data set of Vacavant et al. (2012) was chosen, and itcontains several synthetic and real world videos of outdoor situations (urban scenes).

Experimental Results

12/17

Figure: Background subtraction results using the GMM method on real world videos of the BMC – (a) originalframe, (b) ground truth, (c) LBP, (d) CS-LBP, (e) CS-LDP and (f) proposed XCS-LBP.

Figure: Performance of the different descriptors on real world videos of the BMC using the GMM method.

Experimental Results

13/17

Speed Comparison

MacBook Pro (OS X 10.9.4, 2.2 GHz Intel Core i7 and 8 GB - 1333 MHz DDR3) withMATLAB R2013a.

Elapsed CPU times needed to segment the foreground masks by ABL and GMM methods,averaged over nine real videos of BMC data set.

The reference is the fastest descriptor (original LBP), and the times are divided by LBPones.

XCS-LBP shows slightly better time performance than both CS-LBP and CS-LDP.

Figure: Elapsed CPU times over LBP times

Summary

14/17

1 Introduction: Background Subtraction

2 Brief overview of LBP and its variants

3 The XCS-LBP Descriptor

4 Experimental Results

5 Conclusion and Future Research

Conclusion and Future Research

15/17

The experimental results show that the XCS-LBP outperforms qualitatively andquantitatively its direct competitors, making it a serious candidate for the backgroundsubtraction task in computer vision applications.

The XCS-LBP produces a shorter histogram and it is more tolerant to illuminationchanges and robust to noise.

Future works will explore how to extend the proposed descriptor to include temporalrelationships between neighboring pixels.

Source code and related libraries:XCS- LBP Descriptor: http://lolynepacheco.wix.com/carolinesilva

BGSLibrary (Sobral, 2013): http://github.com/andrewssobral/bgslibrary

LRSLibrary (Sobral et al., 2014): http://github.com/andrewssobral/lrslibrary

16/17

THANK YOU!!!

References

17/17

Heikkila, M., Pietikainen, M., and Schmid, C. (2009). Description of interest regions with local binary patterns. Pattern Recognition, 42:425–436.

Ojala, T., Pietikainen, M., and Maenpaa, T. (2002). Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans. onPattern Analysis and Machine Intelligence, pages 971–987.

Sobral, A. (2013). BGSLibrary: An opencv c++ background subtraction library. In IX Workshop de VisA£o Computacional (WVC’2013), Rio de Janeiro, Brazil.

Sobral, A., Baker, C. G., Bouwmans, T., and Zahzah, E. (2014). Incremental and multi-feature tensor subspace learning applied for background modeling andsubtraction. In International Conference on Image Analysis and Recognition (ICIAR’14). Lecture Notes in Computer Science (Springer LNCS).

Xue, G., Song, L., Sun, J., and Wu, M. (2011). Hybrid center-symmetric local pattern for dynamic background subtraction. In IEEE Int. Conf. on Multimedia andExpo, pages 1–6.