Detection Evolution with Multi-Order Contextual Co-Occurrence

Guang Chen (Missouri)Yuanyuan Ding (Epson)

Jing Xiao (Epson)Tony Han (Missouri)

2

Object Detection

• Sliding Window Based Approach– Classifiers and features are

typically inside the window.

• Context Helps– Context outside the sliding window

can be used to achieve better performances.

3

Context in Computer Vision

• High Level Context– Semantic Context– Geometric Context

• Low Level Context– Pixel Context– Shape Context

• Murphy et al, 2003• Hoiem et al, 2006• Avidan, 2006• Shotton et al, 2006• Rabinovich et al, 2007• Oliva & Torralba, 2007• Heitz & Koller, 2008• Desai et al, 2009• Divvala et al, 2009• Li, Socher & Fei-Fei, 2009• Marszalek et al, 2009• Bao & Savarese, 2010• Yao & Fei-Fei, 2010• Tu & Bai, 2010• Li, Parikh & Chen, 2011• Wolf & Bileschi, 2006• Belongie et al, 2000

[Rabinovich et al, 2007][Yao & Fei-Fei, 2010] [Hoiem et al, 2006]

4

Classification Context for Segmentation

• Spatialboost and Auto-context– Integrate classifier responses from

nearby individual pixels for pixel level segmentation or labeling

Spatial boost [Avidan 2006]Auto-context [Tu & Bai, 2010]

5

Classification Context for Object Detection

• Contextual Boost [Ding & Xiao, 2012]

–Directly uses the detector responses

Adaboost Classification

Based on Image Context

Image Context + Adaboost

Image Context

Multi-scale HOG-LDP for Each Scan

Window

Classification Context

Responses at Scale & Spatial Neighborhood

Adaboost Classification

Based on Augmented

Context

Contextual Boost

6

Co-Occurrence Context

• Can we further exploit co-occurrence information given only detectors for a single object type?

7

Co-Occurrence Context

• Co-Occurrence from Detector Response Map.

8

Our Contribution

• An Effective and Efficient Multi-Order Co-Occurrence Context Representation Using a Single Object Detector.

9

Our Contribution

• An Effective and Efficient Multi-Order Co-Occurrence Context Representation Using a Single Object Detector.

• Multi-Order Contextual Co-Occurrence (MOCO)– 0th order: Classification Context– 1st order: Randomized Binary

Comparison– High order: Co-Occurrence Descriptor

10

Constructing MOCO

0th Order Context

• Directly Using Classifier Responses

Classifier response map (window width=25pixels)Classifier response map (window width=50pixels)Classifier response map (window width=100pixels)

12

0th Order Context

• Define Scale and Space Neighborhood

– Spatial (x, y)– Scale (l)

y

xl

P

13

1st Order Context

• Comparison of Response Values

P

14

1st Order Context

• Randomized Arrangement

15

High Order Context

• 1. Closeness Vector• 2. Histogram

16

High Order Context

• 3. High Order Representation– Tensor Product of Normalized

Histogram

17

Detection Evolution

• Bootstrap training samples using detector responses from the previous iteration.

• Add MOCO context from previous iteration as additional features.

18

Baseline Detector

• Any Object Detection Algorithm Can be Used as Baseline Detector.

19

Baseline Detector

• Any Object Detection Algorithm Can be Used as Baseline Detector.

• Deformable-Parts-Model [Felzenszwalb et al,

2010]

– Inner Context: Parts Models Encodes Relationship between Parts.

–Outer Context: MOCO deals with Co-Occurrence among Scanning Windows

20

Experiments

• Datasets– PASCAL VOC 2007, 20 Object

Categories– Caltech Pedestrian

• Deformable-Parts-Model–Default setting ( 3 components,

each with 1 root and 8 part filters)

21

Experiment – 1st Order

• 1st Order & Context Neighbor Size

22

Experiment – 1st Order

• Pairwise Comparison: Arrangements

23

Experiment – High Order

• High Order Context

–Dimension

24

Experiment – Combinations

• Combinations

• Iterations

25

Comparison on Caltech Dataset

26

Comparison on PASCAL’07

• Mean AP on 20 Categories

27

Conclusion

• An Efficient Context Representation –Only Relying on Detectors for a

Single Object Type– Combining Deformable Parts Model

to Model both inner and Outer Context around Detection Window

• Future Work– Exploit Context With Detectors of

Multiple Object Types?

28

Questions?

Thanks for your attention!