![Page 1: Object Recognition as Ranking Holistic Figure-Ground Hypotheses Fuxin Li and Joao Carreira and Cristian Sminchisescu 1](https://reader035.vdocuments.us/reader035/viewer/2022062413/5a4d1b2c7f8b9ab059999671/html5/thumbnails/1.jpg)
1
Object Recognition as Ranking Holistic Figure-Ground
Hypotheses
Fuxin Li and Joao Carreira and Cristian Sminchisescu
![Page 2: Object Recognition as Ranking Holistic Figure-Ground Hypotheses Fuxin Li and Joao Carreira and Cristian Sminchisescu 1](https://reader035.vdocuments.us/reader035/viewer/2022062413/5a4d1b2c7f8b9ab059999671/html5/thumbnails/2.jpg)
2
Outline Introduction Method Overview Segment Categorization Segment Post-Processing Experiment Conclusion
![Page 3: Object Recognition as Ranking Holistic Figure-Ground Hypotheses Fuxin Li and Joao Carreira and Cristian Sminchisescu 1](https://reader035.vdocuments.us/reader035/viewer/2022062413/5a4d1b2c7f8b9ab059999671/html5/thumbnails/3.jpg)
3
Introduction Object detector : Top-down approaches
![Page 4: Object Recognition as Ranking Holistic Figure-Ground Hypotheses Fuxin Li and Joao Carreira and Cristian Sminchisescu 1](https://reader035.vdocuments.us/reader035/viewer/2022062413/5a4d1b2c7f8b9ab059999671/html5/thumbnails/4.jpg)
4
Introduction Semantic segmentation results
produced by our algorithm
![Page 5: Object Recognition as Ranking Holistic Figure-Ground Hypotheses Fuxin Li and Joao Carreira and Cristian Sminchisescu 1](https://reader035.vdocuments.us/reader035/viewer/2022062413/5a4d1b2c7f8b9ab059999671/html5/thumbnails/5.jpg)
5
Introduction Ideally segment1. Can model entire object
2. At least sufficiently distinct parts of them
![Page 6: Object Recognition as Ranking Holistic Figure-Ground Hypotheses Fuxin Li and Joao Carreira and Cristian Sminchisescu 1](https://reader035.vdocuments.us/reader035/viewer/2022062413/5a4d1b2c7f8b9ab059999671/html5/thumbnails/6.jpg)
6
![Page 7: Object Recognition as Ranking Holistic Figure-Ground Hypotheses Fuxin Li and Joao Carreira and Cristian Sminchisescu 1](https://reader035.vdocuments.us/reader035/viewer/2022062413/5a4d1b2c7f8b9ab059999671/html5/thumbnails/7.jpg)
7
Introduction Constrained Parametric Min Cuts
algorithm (CPMC) [6]
![Page 8: Object Recognition as Ranking Holistic Figure-Ground Hypotheses Fuxin Li and Joao Carreira and Cristian Sminchisescu 1](https://reader035.vdocuments.us/reader035/viewer/2022062413/5a4d1b2c7f8b9ab059999671/html5/thumbnails/8.jpg)
8
Method Overview
This paper focus
CPMC
Number of segment
![Page 9: Object Recognition as Ranking Holistic Figure-Ground Hypotheses Fuxin Li and Joao Carreira and Cristian Sminchisescu 1](https://reader035.vdocuments.us/reader035/viewer/2022062413/5a4d1b2c7f8b9ab059999671/html5/thumbnails/9.jpg)
9
Method Overview Recognition framework1. Segment categorization
2. Segment post-processing
![Page 10: Object Recognition as Ranking Holistic Figure-Ground Hypotheses Fuxin Li and Joao Carreira and Cristian Sminchisescu 1](https://reader035.vdocuments.us/reader035/viewer/2022062413/5a4d1b2c7f8b9ab059999671/html5/thumbnails/10.jpg)
10
Segment categorization1. Scoring function2. Sort3. Combine high-rank segment
![Page 11: Object Recognition as Ranking Holistic Figure-Ground Hypotheses Fuxin Li and Joao Carreira and Cristian Sminchisescu 1](https://reader035.vdocuments.us/reader035/viewer/2022062413/5a4d1b2c7f8b9ab059999671/html5/thumbnails/11.jpg)
11
![Page 12: Object Recognition as Ranking Holistic Figure-Ground Hypotheses Fuxin Li and Joao Carreira and Cristian Sminchisescu 1](https://reader035.vdocuments.us/reader035/viewer/2022062413/5a4d1b2c7f8b9ab059999671/html5/thumbnails/12.jpg)
12
Segment post-processing
COW
![Page 13: Object Recognition as Ranking Holistic Figure-Ground Hypotheses Fuxin Li and Joao Carreira and Cristian Sminchisescu 1](https://reader035.vdocuments.us/reader035/viewer/2022062413/5a4d1b2c7f8b9ab059999671/html5/thumbnails/13.jpg)
13
Segment Categorization Multiple Segment and Features Learning Scoring Functions with
Regression Learning the Kernel Hyperparameters Compare with Structural SVM
![Page 14: Object Recognition as Ranking Holistic Figure-Ground Hypotheses Fuxin Li and Joao Carreira and Cristian Sminchisescu 1](https://reader035.vdocuments.us/reader035/viewer/2022062413/5a4d1b2c7f8b9ab059999671/html5/thumbnails/14.jpg)
14
Multiple Segment and Features
Model object appearance : 1. Extracted four bag of words of SIFT 2. Two on foreground 3. Two on Background, aim to improve
recognition
![Page 15: Object Recognition as Ranking Holistic Figure-Ground Hypotheses Fuxin Li and Joao Carreira and Cristian Sminchisescu 1](https://reader035.vdocuments.us/reader035/viewer/2022062413/5a4d1b2c7f8b9ab059999671/html5/thumbnails/15.jpg)
15
Multiple Segment and Features
Encode shape information :1. A bag of word of local sharp
contexts [2] : measure similarity between shapes
2. Three pyramid HOGs [5] : classifying images by the object
categories they contain
![Page 16: Object Recognition as Ranking Holistic Figure-Ground Hypotheses Fuxin Li and Joao Carreira and Cristian Sminchisescu 1](https://reader035.vdocuments.us/reader035/viewer/2022062413/5a4d1b2c7f8b9ab059999671/html5/thumbnails/16.jpg)
16
Multiple Segment and Features
Chi-square kernel :
Computed from each histogram feature and use a weighted sum of such kernel for regression
![Page 17: Object Recognition as Ranking Holistic Figure-Ground Hypotheses Fuxin Li and Joao Carreira and Cristian Sminchisescu 1](https://reader035.vdocuments.us/reader035/viewer/2022062413/5a4d1b2c7f8b9ab059999671/html5/thumbnails/17.jpg)
17
Learning Scoring Functions with Regression
: Image I with ground truth segments
: Segmentation algorithm provides a set
of segment : Denote the K object
categories : Indicator function
![Page 18: Object Recognition as Ranking Holistic Figure-Ground Hypotheses Fuxin Li and Joao Carreira and Cristian Sminchisescu 1](https://reader035.vdocuments.us/reader035/viewer/2022062413/5a4d1b2c7f8b9ab059999671/html5/thumbnails/18.jpg)
18
Learning Scoring Functions with Regression
Quality function :
Measure overlap with all denote
the value for and is the maximal overlap with ground truth segments belonging to , and do not appear
![Page 19: Object Recognition as Ranking Holistic Figure-Ground Hypotheses Fuxin Li and Joao Carreira and Cristian Sminchisescu 1](https://reader035.vdocuments.us/reader035/viewer/2022062413/5a4d1b2c7f8b9ab059999671/html5/thumbnails/19.jpg)
19
Learning Scoring Functions with Regression
Learn the function for each :
use nonlinear SVR(Support Vector Regression) to regress against ,the features extracted from
![Page 20: Object Recognition as Ranking Holistic Figure-Ground Hypotheses Fuxin Li and Joao Carreira and Cristian Sminchisescu 1](https://reader035.vdocuments.us/reader035/viewer/2022062413/5a4d1b2c7f8b9ab059999671/html5/thumbnails/20.jpg)
20
Learning Scoring Functions with Regression
Use kernel trick : : support vector from training set : obtained by the SVR optimizer : maximal score of the
segment : final class of the
segment
![Page 21: Object Recognition as Ranking Holistic Figure-Ground Hypotheses Fuxin Li and Joao Carreira and Cristian Sminchisescu 1](https://reader035.vdocuments.us/reader035/viewer/2022062413/5a4d1b2c7f8b9ab059999671/html5/thumbnails/21.jpg)
21
Learning the Kernel Hyperparameters
Fundamental equation (3) is infeasible to estimate all kernel hyperparameter via grid search
Use subset of data comprised segments that best overlap each ground truth segment
![Page 22: Object Recognition as Ranking Holistic Figure-Ground Hypotheses Fuxin Li and Joao Carreira and Cristian Sminchisescu 1](https://reader035.vdocuments.us/reader035/viewer/2022062413/5a4d1b2c7f8b9ab059999671/html5/thumbnails/22.jpg)
22
Compare with Structural SVM The structural SVM(in [3])
formulation for sliding window prediction is :
![Page 23: Object Recognition as Ranking Holistic Figure-Ground Hypotheses Fuxin Li and Joao Carreira and Cristian Sminchisescu 1](https://reader035.vdocuments.us/reader035/viewer/2022062413/5a4d1b2c7f8b9ab059999671/html5/thumbnails/23.jpg)
23
Connections with Structural SVM
Our algorithm VS Structural SVM Structural SVM score the bounding box
and Our algorithm score the segment
Important advantage1. Guarantee the highest rank for the
ground truth2. Correct ranking for all segment
![Page 24: Object Recognition as Ranking Holistic Figure-Ground Hypotheses Fuxin Li and Joao Carreira and Cristian Sminchisescu 1](https://reader035.vdocuments.us/reader035/viewer/2022062413/5a4d1b2c7f8b9ab059999671/html5/thumbnails/24.jpg)
24
Segment Post-Processing Simple decision rule : avoid the post-
processing and direct choose the segment , cannot detect multiple objects
Our methodology : weighted consolidation of segment and sequential interpretation strategy
![Page 25: Object Recognition as Ranking Holistic Figure-Ground Hypotheses Fuxin Li and Joao Carreira and Cristian Sminchisescu 1](https://reader035.vdocuments.us/reader035/viewer/2022062413/5a4d1b2c7f8b9ab059999671/html5/thumbnails/25.jpg)
25
Segment Post-Processing
![Page 26: Object Recognition as Ranking Holistic Figure-Ground Hypotheses Fuxin Li and Joao Carreira and Cristian Sminchisescu 1](https://reader035.vdocuments.us/reader035/viewer/2022062413/5a4d1b2c7f8b9ab059999671/html5/thumbnails/26.jpg)
26
Segment Post-Processing To decide which segments to
combine
Consider segment with intersection
> 0.75 for combination
![Page 27: Object Recognition as Ranking Holistic Figure-Ground Hypotheses Fuxin Li and Joao Carreira and Cristian Sminchisescu 1](https://reader035.vdocuments.us/reader035/viewer/2022062413/5a4d1b2c7f8b9ab059999671/html5/thumbnails/27.jpg)
27
Segment Post-Processing1. Highest-scoring segment as seed2. Group segments that intersect it3. Generated a final mask 4. Proceed with the next higher rank
segment5. Choose segment that are not
overlapping with 3
![Page 28: Object Recognition as Ranking Holistic Figure-Ground Hypotheses Fuxin Li and Joao Carreira and Cristian Sminchisescu 1](https://reader035.vdocuments.us/reader035/viewer/2022062413/5a4d1b2c7f8b9ab059999671/html5/thumbnails/28.jpg)
28
Segment Post-Processing Generate the score for the pixels in
the mask by (9), only pixels with score > 0.65 are displayed in the mask.
![Page 29: Object Recognition as Ranking Holistic Figure-Ground Hypotheses Fuxin Li and Joao Carreira and Cristian Sminchisescu 1](https://reader035.vdocuments.us/reader035/viewer/2022062413/5a4d1b2c7f8b9ab059999671/html5/thumbnails/29.jpg)
29
Experiments Classification : Caltech-101 Detection : ETHZ Shape classes Segmentation : VOC 2009
![Page 30: Object Recognition as Ranking Holistic Figure-Ground Hypotheses Fuxin Li and Joao Carreira and Cristian Sminchisescu 1](https://reader035.vdocuments.us/reader035/viewer/2022062413/5a4d1b2c7f8b9ab059999671/html5/thumbnails/30.jpg)
30
Classification
![Page 31: Object Recognition as Ranking Holistic Figure-Ground Hypotheses Fuxin Li and Joao Carreira and Cristian Sminchisescu 1](https://reader035.vdocuments.us/reader035/viewer/2022062413/5a4d1b2c7f8b9ab059999671/html5/thumbnails/31.jpg)
31
Classification
![Page 32: Object Recognition as Ranking Holistic Figure-Ground Hypotheses Fuxin Li and Joao Carreira and Cristian Sminchisescu 1](https://reader035.vdocuments.us/reader035/viewer/2022062413/5a4d1b2c7f8b9ab059999671/html5/thumbnails/32.jpg)
32
Detection
![Page 33: Object Recognition as Ranking Holistic Figure-Ground Hypotheses Fuxin Li and Joao Carreira and Cristian Sminchisescu 1](https://reader035.vdocuments.us/reader035/viewer/2022062413/5a4d1b2c7f8b9ab059999671/html5/thumbnails/33.jpg)
33
Detection
![Page 34: Object Recognition as Ranking Holistic Figure-Ground Hypotheses Fuxin Li and Joao Carreira and Cristian Sminchisescu 1](https://reader035.vdocuments.us/reader035/viewer/2022062413/5a4d1b2c7f8b9ab059999671/html5/thumbnails/34.jpg)
34
Detection
![Page 35: Object Recognition as Ranking Holistic Figure-Ground Hypotheses Fuxin Li and Joao Carreira and Cristian Sminchisescu 1](https://reader035.vdocuments.us/reader035/viewer/2022062413/5a4d1b2c7f8b9ab059999671/html5/thumbnails/35.jpg)
35
OWT-UCM Masks
![Page 36: Object Recognition as Ranking Holistic Figure-Ground Hypotheses Fuxin Li and Joao Carreira and Cristian Sminchisescu 1](https://reader035.vdocuments.us/reader035/viewer/2022062413/5a4d1b2c7f8b9ab059999671/html5/thumbnails/36.jpg)
36
Segmentation
Bounding box
![Page 37: Object Recognition as Ranking Holistic Figure-Ground Hypotheses Fuxin Li and Joao Carreira and Cristian Sminchisescu 1](https://reader035.vdocuments.us/reader035/viewer/2022062413/5a4d1b2c7f8b9ab059999671/html5/thumbnails/37.jpg)
37
Conclusion CPMC Categorization Post-
processing Achieve good performance Future work : improve the scalability
to be able to process hundreds of thousands of image