recognition using regions - eecs at uc berkeley
TRANSCRIPT
![Page 1: Recognition using Regions - EECS at UC Berkeley](https://reader031.vdocuments.us/reader031/viewer/2022011918/61d82b71efd0944bf2114118/html5/thumbnails/1.jpg)
Recognition using Regions
Ch h i G J h Li P bl A b lChunhui Gu, Joseph Lim, Pablo Arbelaez, Jitendra Malik
University of California at BerkeleyUniversity of California at Berkeley
![Page 2: Recognition using Regions - EECS at UC Berkeley](https://reader031.vdocuments.us/reader031/viewer/2022011918/61d82b71efd0944bf2114118/html5/thumbnails/2.jpg)
Detection and SegmentationDetection and Segmentation
2
![Page 3: Recognition using Regions - EECS at UC Berkeley](https://reader031.vdocuments.us/reader031/viewer/2022011918/61d82b71efd0944bf2114118/html5/thumbnails/3.jpg)
Regions as PrimitivesRegions as Primitives
• Encode object scale
• Encode object shape
• Are robust to clutter
• High‐performance detector available
b• Robust recognition machinery to errors
Hoiem et al ICCV 05; Rabinovich et al ICCV 07;
3
Hoiem et al. ICCV 05; Rabinovich et al. ICCV 07;Todorovic & Ahuja CVPR 08; Malisiewicz & Efros CVPR 08;
![Page 4: Recognition using Regions - EECS at UC Berkeley](https://reader031.vdocuments.us/reader031/viewer/2022011918/61d82b71efd0944bf2114118/html5/thumbnails/4.jpg)
High performance Region DetectorHigh‐performance Region Detector
Arbelaez, Maire, Fowlkes, Malik. CVPR 09, , ,Poster session 5
4
![Page 5: Recognition using Regions - EECS at UC Berkeley](https://reader031.vdocuments.us/reader031/viewer/2022011918/61d82b71efd0944bf2114118/html5/thumbnails/5.jpg)
OutlineOutline
• Motivation
• Region Representation
• Algorithm• Algorithm
• Experimental Evaluation
• Conclusion
5
![Page 6: Recognition using Regions - EECS at UC Berkeley](https://reader031.vdocuments.us/reader031/viewer/2022011918/61d82b71efd0944bf2114118/html5/thumbnails/6.jpg)
Region TreeRegion Tree
…
… …
……6
![Page 7: Recognition using Regions - EECS at UC Berkeley](https://reader031.vdocuments.us/reader031/viewer/2022011918/61d82b71efd0944bf2114118/html5/thumbnails/7.jpg)
Bag of RegionsBag of Regions
…
7
![Page 8: Recognition using Regions - EECS at UC Berkeley](https://reader031.vdocuments.us/reader031/viewer/2022011918/61d82b71efd0944bf2114118/html5/thumbnails/8.jpg)
Region DescriptionRegion Description
![Page 9: Recognition using Regions - EECS at UC Berkeley](https://reader031.vdocuments.us/reader031/viewer/2022011918/61d82b71efd0944bf2114118/html5/thumbnails/9.jpg)
OutlineOutline
• Motivation
• Region Representation
• Algorithm• Algorithm
• Experimental Evaluation
• Conclusion
9
![Page 10: Recognition using Regions - EECS at UC Berkeley](https://reader031.vdocuments.us/reader031/viewer/2022011918/61d82b71efd0944bf2114118/html5/thumbnails/10.jpg)
Region based Hough VotingRegion‐based Hough Voting
R t f ti f t h d i• Recover transformation from matched regions
• Transform exemplar bounding box to query
T(x,y,sx,sy)
T(x y s s )Exemplar Query
T(x,y,sx,sy)
10
![Page 11: Recognition using Regions - EECS at UC Berkeley](https://reader031.vdocuments.us/reader031/viewer/2022011918/61d82b71efd0944bf2114118/html5/thumbnails/11.jpg)
Region based VotingRegion‐based Voting
E l 1MeanShift
Clustering
Exemplar 1
Query QueryClustering
lExemplar 2
11
![Page 12: Recognition using Regions - EECS at UC Berkeley](https://reader031.vdocuments.us/reader031/viewer/2022011918/61d82b71efd0944bf2114118/html5/thumbnails/12.jpg)
Algorithm PipelineAlgorithm Pipeline
Exemplars
ImagesImages
Ground truths
Weight learning
Region matching g gbased voting
Query Initial Hypotheses
12
![Page 13: Recognition using Regions - EECS at UC Berkeley](https://reader031.vdocuments.us/reader031/viewer/2022011918/61d82b71efd0944bf2114118/html5/thumbnails/13.jpg)
Weight Learning FormulationWeight Learning ‐ Formulation
D(I,J) D(I,K)Exemplar IExemplar J Exemplar Kpp p
D(I,J) = Σi wi ∙ diJDefine: D(I,K) > D(I,J)Want:
Max-margin formulation results in a sparse solutionMax margin formulation results in a sparse solution.
F Si & M lik NIPS 06
13
Frome, Singer & Malik. NIPS 06
![Page 14: Recognition using Regions - EECS at UC Berkeley](https://reader031.vdocuments.us/reader031/viewer/2022011918/61d82b71efd0944bf2114118/html5/thumbnails/14.jpg)
Weight Learning ResultsWeight Learning ‐ Results
S ( 30%) t bl i d• Sparse (<30%), non‐repeatable regions are removedMore
DiscriminativeLess
2.2194 2.2156 1.9564 0 0 0Discriminative Discriminative
2.22 2.22 1.96 0 00Weights = 2.22 2.22 1.96 0 00Weights
2.91 1.45 00.94 00Weights =
14
![Page 15: Recognition using Regions - EECS at UC Berkeley](https://reader031.vdocuments.us/reader031/viewer/2022011918/61d82b71efd0944bf2114118/html5/thumbnails/15.jpg)
Algorithm PipelineAlgorithm Pipeline
Exemplars
ImagesImages
Ground truthsDetection
Verification classifierWeight learning
Region matching Constrained g gbased voting segmenter
Query Initial Hypotheses Segmentation
15
![Page 16: Recognition using Regions - EECS at UC Berkeley](https://reader031.vdocuments.us/reader031/viewer/2022011918/61d82b71efd0944bf2114118/html5/thumbnails/16.jpg)
Initial Object/Background LabelsInitial Object/Background Labels
l Transformed MaskExemplar Transformed Mask
Initial Labels
Query Matched Part+
Query Matched Part
: Object label: Background label: Unknown label
16
: Unknown labelFully automatic unlike interactive use of Graph Cuts, e.g. Blake et al. ECCV 04
![Page 17: Recognition using Regions - EECS at UC Berkeley](https://reader031.vdocuments.us/reader031/viewer/2022011918/61d82b71efd0944bf2114118/html5/thumbnails/17.jpg)
Propagate Object/Background LabelsPropagate Object/Background Labels
A b l d C h CVPR 08Initial Labels Final Segmentation
17
Arbelaez and Cohen. CVPR 08
![Page 18: Recognition using Regions - EECS at UC Berkeley](https://reader031.vdocuments.us/reader031/viewer/2022011918/61d82b71efd0944bf2114118/html5/thumbnails/18.jpg)
OutlineOutline
• Motivation
• Region Representation
• Algorithm• Algorithm
• Experimental Evaluation (ETHZ shape, Caltech 101)
• Conclusion
18
![Page 19: Recognition using Regions - EECS at UC Berkeley](https://reader031.vdocuments.us/reader031/viewer/2022011918/61d82b71efd0944bf2114118/html5/thumbnails/19.jpg)
ETHZ Shape (Ferrari et al 06)ETHZ Shape (Ferrari et al. 06)
• 255 images of 5 diverse shape‐based categories.g
19
![Page 20: Recognition using Regions - EECS at UC Berkeley](https://reader031.vdocuments.us/reader031/viewer/2022011918/61d82b71efd0944bf2114118/html5/thumbnails/20.jpg)
ApplelogosApplelogos
20
![Page 21: Recognition using Regions - EECS at UC Berkeley](https://reader031.vdocuments.us/reader031/viewer/2022011918/61d82b71efd0944bf2114118/html5/thumbnails/21.jpg)
BottlesBottles
21
![Page 22: Recognition using Regions - EECS at UC Berkeley](https://reader031.vdocuments.us/reader031/viewer/2022011918/61d82b71efd0944bf2114118/html5/thumbnails/22.jpg)
GiraffesGiraffes
22
![Page 23: Recognition using Regions - EECS at UC Berkeley](https://reader031.vdocuments.us/reader031/viewer/2022011918/61d82b71efd0944bf2114118/html5/thumbnails/23.jpg)
MugsMugs
23
![Page 24: Recognition using Regions - EECS at UC Berkeley](https://reader031.vdocuments.us/reader031/viewer/2022011918/61d82b71efd0944bf2114118/html5/thumbnails/24.jpg)
SwansSwans
24
![Page 25: Recognition using Regions - EECS at UC Berkeley](https://reader031.vdocuments.us/reader031/viewer/2022011918/61d82b71efd0944bf2114118/html5/thumbnails/25.jpg)
ResultsResults
Detection: (recall at 0 3 fppi Pascal criterion)
Method Ferrari et al. 07 Voting only Voting + Verify
Detection: (recall at 0.3 fppi, Pascal criterion)
Recall Rate (%) 67.2 82.9±4.3 87.1±2.8
Segmentation: (pixel-wise average precision)
Method Bounding box SegmenterMethod Bounding box Segmenter
AP Rate (%) 51.6±2.5 75.7±3.2
All results are averaged over 5 training/test splits.
25
![Page 26: Recognition using Regions - EECS at UC Berkeley](https://reader031.vdocuments.us/reader031/viewer/2022011918/61d82b71efd0944bf2114118/html5/thumbnails/26.jpg)
ResultsResults
Number of windows compared to window scanning:
Categories (#) Scanned Regions Bounding Catego es ( ) Sca edwindows
eg o s ou d gboxes
Applelogos ~30,000 115 3.1
Bottles ~1,500 168 1.1
Giraffes ~14,000 156 6.9
Mugs ~16,000 189 5.3
Swans ~10,000 132 2.3
All results are averaged over 5 training/test splits.
26
![Page 27: Recognition using Regions - EECS at UC Berkeley](https://reader031.vdocuments.us/reader031/viewer/2022011918/61d82b71efd0944bf2114118/html5/thumbnails/27.jpg)
Caltech 101 (Fei Fei et al 04)Caltech 101 (Fei‐Fei et al. 04)
• 102 classes, 31‐800 images/class
27
![Page 28: Recognition using Regions - EECS at UC Berkeley](https://reader031.vdocuments.us/reader031/viewer/2022011918/61d82b71efd0944bf2114118/html5/thumbnails/28.jpg)
Image RepresentationImage RepresentationFeature vectors
Region 1
Feature vectors
Region 2Input
Region 3
Sh
Region N
ShapeColorTexturePoint-basedPoint based
28
![Page 29: Recognition using Regions - EECS at UC Berkeley](https://reader031.vdocuments.us/reader031/viewer/2022011918/61d82b71efd0944bf2114118/html5/thumbnails/29.jpg)
Performance with cue combinationPerformance with cue combination
Image Cues 15 train (%) 30 train (%)
(R) Shape (contour) 55 1 60 4(R) Shape (contour) 55.1 60.4
(R) Shape (edge) 42.9 48.0
(R) Color 27.1 27.2
(R) Texture 31.4 32.7
(R) Combination 59.0 65.2
(IP) Geometric Blur 58.4 63.2( )
(R) Shape + (IP) GB 65.0 73.1
29
![Page 30: Recognition using Regions - EECS at UC Berkeley](https://reader031.vdocuments.us/reader031/viewer/2022011918/61d82b71efd0944bf2114118/html5/thumbnails/30.jpg)
ConclusionConclusion
• Introduce a novel Hough voting scheme using REGIONS as primitives
• Solves detection and segmentation in a combined framework
• Cue combination improves recognition performance
30
![Page 31: Recognition using Regions - EECS at UC Berkeley](https://reader031.vdocuments.us/reader031/viewer/2022011918/61d82b71efd0944bf2114118/html5/thumbnails/31.jpg)
Thank you!Thank you!