detecting object instances without discriminative featuresehsiao/thesis/ehsiao_thesis_slides.pdf ·...
TRANSCRIPT
![Page 1: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/1.jpg)
Detecting Object Instances Without Discriminative Features
Edward Hsiao
June 19, 2013
Thesis Committee: Martial Hebert, Chair
Alexei Efros Takeo Kanade
Andrew Zisserman, University of Oxford 1
![Page 2: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/2.jpg)
Object Instance Detection
Find this object under arbitrary viewpoint, lighting, clutter and occlusions 2
![Page 3: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/3.jpg)
3
![Page 4: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/4.jpg)
4
![Page 5: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/5.jpg)
Robotic Manipulation
5
![Page 6: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/6.jpg)
Scene Understanding
6
![Page 7: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/7.jpg)
Scene Understanding
Stove Refrigerator
Microwave Coffee maker
Paper towel
Dishwasher
Faucet
7
![Page 8: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/8.jpg)
Visual Search
8
![Page 9: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/9.jpg)
Recognition Using Discriminative Features
model test image
9
[SIFT, Lowe 2004]
![Page 10: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/10.jpg)
Extract Keypoints
test image
10
model
[SIFT, Lowe 2004]
![Page 11: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/11.jpg)
Generate 1-To-1 Correspondences
test image
11
model
[SIFT, Lowe 2004]
![Page 12: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/12.jpg)
Enforce Geometric Constraints
test image
12
model
[SIFT, Lowe 2004]
![Page 13: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/13.jpg)
Recognized Object
test image
13
model
[SIFT, Lowe 2004]
![Page 14: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/14.jpg)
Failure of Feature Matching
14
test image model
0 correct correspondences
![Page 15: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/15.jpg)
Overview Lack of Discriminative Features
Ambiguous Keypoint Features
Feature-poor objects
Occlusions
15
![Page 16: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/16.jpg)
Overview Lack of Discriminative Features
Ambiguous Keypoint Features
Feature-poor objects
Occlusions
16
![Page 17: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/17.jpg)
Ambiguous Keypoint Features
17
![Page 18: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/18.jpg)
Repeated Patterns
18
![Page 19: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/19.jpg)
Failure of Discriminative Matching
Geometric model
mdesc2
Model descriptors
mdesc1
.
.
.
Image keypoint descriptor
19
![Page 20: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/20.jpg)
Failure of Discriminative Matching
Geometric model
mdesc2
Model descriptors
mdesc1
.
.
.
Image keypoint descriptor
? or
One-to-one matching
20
![Page 21: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/21.jpg)
Failure of Discriminative Matching
Geometric model
mdesc2
Model descriptors
mdesc1
.
.
.
Image keypoint descriptor
? or
One-to-one matching
Most approaches discard ambiguous features 21
![Page 22: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/22.jpg)
Quantized Matching
Geometric model
qdesc2
Quantized model descriptors
qdesc1
.
.
.
Image keypoint descriptor
22
![Page 23: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/23.jpg)
Quantized Matching
Geometric model
qdesc2
Quantized model descriptors
qdesc1
.
.
.
Image keypoint descriptor
Quantized matching
Preserve ambiguity of match until geometric verification 23
![Page 24: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/24.jpg)
Detection Performance
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
Average Precision (higher is better)
24
CMU Grocery Dataset
620 images, 10 household objects
one-to-one matching
[Collet et al. 2009]
quantized matching
![Page 25: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/25.jpg)
Failure of Feature Matching
test image
25 0 correct correspondences
model
![Page 26: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/26.jpg)
Keypoint Comparison
Success Failure
26
![Page 27: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/27.jpg)
Uninformative Keypoints
27
![Page 28: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/28.jpg)
Uninformative Keypoints
28
![Page 29: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/29.jpg)
Uninformative Keypoints
29
![Page 30: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/30.jpg)
“Informative” Keypoints
980 keypoints 10 keypoints
Keypoints contained entirely within the object 30
![Page 31: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/31.jpg)
“Informative” Keypoints
980 keypoints 10 keypoints
Keypoints due to specularities 31
![Page 32: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/32.jpg)
Less keypoints More keypoints
Feature-richness
32
![Page 33: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/33.jpg)
Less keypoints More keypoints
Feature-richness
33
![Page 34: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/34.jpg)
Less keypoints More keypoints
Feature-richness
34
![Page 35: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/35.jpg)
Less keypoints More keypoints
Feature-richness
35
![Page 36: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/36.jpg)
Feature Matching Experiment
36
![Page 37: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/37.jpg)
Feature Matching Experiment
37
![Page 38: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/38.jpg)
Feature Matching Experiment
38
![Page 39: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/39.jpg)
Feature Matching Experiment
39
At least 5 good correspondences between all pairs of images
![Page 40: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/40.jpg)
Less keypoints More keypoints
Works Fails
40
![Page 41: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/41.jpg)
Less keypoints More keypoints
Works Fails
41
![Page 42: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/42.jpg)
Less keypoints More keypoints
Works Fails
42
![Page 43: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/43.jpg)
Less keypoints More keypoints
Feature-rich Feature-poor
43
![Page 44: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/44.jpg)
Less keypoints More keypoints
Feature-rich Feature-poor
44
![Page 45: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/45.jpg)
Overview Lack of Discriminative Features
Ambiguous Keypoint Features
Feature-poor objects
Occlusions
45
![Page 46: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/46.jpg)
46
![Page 47: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/47.jpg)
Feature-poor Objects Shape Matching
Template shape Input window Matched shape
47
![Page 48: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/48.jpg)
Representing Feature-poor Objects
Sparse Edge Points [Berg 2005], [Leordeanu 2007],
[Duchenne 2009], [Hinterstoisser 2011]
Lines & Contour Fragments [Ferrari 2006 & 2008],
[Opelt 2006], [Srinivasan 2010]
Histogram of Oriented Gradients (HOG) [Dalal and Triggs 2005], [Lai 2011]
48
![Page 49: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/49.jpg)
Sparse Edge Points
49
Local information: gradient orientation and color
![Page 50: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/50.jpg)
Sparse Edge Points Matched Not matched
50
![Page 51: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/51.jpg)
Sparse Edge Points Matched Not matched
51
![Page 52: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/52.jpg)
Sparse Edge Points
Edge connectivity is lost
Matched Not matched
52
![Page 53: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/53.jpg)
Lines & Contour Fragments
53
![Page 54: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/54.jpg)
Lines & Contour Fragments
Line fitting is brittle
Difficult to parameterize
Dependent on edge extraction
Splines sensitive to occlusions
54
![Page 55: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/55.jpg)
Lines & Contour Fragments
Line fitting is brittle
Difficult to parameterize
Dependent on edge extraction
Splines sensitive to occlusions
55
![Page 56: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/56.jpg)
Histogram of Oriented Gradients
56
Coarse statistics of gradient orientation and magnitude
![Page 57: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/57.jpg)
Histogram of Oriented Gradients
Corrupted by background clutter Ambiguous shape
patch HOG HOG patch
57
![Page 58: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/58.jpg)
Histogram of Oriented Gradients
Corrupted by background clutter Ambiguous shape
patch HOG HOG patch
58
![Page 59: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/59.jpg)
Gradient Networks Our Approach
1. Match shape explicitly 2. Enforce connectivity without extracting edges
59
![Page 60: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/60.jpg)
Gradient Networks Overview
Shape template Input window
60
![Page 61: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/61.jpg)
Gradient Networks Overview
Shape template Input window
61
![Page 62: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/62.jpg)
Gradient Networks Local Shape Potential
How well does each pixel match locally? 62
![Page 63: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/63.jpg)
Gradient Networks Predicted Shape Match
Find long connected components which follow shape 63
![Page 64: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/64.jpg)
Local Shape Potential
Distance to template Local orientation
Color Edge potential 64
![Page 65: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/65.jpg)
Local Shape Potential
Distance to template Local orientation
Color Edge potential 65
![Page 66: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/66.jpg)
Local Shape Potential
Distance to template Local orientation
Color Edge potential 66
![Page 67: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/67.jpg)
Local Orientation Potential
67
model test
local orientation potential
![Page 68: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/68.jpg)
Local Shape Potential
Distance to template Local orientation
Color Edge potential 68
![Page 69: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/69.jpg)
Local Shape Potential
Distance to template Local orientation
Color Edge potential 69
![Page 70: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/70.jpg)
Local Shape Potential
70
![Page 71: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/71.jpg)
Gradient Networks
𝑝
𝑝
Each pixel is a node in the network 71
![Page 72: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/72.jpg)
𝑝
Gradient Networks
pQ0
pQ1
q
𝑝
Connect each node to neighbors in tangent direction 72
![Page 73: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/73.jpg)
Gradient Networks
𝑝
𝑝
Find paths in the network that match the shape well 73
![Page 74: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/74.jpg)
𝑝
Message Passing Local shape potential
shape similarity
local shape potential
message from left
message from right
[Bhat et al. 2010]
74
![Page 75: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/75.jpg)
𝑝
Message Passing Local shape potential
Initially, it is just the local shape potential 75
![Page 76: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/76.jpg)
Message Passing
𝑝
Local shape potential
76
![Page 77: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/77.jpg)
Message Passing
𝑝
Local shape potential
77
![Page 78: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/78.jpg)
Message Passing
𝑝
Local shape potential
78
![Page 79: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/79.jpg)
Predicted Shape Match
Local shape potential Predicted match
Message passing
79
![Page 80: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/80.jpg)
CMU Kitchen Occlusion Dataset
• 1600 images of 8 feature-poor objects • Single and multiple viewpoints • Cluttered scenes and occlusions
80
Objects Example images
![Page 81: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/81.jpg)
Shape Matching Results
Template Input window Local shape potential
Predicted match 81
![Page 82: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/82.jpg)
Shape Matching Results
Template Input window Local shape potential
Predicted match 82
![Page 83: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/83.jpg)
Shape Matching Results
Template Input window Local shape potential
Predicted match 83
![Page 84: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/84.jpg)
Object Detection Sliding Window
84
![Page 85: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/85.jpg)
Object Detection Sliding Window
85
![Page 86: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/86.jpg)
Detection Performance
86
better
![Page 87: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/87.jpg)
False positives with shape only
Object False positive window
GN point-wise confidences
88
![Page 88: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/88.jpg)
Interior Appearance
Object False positive window
GN point-wise confidences
89
![Page 89: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/89.jpg)
BaRT Boundary and Region Templates
90
![Page 90: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/90.jpg)
BaRT Boundary and Region Templates
91
![Page 91: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/91.jpg)
Boundary
Explicit shape: rLINE2D and GN 92
![Page 92: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/92.jpg)
BaRT Boundary and Region Templates
93
![Page 93: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/93.jpg)
Region
Consider appearance within the object interior HOG and color
94
![Page 94: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/94.jpg)
BaRT Boundary and Region Templates
95
![Page 95: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/95.jpg)
BaRT
Combines explicit boundary and region information
96
![Page 96: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/96.jpg)
HOG Uniform Regions
Uniform regions not represented well
97
![Page 97: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/97.jpg)
HOG Normalization
98
Each cell normalized with respect to magnitude of neighbors
![Page 98: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/98.jpg)
HOG Normalization
99
Amplifies noise if magnitude close to 0
![Page 99: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/99.jpg)
Uniform Regions
100
![Page 100: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/100.jpg)
Learning?
…
HOG + SVM Multiple images
weight = 0
HOG + exemplar SVM Single image
weight = random
101
![Page 101: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/101.jpg)
Learning?
…
HOG + SVM Multiple images
weight = 0
HOG + exemplar SVM Single image
weight = random
102
![Page 102: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/102.jpg)
Learning?
…
HOG + SVM Multiple images
weight = 0
HOG + exemplar SVM Single image
weight = random
103
![Page 103: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/103.jpg)
Modify HOG Normalization
Modified HOG HOG
Set cell to zero if normalization below threshold
104
![Page 104: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/104.jpg)
Matching Uniform Regions
Ours HOG
Test image:
105
HOG Ours
![Page 105: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/105.jpg)
Matching Uniform Regions
Ours HOG
Test image:
HOG Ours
106
![Page 106: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/106.jpg)
Matching Uniform Regions
Ours HOG
More accurate confidences in uniform regions
Test image:
HOG Ours
107
![Page 107: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/107.jpg)
Example Detections
detection zoomed in boundary (GN)
region (HOG+color) 108
![Page 108: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/108.jpg)
Example Detections
detection zoomed in boundary (GN)
region (HOG+color) 109
![Page 109: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/109.jpg)
Example Detections
detection zoomed in boundary (GN)
region (HOG+color) 110
![Page 110: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/110.jpg)
Detection Performance
112
![Page 111: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/111.jpg)
Detection Performance
113
![Page 112: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/112.jpg)
Detection Performance Under Different Occlusion Levels
114
![Page 113: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/113.jpg)
Detection Performance Under Different Occlusion Levels
115
![Page 114: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/114.jpg)
Overview Lack of Discriminative Features
Ambiguous Keypoint Features
Feature-poor objects
Occlusions
116
![Page 115: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/115.jpg)
Occlusions
117
![Page 116: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/116.jpg)
Occlusions
118
![Page 117: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/117.jpg)
Occlusions happen in 3D
119
![Page 118: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/118.jpg)
Occlusions happen in 3D
120
![Page 119: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/119.jpg)
Occlusions happen in 3D
121
![Page 120: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/120.jpg)
Occlusions happen in 3D
122
![Page 121: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/121.jpg)
Occlusion Reasoning
Matched Not matched
Which of these hypotheses is most likely? 123
![Page 122: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/122.jpg)
Occlusion Reasoning
Matched Not matched
Which of these hypotheses is most likely? 124
![Page 123: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/123.jpg)
Occlusion Reasoning
Matched Not matched
Which of these hypotheses is most likely? 125
![Page 124: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/124.jpg)
Occlusion Reasoning
Matched Not matched
Which of these hypotheses is most likely? 126
![Page 125: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/125.jpg)
Occlusion Reasoning
Local Coherency Fransens ‘06, Wang ‘09
Learn Occlusion Structure Gao ’11, Kwak ‘11
Object Detection Depth Ordering Wu ‘05, Wang ‘11
127
![Page 126: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/126.jpg)
Structure of Occlusions
Binary variable that equals 1 if is visible
Probability a point is visible given the visibility labeling of all other points
Occlusion Conditional Likelihood
Occlusion under a given camera view point c
128
Matched Not matched
![Page 127: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/127.jpg)
Occlusion Reasoning Per Environment
objH
objWobjL
Estimate of object dimensions Distribution of object dimensions for a given environment
129
![Page 128: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/128.jpg)
Occlusion Model
130
![Page 129: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/129.jpg)
Occlusion Model
Occluder
Object
131
![Page 130: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/130.jpg)
Occlusion Model objW
objH
h
w
Occluder
Object
132
![Page 131: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/131.jpg)
Occlusion Model objW
objH
h
w
Occluder
Object
133
![Page 132: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/132.jpg)
Occlusion Conditional Likelihood
jX
iX
𝐴𝑉𝑖,𝑉𝑗,𝑂𝑐
jX
𝐴𝑉𝑗,𝑂𝑐
Integral Geometry 134
![Page 133: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/133.jpg)
Occlusion Conditional Likelihood
jX
iX
𝐴𝑉𝑖,𝑉𝑗,𝑂𝑐
jX
𝐴𝑉𝑗,𝑂𝑐
Area covering all positions where Xj is visible and object occluded 135
![Page 134: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/134.jpg)
Occlusion Conditional Likelihood
jX
iX
𝐴𝑉𝑖,𝑉𝑗,𝑂𝑐
jX
𝐴𝑉𝑗,𝑂𝑐
Area covering all positions where Xj is visible and object occluded 136
![Page 135: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/135.jpg)
Occlusion Conditional Likelihood
jX
iX
𝐴𝑉𝑖,𝑉𝑗,𝑂𝑐
jX
𝐴𝑉𝑗,𝑂𝑐
Area covering all positions where Xj is visible and object occluded 137
![Page 136: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/136.jpg)
Occlusion Conditional Likelihood
jX
iX
𝐴𝑉𝑖,𝑉𝑗,𝑂𝑐
jX
𝐴𝑉𝑗,𝑂𝑐
Area covering all positions where Xj and Xj are visible and object occluded 138
![Page 137: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/137.jpg)
Occlusion Conditional Likelihood
139
𝑿𝒋
![Page 138: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/138.jpg)
Occlusion Conditional Likelihood Under Different Viewpoints
140
![Page 139: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/139.jpg)
Occlusion Conditional Likelihood Under Different Viewpoints
141
![Page 140: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/140.jpg)
Occlusion Conditional Likelihood Penalty (OCLP)
iX
High penalty if unlikely to be occluded by a valid object on same support surface
Matched Not matched
:OCLPf
142
![Page 141: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/141.jpg)
Occlusion Conditional Likelihood Penalty (OCLP)
iX
Low penalty if likely to be occluded by a valid object on same support surface
Matched Not matched
:OCLPf
143
![Page 142: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/142.jpg)
Occlusion Conditional Likelihood Penalty (OCLP)
iXMatched Not matched
Low penalty if likely to be occluded by a valid object on same support surface :OCLPf
144
![Page 143: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/143.jpg)
Example Detections
145
![Page 144: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/144.jpg)
Detection Performance
146
![Page 145: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/145.jpg)
Detection Performance Under Different Occlusion Levels
147
![Page 146: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/146.jpg)
Limitation
Binary Matching Pattern Occlusion Conditional Likelihood
148
![Page 147: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/147.jpg)
Limitation
Misclassifications can have impact on distribution
Binary Matching Pattern Occlusion Conditional Likelihood
149
![Page 148: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/148.jpg)
Occlusion Efficient Subwindow Search (OESS)
Probabilistic Matching Pattern 150 Probabilistic Matching Pattern
![Page 149: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/149.jpg)
OESS for True Positive
Occlusion can be explained well 151
![Page 150: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/150.jpg)
OESS for True Positive
95% explained 152
![Page 151: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/151.jpg)
OESS for False Positive
153
![Page 152: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/152.jpg)
OESS for False Positive
Only 50% explained 154
![Page 153: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/153.jpg)
OESS Scoring Matching Pattern
-1
+1 +1
-1
𝑝 = 1
𝑝 = 0
score = (1) + (1) + (-1) + (-1) = 0 155
![Page 154: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/154.jpg)
+1
+1 +1
-1
Occluding block
𝑝 = 1
𝑝 = 0
OESS Scoring Matching Pattern
score = (1) + (1) + (1) + (-1) = 2 156
rewarded
![Page 155: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/155.jpg)
OESS Scoring Matching Pattern
+1
-1 +1
-1
Occluding block
𝑝 = 1
𝑝 = 0
penalized
score = (-1) + (1) + (1) + (-1) = 0 157
![Page 156: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/156.jpg)
OESS
Reformulate as Efficient Subwindow Search (ESS) 158
![Page 157: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/157.jpg)
OESS
Find best occluder object 159
![Page 158: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/158.jpg)
OESS
Remove all explained points 160
![Page 159: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/159.jpg)
OESS
Iterate 161
![Page 160: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/160.jpg)
OESS
Iterate 162
![Page 161: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/161.jpg)
OESS
Iterate 163
![Page 162: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/162.jpg)
OESS
Final prediction 164
![Page 163: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/163.jpg)
Results
groundtruth predicted oboxes boundary region window detection
165
![Page 164: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/164.jpg)
Results
groundtruth predicted oboxes boundary region window detection
166
![Page 165: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/165.jpg)
Results
groundtruth predicted oboxes boundary region window detection
167
![Page 166: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/166.jpg)
Results
groundtruth predicted oboxes boundary region window detection
168
![Page 167: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/167.jpg)
Occlusion Prediction Performance
vs. predicted groundtruth
169
Average Intersection over Union (IoU)
![Page 168: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/168.jpg)
Occlusion Prediction Performance
vs.
predicted groundtruth 170
![Page 169: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/169.jpg)
Detection Performance
171
![Page 170: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/170.jpg)
172
![Page 171: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/171.jpg)
Summary Lack of Discriminative Features
Gradient Networks
Boundary and Region Templates
Occlusion Conditional Likelihood
Occlusion Efficient Subwindow Search
Ambiguous Keypoint Features
Feature-poor objects
Occlusions 173
![Page 172: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/172.jpg)
Main Contributions Ambiguous Keypoint Features
Making specific features less discriminative 174
![Page 173: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/173.jpg)
Main Contributions Representing Feature-poor Objects
Gradient Networks Boundary and Region Templates Explicit shape matching without
extracting edges Capture explicit boundary
and region information
175
![Page 174: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/174.jpg)
Main Contributions Representing Feature-poor Objects
Gradient Networks Boundary and Region Templates Explicit shape matching without
extracting edges Capture explicit boundary
and region information
176
![Page 175: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/175.jpg)
Main Contributions Representing Feature-poor Objects
Gradient Networks Boundary and Region Templates Explicit shape matching without
extracting edges Capture explicit boundary
and region information
177
![Page 176: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/176.jpg)
Main Contributions Occlusion Reasoning
Occlusion Conditional Likelihood
Representing occlusion structure under arbitrary viewpoint
Occlusion Efficient Subwindow Search
Directly search for occluding blocks to explain matching pattern
178
![Page 177: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/177.jpg)
Main Contributions Occlusion Reasoning
Occlusion Conditional Likelihood
Representing occlusion structure under arbitrary viewpoint
Occlusion Efficient Subwindow Search
Directly search for occluding blocks to explain matching pattern
179
![Page 178: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/178.jpg)
Main Contributions Occlusion Reasoning
Occlusion Conditional Likelihood
Representing occlusion structure under arbitrary viewpoint
Occlusion Efficient Subwindow Search
Directly search for occluding blocks to explain matching pattern
180
![Page 179: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/179.jpg)
Acknowledgements
181
Martial Hebert Alexei Efros Takeo Kanade Andrew Zisserman
![Page 180: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/180.jpg)
182
![Page 181: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/181.jpg)
183
![Page 182: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/182.jpg)
184
Background
![Page 183: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/183.jpg)
Augmented Reality
185
3D model Target environment
![Page 184: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/184.jpg)
Augmented Reality
186
3D model Target environment
![Page 185: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/185.jpg)
Instance vs. Category Recognition
187
Instance Arbitrary viewpoint and lighting
Single image per view
Category Intra-class variations
Many images per view
![Page 186: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/186.jpg)
Ambiguous Viewpoint
188
![Page 187: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/187.jpg)
Failure of SIFT Matching
189
![Page 188: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/188.jpg)
Invariant Approaches
190
![Page 189: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/189.jpg)
Future Directions
Fine-grained verification
Scalability 3D
191
![Page 190: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/190.jpg)
Fine-grained Verification
192
![Page 191: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/191.jpg)
Scalability
193
![Page 192: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/192.jpg)
3D
194
![Page 193: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/193.jpg)
195
Datasets
![Page 194: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/194.jpg)
CMU Grocery Dataset
• 620 images of household objects – 10 objects
• 25 single instance, 25 double instance • 12 with ground truth pose
– Clutter, viewpoint, lighting, occlusion
![Page 195: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/195.jpg)
CMU Kitchen Occlusion Dataset
• 1600 images of 8 household objects • Single and multiple viewpoints • Cluttered scenes and occlusions
197 Hsiao and Hebert, CVPR 2012.
Objects Example images
![Page 196: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/196.jpg)
198
Gradient Networks
![Page 197: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/197.jpg)
Local Shape Potential
199
Region of influence Appearance Edge
![Page 198: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/198.jpg)
Local Appearance
200
Gradient Orientation Color
![Page 199: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/199.jpg)
Potentials
201
Pairwise
Unary
![Page 200: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/200.jpg)
Message Passing
202
Shape Similarity
![Page 201: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/201.jpg)
Probability Calibration
scores
Scheirer et al. CVPR 2012
NOT Object Object
Dens
ity o
f N
OT
Obj
ect
Probability of O
bject Weibull fit to
tail of negative distribution
…
CDF of NOT Object
203
![Page 202: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/202.jpg)
Soft Shape Model
204
![Page 203: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/203.jpg)
Additional Results
205
![Page 204: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/204.jpg)
Color Potential
206
![Page 205: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/205.jpg)
LINE2D Similarity
207
ipModel point
∑=
∆=N
iiDLINEscore
12 )cos( θ
LINE2D (Hinterstoisser et al., PAMI 2011)
00.1)0cos( =o
71.0)45cos( =o
iθ∆
Quantized gradient orientation of model point, pi
Quantized gradient orientation of the best matching image point in a local neighborhood
![Page 206: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/206.jpg)
∑=
=∆=N
iiDrLINEscore
12 )0( θδ
Robust LINE2D Similarity
208
iθ∆
ipModel point
rLINE2D (Hsiao and Hebert, CVPR 2012)
Quantized gradient orientation of model point, pi
Quantized gradient orientation of the best matching image point in a local neighborhood
![Page 207: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/207.jpg)
Message Passing Iterations
209
![Page 208: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/208.jpg)
Probability Calibration
210
![Page 209: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/209.jpg)
F-Measure of Shape Matching
211
![Page 210: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/210.jpg)
Single View
212
![Page 211: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/211.jpg)
Multiple View
213
![Page 212: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/212.jpg)
Detection Rate @ 1.0 FPPI
214
![Page 213: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/213.jpg)
Detection Rate @ 1.0 FPPI
215
![Page 214: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/214.jpg)
False Positives
216
![Page 215: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/215.jpg)
217
BaRT
![Page 216: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/216.jpg)
Grid Optimization
Un-optimized : 57 cells Optimized : 60 cells
218
![Page 217: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/217.jpg)
HOG Normalization
219
Amplifies noise in uniform region!
![Page 218: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/218.jpg)
HOG Normalization
220
Sensitive to shading effects!
![Page 219: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/219.jpg)
HOG Normalization Pedestrians
221
![Page 220: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/220.jpg)
Average Precision
222
![Page 221: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/221.jpg)
Single View
223
![Page 222: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/222.jpg)
Multiple View
224
![Page 223: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/223.jpg)
False positives
Match both boundary and region
225
![Page 224: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/224.jpg)
BaRT False Positives Insufficient edge evidence
Unlikely occlusion configuration
Region information is only informative after there is a plausible hypothesis based on the boundary
226
![Page 225: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/225.jpg)
227
Occlusion Reasoning
![Page 226: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/226.jpg)
Occlusion Model
228
![Page 227: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/227.jpg)
Occlusion Scoring
Sliding window
Object detector
Occlusion hypothesis (binary)
Score of window
Occlusion model
229
![Page 228: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/228.jpg)
Occlusion Conditional Likelihood
230
![Page 229: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/229.jpg)
Occlusion Conditional Likelihood
Approximation
Analytic Approximate
231
![Page 230: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/230.jpg)
Distribution of Physical Dimensions
Household Objects
232
![Page 231: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/231.jpg)
Occlusion Statistics
233
![Page 232: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/232.jpg)
Validity of Occlusion Model
234
![Page 233: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/233.jpg)
Occlusion Penalty
Occlusion Prior Penalty (OPP)
Occlusion Conditional Likelihood Penalty (OCLP)
235
![Page 234: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/234.jpg)
Average Precision
236
![Page 235: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/235.jpg)
Performance vs. Occlusion
237
![Page 236: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/236.jpg)
Learning from Data
238
![Page 237: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/237.jpg)
Parameter Sensitivity
239
![Page 238: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/238.jpg)
240
OESS
![Page 239: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/239.jpg)
Occlusion Upper Bound
241
![Page 240: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/240.jpg)
OESS Algorithm
242
![Page 241: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/241.jpg)
OESS vs. Brute Force
243
![Page 242: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/242.jpg)
Occlusion Prediction
244
![Page 243: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/243.jpg)
Object Detection Performance
245
![Page 244: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/244.jpg)
246
Ambiguous Features
![Page 245: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/245.jpg)
Problem
• Not enough correct matches
Result of our system Difficult to obtain matches
![Page 246: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/246.jpg)
Discriminative hierarchical matching (DHM)
Model features (Level 0)
Quantized features (Level 1)
Quantized features (Level 2)
Image features
discriminative match
discriminative match
discriminative match
Candidate correspondences
aggregate
![Page 247: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/247.jpg)
DHM example
All features
![Page 248: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/248.jpg)
DHM result
DHM – 11 correct matches (soymilk can)
Ratio test – 3 correct matches (soymilk can)
![Page 249: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/249.jpg)
Simulated Affine (SA)
Morel & Yu 2009
![Page 250: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/250.jpg)
Baseline systems • Gordon & Lowe
– SIFT + RANSAC – Levenberg-Marquardt non-linear optimization
• Enhanced PnP (EPnP)
– Gordon & Lowe – EPnP non-iterative pose estimation algorithm
• Collet et al.
– Gordon & Lowe – Mean-shift spatial clustering of image features
![Page 251: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/251.jpg)
Averaged precision-recall
![Page 252: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/252.jpg)
Average Precision
![Page 253: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/253.jpg)
Object detection results
![Page 254: Detecting Object Instances Without Discriminative Featuresehsiao/thesis/ehsiao_thesis_slides.pdf · Detecting Object Instances Without Discriminative Features Edward Hsiao . June](https://reader034.vdocuments.us/reader034/viewer/2022042710/5f6737a293f10f15167b7ad9/html5/thumbnails/254.jpg)
Failure cases
Pose ambiguity
Repeated patterns
Extreme lighting, occlusion, viewpoint…etc