seeing 3d chairs - computer...
TRANSCRIPT
![Page 1: Seeing 3D chairs - Computer Graphicsgraphics.cs.cmu.edu/courses/16-824-S15/16824_2015/presentation... · Technical details • select 10 most discriminative patches per 3D view based](https://reader033.vdocuments.us/reader033/viewer/2022050515/5f9f4145aad9d276ad31b44d/html5/thumbnails/1.jpg)
Seeing 3D chairs: Exemplar part-based 2D-3D
alignment using a large dataset of CAD models
16-824 Learning Based Method in Vision
Namhoon Lee
The figures in the slides are from the original paper as well as the author’s presentation at CVPR14.http://www.di.ens.fr/willow/research/seeing3Dchairs/
![Page 2: Seeing 3D chairs - Computer Graphicsgraphics.cs.cmu.edu/courses/16-824-S15/16824_2015/presentation... · Technical details • select 10 most discriminative patches per 3D view based](https://reader033.vdocuments.us/reader033/viewer/2022050515/5f9f4145aad9d276ad31b44d/html5/thumbnails/2.jpg)
Motivation
![Page 3: Seeing 3D chairs - Computer Graphicsgraphics.cs.cmu.edu/courses/16-824-S15/16824_2015/presentation... · Technical details • select 10 most discriminative patches per 3D view based](https://reader033.vdocuments.us/reader033/viewer/2022050515/5f9f4145aad9d276ad31b44d/html5/thumbnails/3.jpg)
Is there a chair?
![Page 4: Seeing 3D chairs - Computer Graphicsgraphics.cs.cmu.edu/courses/16-824-S15/16824_2015/presentation... · Technical details • select 10 most discriminative patches per 3D view based](https://reader033.vdocuments.us/reader033/viewer/2022050515/5f9f4145aad9d276ad31b44d/html5/thumbnails/4.jpg)
There is a chair!
![Page 5: Seeing 3D chairs - Computer Graphicsgraphics.cs.cmu.edu/courses/16-824-S15/16824_2015/presentation... · Technical details • select 10 most discriminative patches per 3D view based](https://reader033.vdocuments.us/reader033/viewer/2022050515/5f9f4145aad9d276ad31b44d/html5/thumbnails/5.jpg)
Can we say more about it?
![Page 6: Seeing 3D chairs - Computer Graphicsgraphics.cs.cmu.edu/courses/16-824-S15/16824_2015/presentation... · Technical details • select 10 most discriminative patches per 3D view based](https://reader033.vdocuments.us/reader033/viewer/2022050515/5f9f4145aad9d276ad31b44d/html5/thumbnails/6.jpg)
Can we sit on it?
![Page 7: Seeing 3D chairs - Computer Graphicsgraphics.cs.cmu.edu/courses/16-824-S15/16824_2015/presentation... · Technical details • select 10 most discriminative patches per 3D view based](https://reader033.vdocuments.us/reader033/viewer/2022050515/5f9f4145aad9d276ad31b44d/html5/thumbnails/7.jpg)
Style and pose
![Page 8: Seeing 3D chairs - Computer Graphicsgraphics.cs.cmu.edu/courses/16-824-S15/16824_2015/presentation... · Technical details • select 10 most discriminative patches per 3D view based](https://reader033.vdocuments.us/reader033/viewer/2022050515/5f9f4145aad9d276ad31b44d/html5/thumbnails/8.jpg)
Goal
![Page 9: Seeing 3D chairs - Computer Graphicsgraphics.cs.cmu.edu/courses/16-824-S15/16824_2015/presentation... · Technical details • select 10 most discriminative patches per 3D view based](https://reader033.vdocuments.us/reader033/viewer/2022050515/5f9f4145aad9d276ad31b44d/html5/thumbnails/9.jpg)
Dataset
![Page 10: Seeing 3D chairs - Computer Graphicsgraphics.cs.cmu.edu/courses/16-824-S15/16824_2015/presentation... · Technical details • select 10 most discriminative patches per 3D view based](https://reader033.vdocuments.us/reader033/viewer/2022050515/5f9f4145aad9d276ad31b44d/html5/thumbnails/10.jpg)
![Page 11: Seeing 3D chairs - Computer Graphicsgraphics.cs.cmu.edu/courses/16-824-S15/16824_2015/presentation... · Technical details • select 10 most discriminative patches per 3D view based](https://reader033.vdocuments.us/reader033/viewer/2022050515/5f9f4145aad9d276ad31b44d/html5/thumbnails/11.jpg)
Large dataset of 3D Chairs
• 1,393 3D chair models from 3D Warehouse • 62 different viewpoints => 1,393*62 = 86,366 synthesized images • 10 discriminative patches for each view ~ 800K patches
![Page 12: Seeing 3D chairs - Computer Graphicsgraphics.cs.cmu.edu/courses/16-824-S15/16824_2015/presentation... · Technical details • select 10 most discriminative patches per 3D view based](https://reader033.vdocuments.us/reader033/viewer/2022050515/5f9f4145aad9d276ad31b44d/html5/thumbnails/12.jpg)
Approach: use 3D models
62 views
![Page 13: Seeing 3D chairs - Computer Graphicsgraphics.cs.cmu.edu/courses/16-824-S15/16824_2015/presentation... · Technical details • select 10 most discriminative patches per 3D view based](https://reader033.vdocuments.us/reader033/viewer/2022050515/5f9f4145aad9d276ad31b44d/html5/thumbnails/13.jpg)
Style
Viewpoint
![Page 14: Seeing 3D chairs - Computer Graphicsgraphics.cs.cmu.edu/courses/16-824-S15/16824_2015/presentation... · Technical details • select 10 most discriminative patches per 3D view based](https://reader033.vdocuments.us/reader033/viewer/2022050515/5f9f4145aad9d276ad31b44d/html5/thumbnails/14.jpg)
Approach
![Page 15: Seeing 3D chairs - Computer Graphicsgraphics.cs.cmu.edu/courses/16-824-S15/16824_2015/presentation... · Technical details • select 10 most discriminative patches per 3D view based](https://reader033.vdocuments.us/reader033/viewer/2022050515/5f9f4145aad9d276ad31b44d/html5/thumbnails/15.jpg)
How to match CG to photograph?
Implementation: exemplar-LDA
HOG Classifier
HOG mask
Masked classifier
See also: [Shrivastava et al. 2011]
![Page 16: Seeing 3D chairs - Computer Graphicsgraphics.cs.cmu.edu/courses/16-824-S15/16824_2015/presentation... · Technical details • select 10 most discriminative patches per 3D view based](https://reader033.vdocuments.us/reader033/viewer/2022050515/5f9f4145aad9d276ad31b44d/html5/thumbnails/16.jpg)
Approach: CG-to-photograph
Implementation: exemplar-LDA
![Page 17: Seeing 3D chairs - Computer Graphicsgraphics.cs.cmu.edu/courses/16-824-S15/16824_2015/presentation... · Technical details • select 10 most discriminative patches per 3D view based](https://reader033.vdocuments.us/reader033/viewer/2022050515/5f9f4145aad9d276ad31b44d/html5/thumbnails/17.jpg)
Discriminative visual element detectors using exemplar-LDA based training
: A square image patch (represented as a HOG descriptor)
: patch in the input image
similarity:
- exemplar-classifier: q VS all the negatives
- LDA
sq(x) = w
Tq x
q
x
wq = ⌃�1(q � µn) µn =1
N
NX
i=1
xi ⌃ =1
N
NX
i=1
(xi � µn)(xi � µn)T
Sq(x) = �(q)T�(x)� �(q)T�(0) �(x) = ⌃� 12 (x� µn)
![Page 18: Seeing 3D chairs - Computer Graphicsgraphics.cs.cmu.edu/courses/16-824-S15/16824_2015/presentation... · Technical details • select 10 most discriminative patches per 3D view based](https://reader033.vdocuments.us/reader033/viewer/2022050515/5f9f4145aad9d276ad31b44d/html5/thumbnails/18.jpg)
Technical details
• select 10 most discriminative patches per 3D view based on
• Non-max suppression intersection area to union ratio of 0.25
• filter out patches less than 100x100 pixels
• 10x10 HOG cells —> 900 dimensional feature
• set to 0 for the component in the weight, which has the sum of the absolute value across the HOG channels < 0.01
k�(q)k2
![Page 19: Seeing 3D chairs - Computer Graphicsgraphics.cs.cmu.edu/courses/16-824-S15/16824_2015/presentation... · Technical details • select 10 most discriminative patches per 3D view based](https://reader033.vdocuments.us/reader033/viewer/2022050515/5f9f4145aad9d276ad31b44d/html5/thumbnails/19.jpg)
Further details• Calibrating visual element detectors (800K) 1. run on a 200K negative patches 2. select (false positive rate: 0.01%) 3. : mean HOG feature 4. set
• Part-based detection 1. find a patch whose response > 0.2 —> root 2. local search around the expected location 3. final score:= sum of the visual element detection scores
xn
µn
S
0
q(xn) = 0 S0
q(µn) = �1
S
0
q(x) = aqSq(x) + bq
![Page 20: Seeing 3D chairs - Computer Graphicsgraphics.cs.cmu.edu/courses/16-824-S15/16824_2015/presentation... · Technical details • select 10 most discriminative patches per 3D view based](https://reader033.vdocuments.us/reader033/viewer/2022050515/5f9f4145aad9d276ad31b44d/html5/thumbnails/20.jpg)
Results
![Page 21: Seeing 3D chairs - Computer Graphicsgraphics.cs.cmu.edu/courses/16-824-S15/16824_2015/presentation... · Technical details • select 10 most discriminative patches per 3D view based](https://reader033.vdocuments.us/reader033/viewer/2022050515/5f9f4145aad9d276ad31b44d/html5/thumbnails/21.jpg)
Results
![Page 22: Seeing 3D chairs - Computer Graphicsgraphics.cs.cmu.edu/courses/16-824-S15/16824_2015/presentation... · Technical details • select 10 most discriminative patches per 3D view based](https://reader033.vdocuments.us/reader033/viewer/2022050515/5f9f4145aad9d276ad31b44d/html5/thumbnails/22.jpg)
Results
![Page 23: Seeing 3D chairs - Computer Graphicsgraphics.cs.cmu.edu/courses/16-824-S15/16824_2015/presentation... · Technical details • select 10 most discriminative patches per 3D view based](https://reader033.vdocuments.us/reader033/viewer/2022050515/5f9f4145aad9d276ad31b44d/html5/thumbnails/23.jpg)
Results
![Page 24: Seeing 3D chairs - Computer Graphicsgraphics.cs.cmu.edu/courses/16-824-S15/16824_2015/presentation... · Technical details • select 10 most discriminative patches per 3D view based](https://reader033.vdocuments.us/reader033/viewer/2022050515/5f9f4145aad9d276ad31b44d/html5/thumbnails/24.jpg)
Results
![Page 25: Seeing 3D chairs - Computer Graphicsgraphics.cs.cmu.edu/courses/16-824-S15/16824_2015/presentation... · Technical details • select 10 most discriminative patches per 3D view based](https://reader033.vdocuments.us/reader033/viewer/2022050515/5f9f4145aad9d276ad31b44d/html5/thumbnails/25.jpg)
Results
![Page 26: Seeing 3D chairs - Computer Graphicsgraphics.cs.cmu.edu/courses/16-824-S15/16824_2015/presentation... · Technical details • select 10 most discriminative patches per 3D view based](https://reader033.vdocuments.us/reader033/viewer/2022050515/5f9f4145aad9d276ad31b44d/html5/thumbnails/26.jpg)
Thank you