3. Depth transforms with inv. scaling
Sufficient to train a classifier for a single dC
For other depths d :
4. Multiple semantic classes
1. Pixel-wise classifier
superpixels not necessarily planar
2. Translation invariant
Pulling Things out of Perspective
Single View Depth Estimation
Ľubor Ladický1, Jianbo Shi2, Marc Pollefeys1
1 ETH Zürich, Switzerland 2 University of Pennsylvania, Philadelphia, USA
Experiments Our classifier
Training of the classifier
Standard approaches
1. Model fitting [Barinova et al. ECCV08]
• Requires strong prior knowledge
• Ignores small objects
2. 3D-Detection based [Hoiem et al. CVPR06]
• Works only for foreground objects (things)
3. Depth from semantic labels [Liu et al, CVPR10]
• Requires strong priors about semantic classes
4. Data driven [Saxena et al, NIPS05]
• Requires lots of data
• A problem with balancing data
General problem
•No common structure of the scene
•Ground plane not always visible
•Large variation of viewpoints and of objects in the scene
•Both things and stuff in the scene
•Impossible ?
Classifier response for x and at a depth d
window wh around the point xI
semantic label
1. Image pyramid is built
2. Training data randomly sampled
3. Samples of each class at dCused as positives
4. Samples of other classes or at d ≠ dC used as negatives
5. Multi-class classifier trained
• Dense Features SIFT, LBP, Self Similarity, Texton
• Representation Soft BOW representations in the set of rectangles
• Classifier AdaBoost
Patch classification
KITTI dataset
• 30 training & 30 test images
• 12 semantic labels
• depth range 2-50m (except sky)
• neighbouring depths di+1 / di = 1.25
NYU2 dataset
• 725 training & 724 test images
• 40 semantic labels
• depth range 1-10 m
• neighbouring depths di+1 / di = 1.25
KITTI dataset
The ratio of pixels below the relative error
NYU2 dataset
Semantic segmentation results
