rgb(d) scene labeling- features and algorithms
Post on 05-Dec-2014
431 Views
Preview:
DESCRIPTION
TRANSCRIPT
RGB-(D) Scene Labeling: Features and Algorithms
Ahmed TahaMay 2014
Supervised by Dr : Marwan A. Torki
Introduction Scene labeling challenges Pipeline
Feature Extraction Super-pixel formulation and classification Classifying segmentation tree paths Classifying super-pixels MRF
Datasets and results
Agenda
Scene Labeling Labeling of each pixel in an image to a certain
class Scene Labeling can be done
Indoors Label a Sofa in a Bedroom Label a door in a living room
Outdoors Label a car in street Label building in street
Scene Labeling
Scene Labeling
Scene Labeling
Indoor scene labeling challenges Large variations of scene types Lack of distinctive features Poor illumination
Scene Labeling
Benefits of using depth feature in scene labeling Increased accuracy and robustness Body pose estimation 3D mapping Object recognition 3D modeling and interaction
Scene Labeling
Pipeline
1. Extract features using Kernel descriptor (KDES).
2. Aggregate descriptors in dense region into super-pixels using Efficient match kernels (EMK)
3. Classify super-pixels using Linear support vector machine (SVM)
4. Label super-pixels by classifying paths of segmentation tree.
5. Label super-pixels using super-pixel MRF
Pipeline
Kernel Descriptors (KDES), a unified framework that uses different aspects of similarity (kernel) to derive patch descriptors. Image gradient Spin/normal Color Depth gradient
Features Extraction (Step 1)
Efficient match kernels (EMK) to transform and aggregate descriptors in a set S (grid locations in the interior of a superpixel ‘s’).
Super-pixels are not of the same size.
Super-pixel formation (Step 2)
Linear Support vector machine (SVM) Non-probabilistic binary linear classifier.
Classify superpixels (Step 3)
Classifying paths in segmentation tree
Contextual Models (Step 4)
Classifying paths in segmentation tree
Contextual Models (Step 4)
Classifying paths in segmentation tree
Contextual Models
Classifying paths in segmentation tree If we accumulate features over paths, the
accuracy continues to increase to the top level The initial part of the curves overlap,
suggesting there is little benefit going to superpixels at too fine scales
Contextual Models
Superpixel MRF with gPb
Contextual Models (Step 5)
Superpixel MRF with gPb standard MRF formulation. We use Graph Cut
to find the labeling that mini- mizes the energy of a pairwise MRF
Contextual Models (Step 5)
Pipeline
NYU-D dataset Improve accuracy from 50% to 76%
Stanford Background dataset Improve accuracy 79% to 82%
Datasets - Results
Datasets - Results
Rgb-(d) scene labeling: Features and algorithms X Ren, L Bo, D Fox - Computer Vision and
Pattern Recognition 2012 - ieeexplore.ieee.org Context by region ancestry
JJ Lim, P Arbeláez, C Gu, J Malik - Computer Vision, 2009 IEEE 2009 - ieeexplore.ieee.org
References
top related