TEMPLATE DESIGN © 2008

www.PosterPresentations.com

Context by Region Ancestry

Joseph J. Lim, Pablo Arbeláez, Chunhui Gu, Jitendra Malik

University of California at Berkeley{lim, arbelaez, chunhui, malik}@eecs.berkeley.edu

Introduction Comparing Leaves by Ancestry

Learning the Importance of Ancestors

Leaf Classification

Results on MSRC

Conclusion

• Our goal is to propose a new approach for modeling visual context.

• Main ideas:• Leverage on grouping produced by

generic segmentation• Each leaf (superpixel) is represented by

its ancestors and context comes from comparing them.

Leaf Only Ancestral Set Improvement

LR 47% 67% +20%

SVM 48% 62% +14%

RL 45% 57% +12%

Region Tree and Region Features

• We start with a new high-performance region detector [1] that produces a robust set of regions for each input image.

[1] P. Arbeláez, M. Maire, C. Fowlkes, J. Malik. CVPR 2009

Input image Selected region gPb signalContour

descriptor

Region representation – Contour shape descriptor

• Each leaf is represented by its ancestral set (AS), and each ancestor is described by appearance features (color, texture, shape).

• Comparison between two leafs is done by computing dissimilarity between their ancestral sets, based on feature-to-feature distances and learned weights.

• Each region in the ancestral set has different importance/role.e.g. Shape is predominant cue for airplane body, whereas sky is best described by color.

• We learn feature weights for each ancestor with three different methods:• Logistic Regression (LR)• SVM• Rank Learning (RL) [2]

Results on MIT Scene Dataset

• For each test leaf, we compute its dissimilarity to exemplar leaves, for which we learned weights.

• Weights of different ancestral sets may not be directly comparable. We calibrate the dissimilarities with logistic regression, obtaining thus probabilistic outputs.

[2] A. Frome, Y. Singer, J. Malik. NIPS 2006

• Per category score is an average of probabilities computed on exemplar leaves of that category.

• Each leaf is assigned the category with highest score.

Leaf Classification (cont.)

• Our model of context provides a significant improvement over the baseline (leaf only).

• We obtain an average classification performance of 67% on MSRC, which is competitive with other recent methods.

Example Results on MSRC

• Our approach is simple yet powerful for modeling contextual information by considering the ancestral sets of leaves in a segmentation tree.

• We obtain competitive results in both multi-class segmentation and scene classification tasks.

grass tree sky airplane

Original Image Initial Partition GT Our Result

Ours

(LO)89 27 83 44 80 67 48 96 67

Ours

(AS)93 81 88 64 77 79 80 96 82

Improve

ment4 54 5 20 -3 12 32 0 15

Method

Bu

ildin

g

City

Str

ee

t

Hig

hw

ay

Coa

st

Cou

ntr

y

Mo

unta

in

Fo

rest

Avera

ge

[3] 82 90 89 87 79 71 81 91 84

• We obtain state-of-the-art performance in 10 out of 21 categories. All these 10 categories are objects, for which context is relatively important.

[3] A. Oliva and A. Torralba. IJCV 2001.