learning to detect natural image boundaries using local brightness, color and texture cues by david...
Post on 22-Dec-2015
217 views
TRANSCRIPT
Learning to Detect Natural Image Boundaries Using Local
Brightness, Color and Texture Cues
by David R. Martin, Charless C. Fowlkes, Jitendra Malik
Heather Dunlop16-721: Advanced Perception
January 25, 2006
What is a Boundary?
CannyMartin,
2002
Human
Dataset“You will be presented a photographic image. Divide the image into some number of segments, where the segments represent ‘things’ or ‘parts of things’ in the scene. The number of segments is up to you, as it depends on the image. Something between 2 and 30 is likely to be appropriate. It is important that all of the segments have approximately equal importance.”
DatasetDatabase of over 1000 images and 5-10 segmentations for each
Martin, 2002
Boundaries
Intensity
Texture
Brightness
Color
Non-boundaries Boundaries
Martin, 2002
Method
ImageOptimized Cues
Boundary Strength
Brightness
Color
Texture
Benchmark
Human Segmentations
Cue Combination
Model
Martin, 2002
Goal: learn the probability of a boundary, Pb(x,y,θ)
Image FeaturesCIE L*a*b* color space (luminance, red-green, yellow-blue)Oriented Energy:
fe: Gaussian second derivativefo: Its Hilbert transform
BrightnessL* distribution
Colora* and b* distributions (joint or marginal)
Texture
2,
2
,,oe fIfIOE
TextureConvolve with a filter bank:
Gaussian second derivativeIts Hilbert transformDifference of Gaussians
Filter responses give a measure of texture
Other Filter BanksLeung-Malik filter set: Schmid filter set:
Maximum Response 8 filter set:
TextonsConvolve image with filter bankCluster filter responses to form textons
Adapted from Martin, 2002 and Varma, Zisserman, 2005
Texton DistributionAssign each pixel to nearest textonForm distribution of textons
Adapted from Martin, 2002 and Varma, Zisserman, 2005
Gradient-based FeaturesBrightness (BG), color (CG), texture (TG) gradientsHalf-disc regions described by histogramsCompare distributions with χ2 statistic
r(x,y)
i ii
ii
hg
hghg
22 )(
2
1),(
Texture GradientTexton distribution in two half circles
Martin, 2002
LocalizationTightly localize boundariesReduce noiseCoalesce double detectionsImprove OE and TG features
OE
TG localized
OE localized
TG
Martin, Fowlkes, Malik, 2004
OptimizationTexture parameters:
type of filter bankscale of filtersnumber of textonsuniversal or image-specific textons
Other possible distance/histogram comparison metricsNumber of bins for histogramsScale parameter for all cues
Evaluation MethodologyPosterior probability of boundary: Pb(x,y,θ)
Evaluation measure: precision recall curveF-measure: Martin,
2002
5.0
)1(
PRPRF
Cue CombinationWhich cues should be used?
OE is redundant when other cues are presentBG+CG+TG produces best results
Martin, 2002
ClassifiersUntil now, only logistic regression was usedOther possible classifiers:
Density estimationClassification treesHierarchical mixtures of expertsSupport vector machines Martin,
2002
Result ComparisonAlternative methods:
Matlab’s Canny edge detector with and without hysteresisSpatially-averaged second moment matrix (2MM) Martin,
2002
ResultsCanny 2MM BG+CG+TG HumanImage
Martin, 2002
Results
Martin, 2002
Canny 2MM BG+CG+TG HumanImage
Results
Martin, 2002
Canny 2MM BG+CG+TG HumanImage
ConclusionsLarge data set used for testingTexture gradients are a powerful cueSimple linear model sufficient for cue combinationOutperforms existing methodsAn approach that is useful for higher-level algorithmsCode is available online:http://www.eecs.berkeley.edu/Research/Projects/CS/vision/grouping/segbench/