a statistically selected part-based probabilistic model for object recognition zhipeng zhao, ahmed...
TRANSCRIPT
![Page 1: A Statistically Selected Part-Based Probabilistic Model for Object Recognition Zhipeng Zhao, Ahmed Elgammal Department of Computer Science, Rutgers, The](https://reader033.vdocuments.us/reader033/viewer/2022051516/56649ebd5503460f94bc6ab0/html5/thumbnails/1.jpg)
A Statistically Selected Part-Based Probabilistic Model for Object Recognition
Zhipeng Zhao, Ahmed Elgammal
Department of Computer Science, Rutgers, The State University of New Jersey,
Piscataway, NJ, USA
![Page 2: A Statistically Selected Part-Based Probabilistic Model for Object Recognition Zhipeng Zhao, Ahmed Elgammal Department of Computer Science, Rutgers, The](https://reader033.vdocuments.us/reader033/viewer/2022051516/56649ebd5503460f94bc6ab0/html5/thumbnails/2.jpg)
Outline
Background and motivations Statistical selection of image parts Parts based probabilistic model Experiments Conclusion
![Page 3: A Statistically Selected Part-Based Probabilistic Model for Object Recognition Zhipeng Zhao, Ahmed Elgammal Department of Computer Science, Rutgers, The](https://reader033.vdocuments.us/reader033/viewer/2022051516/56649ebd5503460f94bc6ab0/html5/thumbnails/3.jpg)
Generic Object Recognition
• Variations in scale, orientation and visibility• Variability within Specificity• Object of interest has to be recognized in context of
multiple other objects and cluttered background
![Page 4: A Statistically Selected Part-Based Probabilistic Model for Object Recognition Zhipeng Zhao, Ahmed Elgammal Department of Computer Science, Rutgers, The](https://reader033.vdocuments.us/reader033/viewer/2022051516/56649ebd5503460f94bc6ab0/html5/thumbnails/4.jpg)
Recognition of Object Categories has recently gained a lot of attention in computer vision.
3D model based method traditionally handle specific object instance, e.g. with the goal of recovering object pose.
Appearance template search-based methods, e.g., Schneiderman et al 00,Viola et al 02
![Page 5: A Statistically Selected Part-Based Probabilistic Model for Object Recognition Zhipeng Zhao, Ahmed Elgammal Department of Computer Science, Rutgers, The](https://reader033.vdocuments.us/reader033/viewer/2022051516/56649ebd5503460f94bc6ab0/html5/thumbnails/5.jpg)
Part-based object recognition
Part-based methods [Fischler73, Lowe99, Fergus03, Fergus05]:
Objects are modeled as constellation, a collection of parts or local features with distinctive appearance and spatial position.
the recognition is based on inferring object class based on similarity in parts’ appearance and their spatial arrangement.
![Page 6: A Statistically Selected Part-Based Probabilistic Model for Object Recognition Zhipeng Zhao, Ahmed Elgammal Department of Computer Science, Rutgers, The](https://reader033.vdocuments.us/reader033/viewer/2022051516/56649ebd5503460f94bc6ab0/html5/thumbnails/6.jpg)
Bag-of-features: Zhang et al (1995), Mohr et al (1997), Lowe
(1999), Matas et al (2002), Mikolajczyk et al(2003), Bray et al (2004).
Ignore spatial arrangement of parts Very successful result But..
None of the features are completely affine invariant
Correspondences based on very closely related features
Not conducive for recognition of object categories
![Page 7: A Statistically Selected Part-Based Probabilistic Model for Object Recognition Zhipeng Zhao, Ahmed Elgammal Department of Computer Science, Rutgers, The](https://reader033.vdocuments.us/reader033/viewer/2022051516/56649ebd5503460f94bc6ab0/html5/thumbnails/7.jpg)
Part Structure Vidal-Naquet (2001), Zisserman et al (2003),
Perona et al ( 2003), Schiele et al ( 2004), Triggs et al (2005), Serre et al (2005)
Limitations: Combinatorial combination of parts and
locations Number of parts considered are limited Huge chunk of image information is discarded
![Page 8: A Statistically Selected Part-Based Probabilistic Model for Object Recognition Zhipeng Zhao, Ahmed Elgammal Department of Computer Science, Rutgers, The](https://reader033.vdocuments.us/reader033/viewer/2022051516/56649ebd5503460f94bc6ab0/html5/thumbnails/8.jpg)
Part Based approaches for object recognition
Phases in part based recognition: Part extraction [kadir01]: extract salient
features from the image. Part selection [Dorko04]: select parts that best
characterize the target object. Select parts by classification likelihood. Select parts by mutual information.
Object Model [Fergus03, Fergus05]: construct a model of the target object using the selected parts.
Object recognition [Fergus03, Fergus05]: having selected parts from a test image, perform its recognition using object model.
![Page 9: A Statistically Selected Part-Based Probabilistic Model for Object Recognition Zhipeng Zhao, Ahmed Elgammal Department of Computer Science, Rutgers, The](https://reader033.vdocuments.us/reader033/viewer/2022051516/56649ebd5503460f94bc6ab0/html5/thumbnails/9.jpg)
Discriminative Part Selection - Problem Description
Given : A collection of parts from positive and negative images
Find : Parts that consistently appear in the positive images but rarely in the negative images
Requirements : Focus on part similarities across positive images Ignore similarity between parts within an image Avoid ubiquitous parts - those appear in all classes
A collection of parts from the negative classes
Parts from the positive class images
![Page 10: A Statistically Selected Part-Based Probabilistic Model for Object Recognition Zhipeng Zhao, Ahmed Elgammal Department of Computer Science, Rutgers, The](https://reader033.vdocuments.us/reader033/viewer/2022051516/56649ebd5503460f94bc6ab0/html5/thumbnails/10.jpg)
Our contributions
Part selection: A statistical approaches for unsupervised selection of discriminative parts:
Finds the parts that best discriminate between the positive and negative classes.
Object model and recognition: A probabilistic Bayesian approach for recognition where object model does not need a reference part.
We investigated PCA and 2D PCA for image part representation in our experiment and did a comparison.
![Page 11: A Statistically Selected Part-Based Probabilistic Model for Object Recognition Zhipeng Zhao, Ahmed Elgammal Department of Computer Science, Rutgers, The](https://reader033.vdocuments.us/reader033/viewer/2022051516/56649ebd5503460f94bc6ab0/html5/thumbnails/11.jpg)
Outline
Background and motivations Statistical selection of image parts Parts based probabilistic model Experiments Conclusion
![Page 12: A Statistically Selected Part-Based Probabilistic Model for Object Recognition Zhipeng Zhao, Ahmed Elgammal Department of Computer Science, Rutgers, The](https://reader033.vdocuments.us/reader033/viewer/2022051516/56649ebd5503460f94bc6ab0/html5/thumbnails/12.jpg)
Statistical selection of image parts
Given: image parts from positive training images. Find: parts that constantly appear in multiple instances of
the positive evaluation images but only rarely appear in the negative evaluation images.
We use each image part from training images to build classifier for positive and negative evaluation images.
This classifier is built based on whether the image part appears in the evaluation image or not.
A collection of parts from the negative classes
Parts from the positive class images
![Page 13: A Statistically Selected Part-Based Probabilistic Model for Object Recognition Zhipeng Zhao, Ahmed Elgammal Department of Computer Science, Rutgers, The](https://reader033.vdocuments.us/reader033/viewer/2022051516/56649ebd5503460f94bc6ab0/html5/thumbnails/13.jpg)
Statistical selection of image parts
This classifier is built based on whether the image part appears in the evaluation image or not.
Similar to boosting in building week classifier, but we filter out uncharacteristic image parts and only keep the image parts on which better classifiers are built.
These selected parts should be informative of the object and the model built from these parts is likely to achieve better recognition performance.
![Page 14: A Statistically Selected Part-Based Probabilistic Model for Object Recognition Zhipeng Zhao, Ahmed Elgammal Department of Computer Science, Rutgers, The](https://reader033.vdocuments.us/reader033/viewer/2022051516/56649ebd5503460f94bc6ab0/html5/thumbnails/14.jpg)
Statistical selection of image parts
Build classifier: The appearance of image part v in evaluation image V can
be implied by the Euclidean distance between v and the closest image part in V:
We can use D(V,v) and a threshold t to classify evaluation data:
Image V is positive if the distance between image part v and the image V is less than a threshold t, which implies v appears in V:
Image V is negative if otherwise.
![Page 15: A Statistically Selected Part-Based Probabilistic Model for Object Recognition Zhipeng Zhao, Ahmed Elgammal Department of Computer Science, Rutgers, The](https://reader033.vdocuments.us/reader033/viewer/2022051516/56649ebd5503460f94bc6ab0/html5/thumbnails/15.jpg)
Statistical selection of image parts
Measure the performance of the classifier: Performance is measure by the misclassified
evaluation image:
Select image parts: We select the image parts from which we can
build a classifier with a classification error rate less than a threshold θ.
![Page 16: A Statistically Selected Part-Based Probabilistic Model for Object Recognition Zhipeng Zhao, Ahmed Elgammal Department of Computer Science, Rutgers, The](https://reader033.vdocuments.us/reader033/viewer/2022051516/56649ebd5503460f94bc6ab0/html5/thumbnails/16.jpg)
Experiments
Image parts selection from the statistical methods:
![Page 17: A Statistically Selected Part-Based Probabilistic Model for Object Recognition Zhipeng Zhao, Ahmed Elgammal Department of Computer Science, Rutgers, The](https://reader033.vdocuments.us/reader033/viewer/2022051516/56649ebd5503460f94bc6ab0/html5/thumbnails/17.jpg)
Outline
Background and motivations Statistical selection of image parts Parts based probabilistic model Experiments Conclusion
![Page 18: A Statistically Selected Part-Based Probabilistic Model for Object Recognition Zhipeng Zhao, Ahmed Elgammal Department of Computer Science, Rutgers, The](https://reader033.vdocuments.us/reader033/viewer/2022051516/56649ebd5503460f94bc6ab0/html5/thumbnails/18.jpg)
Parts based probabilistic model We model the probability of an object with its
centroid, given the observed image parts from that image. If the probability is larger than a threshold, it indicates the presence of the object class in the image.
Assuming independence between image parts vk and using Bayes’ rule, the probability model of object class O with its centroid C, given the m observed image parts (k=1,…,m), can be formulated as:
![Page 19: A Statistically Selected Part-Based Probabilistic Model for Object Recognition Zhipeng Zhao, Ahmed Elgammal Department of Computer Science, Rutgers, The](https://reader033.vdocuments.us/reader033/viewer/2022051516/56649ebd5503460f94bc6ab0/html5/thumbnails/19.jpg)
Parts based probabilistic model
P(vk|O,C) can be modeled as mixture-Gaussians learnt from the selected image parts from the training data, which is clustered into n clusters Ai , (i =1..n) according to their appearance and 2D offset of the centroid to them:
![Page 20: A Statistically Selected Part-Based Probabilistic Model for Object Recognition Zhipeng Zhao, Ahmed Elgammal Department of Computer Science, Rutgers, The](https://reader033.vdocuments.us/reader033/viewer/2022051516/56649ebd5503460f94bc6ab0/html5/thumbnails/20.jpg)
Parts based probabilistic model
With further assumption of the independence between P(O) and P(C) and both part vk and the centroid C from one cluster following normal distribution, we can estimate the terms for P(vk|O,C) :
Here and denote the sample mean for vk and C respectively. and denote the sample covariance for vk and C respectively.
Other terms can be approximated using the statistics from each of the cluster Ai.
( , | ) ( | ) ( | )
( | ) ( | , )
( | ) ( | , )
i i i
vvk i k i i
CCi i i
P O C A P O A P C A
P v A N v
P C A N C
vi C
iv
i C
i
1( ) /
( | ) /
n
i i jj
j i ij i
P A n n
P O A n n
![Page 21: A Statistically Selected Part-Based Probabilistic Model for Object Recognition Zhipeng Zhao, Ahmed Elgammal Department of Computer Science, Rutgers, The](https://reader033.vdocuments.us/reader033/viewer/2022051516/56649ebd5503460f94bc6ab0/html5/thumbnails/21.jpg)
Parts based probabilistic model
Recognition: While performing recognition, the P(vk) can be
ignored so we have
The recognition can be viewed as: for each of the image part vk, it will vote for the possible object centroid according the clusters it is close to:
vk
clusteri
clusterj
clusterkWeighted vote for centroid
Test Image
![Page 22: A Statistically Selected Part-Based Probabilistic Model for Object Recognition Zhipeng Zhao, Ahmed Elgammal Department of Computer Science, Rutgers, The](https://reader033.vdocuments.us/reader033/viewer/2022051516/56649ebd5503460f94bc6ab0/html5/thumbnails/22.jpg)
Outline
Background and motivations Statistical selection of image part Parts based probabilistic model Experiments Conclusion
![Page 23: A Statistically Selected Part-Based Probabilistic Model for Object Recognition Zhipeng Zhao, Ahmed Elgammal Department of Computer Science, Rutgers, The](https://reader033.vdocuments.us/reader033/viewer/2022051516/56649ebd5503460f94bc6ab0/html5/thumbnails/23.jpg)
Experiments
Dataset: Dataset from Caltech database with four class
objects: motorbikes, airplane, face, car rear end against background.
train evaluate test
positive positive negative positive negative
Each class 175 50 50 225 225
![Page 24: A Statistically Selected Part-Based Probabilistic Model for Object Recognition Zhipeng Zhao, Ahmed Elgammal Department of Computer Science, Rutgers, The](https://reader033.vdocuments.us/reader033/viewer/2022051516/56649ebd5503460f94bc6ab0/html5/thumbnails/24.jpg)
Experiments
Image part detection and representation: Use Kadir and Brady’s region based feature
detector [Kadir01] for detecting informative image partes.
Normalize and rescale image parts to 11×11 pixels and represent them as 121 dimension intensity vector.
Both PCA and 2D PCA were applied on vectors to get a more compact 18 dimension intensity representation.
![Page 25: A Statistically Selected Part-Based Probabilistic Model for Object Recognition Zhipeng Zhao, Ahmed Elgammal Department of Computer Science, Rutgers, The](https://reader033.vdocuments.us/reader033/viewer/2022051516/56649ebd5503460f94bc6ab0/html5/thumbnails/25.jpg)
Experiments
Experiment setting: Extract 100 image parts from each of the training image
and evaluation image. Applied the statistical methods for removing the image
parts from the background. Statistical method: build a simple classifier from each
image part in training images and select the ones which led to a classifier with classification error rate less than 24%.
Compute the probability of the centroid of a possible object in the image as the indicator of its presence.
Use 2D offset between the image part and the centroid of the image as spatial information for the image part.
Use k-means algorithm to cluster selected parts into 70 clusters and calculate the statistics for them.
![Page 26: A Statistically Selected Part-Based Probabilistic Model for Object Recognition Zhipeng Zhao, Ahmed Elgammal Department of Computer Science, Rutgers, The](https://reader033.vdocuments.us/reader033/viewer/2022051516/56649ebd5503460f94bc6ab0/html5/thumbnails/26.jpg)
Experiments
Image parts selection from the statistical methods:
![Page 27: A Statistically Selected Part-Based Probabilistic Model for Object Recognition Zhipeng Zhao, Ahmed Elgammal Department of Computer Science, Rutgers, The](https://reader033.vdocuments.us/reader033/viewer/2022051516/56649ebd5503460f94bc6ab0/html5/thumbnails/27.jpg)
Experiments
Objects detected in the image:
![Page 28: A Statistically Selected Part-Based Probabilistic Model for Object Recognition Zhipeng Zhao, Ahmed Elgammal Department of Computer Science, Rutgers, The](https://reader033.vdocuments.us/reader033/viewer/2022051516/56649ebd5503460f94bc6ab0/html5/thumbnails/28.jpg)
Experiments
Equal ROC performance using different methods:
dataset No selection
Statistical Method with
2D PCA
Statistical Method
with PCA
[Fergus03] [Opelt04]
Airplane 54.2% 95.8% 94.4% 90.2% 88.9%
Motorbike 67.8% 93.7% 94.9% 92.5% 92.2%
Face 62.7% 97.3% 98.4% 96.4% 93.5%
Car(rear) 65.6% 98.0% 96.7% 90.3% n/a
![Page 29: A Statistically Selected Part-Based Probabilistic Model for Object Recognition Zhipeng Zhao, Ahmed Elgammal Department of Computer Science, Rutgers, The](https://reader033.vdocuments.us/reader033/viewer/2022051516/56649ebd5503460f94bc6ab0/html5/thumbnails/29.jpg)
Conclusion:
We have presented a statistical method for selecting informative image parts for part-based object detection and class recognition. This method yields competitive recognition rates
and surpasses the performance of many existing methods.
It is a general method suitable for selecting a set of features in other application.
Future work: Integrate information regarding the spatial
arrangement between image parts; Develop the framework as a general method with
various hyper-parameters automatically determined.
![Page 30: A Statistically Selected Part-Based Probabilistic Model for Object Recognition Zhipeng Zhao, Ahmed Elgammal Department of Computer Science, Rutgers, The](https://reader033.vdocuments.us/reader033/viewer/2022051516/56649ebd5503460f94bc6ab0/html5/thumbnails/30.jpg)
Main References [Dorko 04] G. Dorko and C. Schmid. Object Class Recognition Using
Discriminative Local Features. Submitted to PAMI04. [Fergus 03] R. Fergus, P. Perona and A. Zisserman. Object class recognition
by unsupervised scale-invariant learning. In CVPR(2) page 264-271, 2003. [Fergus 05] R. Fergus, P. Perona and A. Zisserman. A Sparse Object
Category Model for Efficient Learning and Exhaustive Recogntion. [Fischler 73] M. Fischler and R.Elschlager. The representation and matching
of pictorial structures, 1973. IEEE Transaction on Computer c-22(1): 67-92 [Kadir 01] T. Kadir and M. Brady. Scale, saliency and image description.
IJCV 2001 [Lowe 99] D. G. Lowe. Object recogntion from local scale-invariant
features. In Proc. Of the International Conference on Computer Vision ICCV Corfu, pages 1150-1157, 1999
[Opelt 04] A. Opelt, M. Fussenegger, A. Pinz, P. Auer. Weak hypotheses and boosting for generic object detection and recognition. ECCV(2) 71-84, 2004
[Schneiderman 00] H.Schneiderman and T.Kanade. A statistical method 3d object detection applied to faces and cars pages 45-51, 2000
[Viola 02] P.Viola and M. Jones Robust real time object detection. International Journal of Computer Vision 2002.
[Wolfson 97] H.J. Wolfson and I. Rigoutsos. Geometric hashing: An overview. IEEE Computational Science & Engineer, 4(4):10-21,/1997.