Patch-Based Image Classification Using Image Epitomes

David AndrzejewskiComputer Sciences 766

Fall 2005

Problem Statement

Given Positive and negative example images for a certain classification (contains face, is outdoors, etc)

DoDevelop classifier capable of classifying new images as positive or negative

Image Epitomes

Consists of patches and mappingsPatches and mappings are learned with EMApplications in vision (de-noising,segmentation,others)

Input image A set of image patches Image epitome

www.research.microsoft.com/~jojic/epitome.htm

Image Reconstruction

Original image can then be reconstructed by mosaicing

epitome patches

www.research.microsoft.com/~jojic/epitome.htm

Recognition / Detection / Classification

Epitome of 295 face images

The smiling point

Images with the highest total posterior at the “smiling point”

Images with the lowest total posterior at the “smiling point”

www.research.microsoft.com/~jojic/epitome.htm

Approach● Construct collage of positive and negative examples● Learn the image epitome of the training collage● Find epitome patches that are preferentially mapped into the positive example images in the collage● Calculate P(patch(i)|pos/neg) for these patches (also use psuedo-counts)● Use these patches to classify new images by calculating odds ratio

Preliminary Results Training Collage

Negative Test Images

Epitome

Positive Test Images

Problems with Approach

● Difficult to incorporate new examples ● Would need to add to collage and re-learn

epitome (is there a better way?)● “Bag of words” → Spatial information discarded● Not model-based

● Pose/Illumination/Scale-variant ● Only way to handle variation is to include

training examples for various conditions

Potential Modifications● Cluster training images

● Ex: Training images w/ low vs high illumination● Discriminative patches may map exclusively to one

subset of positive images → take this into account

● Change “winner take all” for P calculations ● Consider relative probabilities of 'near matches'● Account for multiple mappings somehow

References

1. V. Cheung, B. J. Frey, and N. Jojic, Video epitomes, Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2005

2. N. Jojic, B. J. Frey, and A. Kannan, Epitomic analysis of appearance and shape, Proc. 9th Int. Conf. Computer Vision, 2003

3. R. Fergus, P. Perona, A. Zisserman, Object Class Recognition by Unsupervised Scale-Invariant Learning, Proc. of the IEEE Conf on Computer Vision and Pattern Recognition, 2003

Testing images from Google Images and Flickr