student mini-camp project report

Student Mini-Camp Project Report

Pattern Recognition

Participant Students AffiliationsPatrick Choi Claremont Graduate UniversityJoseph McGrath Univ. of Massachusetts, LowellPeizhe Shi University of WashingtonHem Wadhar UC Los AngelesQin Wu West Virginia UniversityFlora Xu Claremont Graduate UniversityAdvisorJen-Mei Chang CSU Long Beach

Problem Statement

Pattern RecognitionThe subject of pattern recognition in data is broadly known as a sub-category of machine learning which is a scientific discipline that is concerned with the design of algorithms that allow artificial intelligence to learn, based on the information given.

We worked on a given set of data which contains distinct images of cats and dogs. The first 160 images are labeled as dogs or cats. And the left 38 images are unlabelled. Our object is to build up pattern recognition architecture on the known data (labeled dogs and cats). Then we use our pattern recognition routine to classify those unknown images (unlabeled dogs and cats) as either dogs or cats correctly.

Pattern Recognition

Can we produce an algorithm/technique/method to train and distinguish between cats and dogs?

Image Pre-Preprocessing

Raw Canny Filtered 2-D Wavelet Transform

PCA LDA

Identification

Model

Image Pre-ProcessingRaw

When using the raw data of a 64x64 pixel image, it was not manipulated. It went directly to either the PCA or LDA method in the next step of the program.

Using the “imread” command in MATLAB, each original TIF image of 80 cats and 79dogs are written to 80 or 79 by 4096 matrix.

Canny Filter Edge Detection Method

Using the matrices created during the raw image pre-processing step, these images were then analyzed using a canny filter edge detection method.

MATLAB automatically calculates the high and low thresholds, and the gaussian filter uses a sigma value of 1.

Image Pre-Processing

Canny Edge Detecting

cat=importdata('cat.mat'); %opening up the matrix “cat”, which contains all of the %raw cat data in one 80x4096 matrix

cat=cat';[m,n]=size(cat);all_cats_edge=zeros(n,m);for j=1:n cat_j=reshape(cat(:,j),64, 64); cat_edge=edge(cat_j,'canny'); file_name=strcat('cat',num2str(j),'.mat'); save(file_name,'cat_edge'); all_cats_edge(j,:)=reshape(cat_edge,1,m);endsave('all_cats_edge.mat','all_cats_edge');

Image Pre-ProcessingRaw Canny

The wavelet decomposition of an 2 D image can be obtained by performing the filtering consecutively along horizontal and vertical directions (separable filter bank). This is depicted schematically in the following figure.

The wavelet decomposition of an 2 D image

LL: low-freq. components

LH: high freq. components in vertical direction

HL: high freq. components in horizontal direction

HH: high freq. components in diagonal direction

Wavelet decompostion


HL

LH

HH

Edge Detection by Wavelet Method

LL: low-freq. components

LH: high freq. components in vertical direction

HL: high freq. components in horizontal direction

HH: high freq. components in diagonal direction

Wavelet decompostion


HL

LH

HH


PCA Method

• PCA transforms many potentially correlated variables to few uncorrelated ones.– This reduces the dimension of the problem so that

we may more easily compare input images to our training sets.

– The lower dimension representation uses the ‘highest energy’ singular vectors as a basis for representation.

PCA Method

The first nine singular vectors for raw image data for dogs & cats.

PCA Method

The first nine singular vectors for the canny filter edge data for dogs.

PCA Method

The first nine singular vectors for the Vertical + Horizontal wavelet data for dogs.

PCA Method

Results using the raw data for training into the PCA methodology

PCA Method

Results using the canny edge filter data for training into the PCA methodology

PCA MethodResults using the Wavelet coefficient horizontal + vertical data, for training into the PCA methodology

PCA Method

Results using the Wavelet coefficient horizontal + vertical + diagonal data, for training into the PCA methodology

PCA Method

• Results:– 17 out of 38 unknown test images identified correctly; test

images were converted to V + H Wavelets

• Take Away:– Potential coding or algorithm flaws

Linear Discriminant Analysis (LDA)

• Idea: project the high dimensional image data linearly into a one dimensional space, where the data is classified using an optimal threshold.

• Main procedures– Feature extraction: Preprocessing the data from training set. – Selecting the optimal direction of projection w.– Determine the optimal threshold c.– Identify the unknown data.

LDA – feature extraction

• Advantage– Lower dimension, faster computation– Discarding redundant information, more efficient classification

• Singular Value Decomposition (SVD)– X: preprocessed images for training

X = USVT

• Feature selection– Features: the first nf columns of U as the principle components.

– New data: the first nf rows of SVT as the extracted information of the images.

– The dimension of the space of data decrease to nf

LDA – Optimal direction of projection

• Goal– maximize the inter-class distance in the projected space– minimize the intra-class distance in the projected space

LDA – Optimal direction of projection

LDA – Optimal threshold

• After projecting all training data to the optimal direction w, pick a threshold such that– Total number of error is minimized– Numbers of error of cats and dogs are equal

• Identification– Feature extraction: project the image on the principle components

xe = Uf x– Compute the projection of the extracted data on optimal direction w

v = wT xe

– Compare the projection with the threshold c to identify the class of the unknown image

LDA - Testing

• Various size of training set• Use the rest for testing• 30 features• 10 trials, shuffled images• Classification rate around 90%

• Training: all 80 dogs and 80 cats• 40 features• Threshold: 43.6• Error: dogs 2, cats 2.

LDA - Comparison

LDA – on the secret data

• Missed 3 out of 38, 2 dogs, 1 cat• Rate of success: 92%

student mini-camp project report

Documents