A Framework for Scene Recognition Using

Convolutional Neural Network as Feature Extractor and

Machine Learning Kernels as Classifier

Tahmid Abtahi, Aniket Badhan and Sri HarshaDepartment of Computer Science & Electrical Engineering

University of Maryland, Baltimore County{abtahi1,yh02299,ksrihar1}@umbc.edu

Frame work Why?

-Scene recognition is valuable in computer vision -Autonomous navigation

-DataSet ?

- VGG set-Three category-High way-Open Country-Streets

-678 Images

Convolutional Neural Network (CNN) – Feature Extractor

Why Feature Extraction ?

-Data Dimensionality Reduction10 x times in propsed case

--Key Feature Extraction

Algorithm :ConvolutionPseudo Code Convolution:

for Patch_x_axis_movement{ initialize sum =0; for Patch_y_axis_movement{ calculate dot product (Patch, Filter);

} result_convolution (x,y) = Dot product; }

Convolution Operation calculation

Data size Filter size Stride

Number of Patch Addition op per patch

Multiplication op. per

patch

Total op Order

nxn mxm 1 (n-m+1)x (n-m+1) m-1 m m(m-1) (n-m+1)2

O(n3)

64x64 9x9 1 56x56 = 3136 8 9 2,25,792

Algorithm : Max Pool, ReLUPseudo Code Maxpool:

for Patch_x_axis_movement{ for Patch_y_axis_movement{ calculate Max (Patch);

} result_maxpool (x,y) = Dot product; }Pseudo Code ReLU:

update F = max(0, x)

Maxpool Operation calculationData size Patch size Factor Number of Patch Comparision op per

patchTotal op Order

nxn mxm m (n/m)x (n/m) m-1 (m-1) (n/m)2 O(n2)56x56 7x7 7 8x8=64 6 384

Linear Rectification Operation calculation

Data size op per element Total op Order

nxn 1 n2 O(n2)8x8 1 64

1x Convolution1/588x Maxpool1/3528x times ReLU

Implementation and Result

Communication dominating computation

10x dimensionality Reduction

Pre Work

- Training CNN in MatConvNet --Creating matlab Prototype for cross checking checking results

Perceptron• Algorithm for supervised learning of binary classifiers• Function that maps its inputs x to an output value f(x)

• w is the vector of real valued weights• f(x) is the output• m is the no. of inputs to the perceptron • b is the bias

Pseudo code

Parallel Implementation

• In the training phase, One vs All is implemented for each output class. This process can be done in parallel, with each processor handling divided output classes.

• While testing , the test data is divided amongst the processors, and each processor will report its local accuracy to the Master node. In the Master node, it will use MPI Reduce to sum the received local accuracy value and return the final accuracy and the accuracy percentage.

Results

Results

Results

Varying the Training and keeping the Testing Data size as constant and Processor count as 4

SVM Classifier• Support Vector Machines (also popularly known as SVM or Support Vector

Networks) are supervised learning models with associated learning algorithms that analyze data used for classification.

• Used for binary Classification

SVM One versus All• From the above discussion on SVM, it’s clear that the above model of

SVM can be used for binary classification only, i.e., it can separate only two classes. • So, for classification of data points with more than 2 possible outputs,

the above model would fail. • Hence, to overcome this, there is an enhancement in SVM which is

SVM OVA (One versus All). • In this model, one output class is separated from the other output

classes and hence, the name One Versus All.

Parallel Implementation• The training phase will be implementation of OVA for each output class.

This process can be done in parallel, with each processor handling divided output classes. • For the testing phase, the number of test data points can be divided

amongst the processor, and each processor can report its local accuracy to the Master node. • The Master node will use MPI Reduce to sum the received local accuracy

and report the final accuracy and the accuracy percentage.• In the implementation, all the processors do work, including the master

node, which ensures load balancing as the tasks are the same.

Results

Results

Increasing the Training Data size and keeping the Testing Data size and Number of Processors as constant

Comparison Between SVM and Perceptron• SVM includes the factor (1/input number) in weight calculation.• This helps in getting better accuracy as any input which has noise can

not have much impact on the weight values.• Also, Lambda is used as an hyper-parameter in weight calculation,

which takes care of the over-fitting and under-fitting.• Accuracy obtained using Perceptron is comparatively lesser than SVM,

taking almost the same time for computationTraining Data Size Testing Data Size Number of Processors Time Required for SVM

Time Required for

Perceptron Accuracy for SVM Accuracy for Perceptron4000 1000 1 0.658 0.617 75.5 72.194000 1000 2 1.255 1.283 75.5 72.194000 1000 3 1.772 1.955 75.5 72.194000 1000 4 2.408 2.366 75.5 72.194000 1000 5 2.966 3.035 75.5 72.194000 1000 6 3.769 4.11 75.5 72.194000 1000 7 5.567 6.987 75.5 72.194000 1000 8 8.016 8.037 75.5 72.19

Future Scope• Task Level Parallel Implementation of SVM and Perceptron• Increasing the Data Size and Output Classes and use the prepared

framework for advanced scene recognition • Comparing the results of PCA (Principal Component Analysis) with

CNN and compare which gives the better accuracy.• Implementing other ML Kernels such as Logistic Regression, k-NN,

etc.• CNN – GO DEEP

Questions?