Computer Vision Scene Classification Using Neural Nets

and a Knowledge Base

Computer Vision Scene Classification Using Neural Nets

and a Knowledge Base

Daniel VevangDaniel Vevang

Object DetectionObject Detection

Object DetectionObject DetectionObject Detection Training

Object DetectionObject Detection

Positive Samples

Object Detection Training

Object DetectionObject Detection

Positive Samples Negative Samples

Object Detection Training

Object DetectionObject Detection

Positive Samples Negative Samples Vector Data

Object Detection Training

Object DetectionObject Detection

Positive Samples Negative Samples Vector Data

XML Haarcascade tree

Object Detection Training

Object DetectionObject Detection

Positive Samples Negative Samples Vector Data

XML Haarcascade tree

OpenCV Output: Object location and scale from an image.

Object Detection Training

Scene DetectionScene DetectionObject Detection Data:

location and scale

Scene DetectionScene DetectionObject Detection Data:

location and scale

Kohonen NetworkScene Detection

Scene DetectionScene DetectionObject Detection Data:

location and scale

Kohonen NetworkScene Detection

NN TrainingInput and Output Data

Scene DetectionScene DetectionObject Detection Data:

location and scale

Kohonen NetworkScene Detection

NN TrainingInput and Output Data

Trained Kohonen Net

Scene DetectionScene DetectionObject Detection Data:

location and scale

Kohonen NetworkScene Detection

NN TrainingInput and Output Data

Trained Kohonen Net

Knowledge Base

Tools: OpenCVTools: OpenCV

Diverse set of computer vision tools Diverse set of computer vision tools

QuickTime™ and a decompressor

are needed to see this picture.

ObjectmarkerObjectmarker

GUI for Creating a text file of bounding box coordinates for a database of images

Additional scripting tools for creating haar xml cascades.

Eyepatch: Advanced scripting tool for training object detectors. Warning! Stability Issues!

GUI for Creating a text file of bounding box coordinates for a database of images

Additional scripting tools for creating haar xml cascades.

Eyepatch: Advanced scripting tool for training object detectors. Warning! Stability Issues!

Kohonen Net ImplementationKohonen Net Implementation

Code modified from Karsten Kutsa Still in the process of creating the data

model for Neural Net input. Currently looking to create 8 input nodes

for each image (8*5 images) for 40 images total.

Code modified from Karsten Kutsa Still in the process of creating the data

model for Neural Net input. Currently looking to create 8 input nodes

for each image (8*5 images) for 40 images total.

Kohenen Net Implementationfor detected images A-E

Kohenen Net Implementationfor detected images A-E

Example inputExample input

A B C D E

1.0 0.0 0.0 0.0 0.0

0.0 1.0 0.0 0.0 0.0

0.0 0.0 1.0 0.0 0.0

0.0 0.0 0.0 1.0 0.0

0.0 0.0 0.0 0.0 1.0

0.0 0.0 0.0 0.0 0.0

0.0 0.0 0.0 0.0 0.0

0.0 0.0 0.0 0.0 0.0

0.0 0.0 0.0 0.0 0.0

Parameters to work withParameters to work with

Learning rate for Kohonen layer Learning rate for output layer Learning rate for step sizes Smoothing factor for score deltas Parameter for width of neighborhood

Learning rate for Kohonen layer Learning rate for output layer Learning rate for step sizes Smoothing factor for score deltas Parameter for width of neighborhood

Additional data to considerAdditional data to consider

x y location scale of each object Multiples of the same object

x y location scale of each object Multiples of the same object

Knowledge baseKnowledge base

Possible implementation of Narl to augment the performance of the Neural Net.

Possible implementation of Narl to augment the performance of the Neural Net.