Content Based Image Organization

CS491 Spring 2006Prof. Chengyu Sun

Kelly Breed

Overview of FindPic

What it does

How it works (from a user standpoint)

Modes

– grayscale– quantized color– texture for both

Technology

Objective-C

Cocoa framework

Local application

Self Organizing Maps

– T. Kohonen– 1980s

Technique: SOM Overview

Gather image data (feature vectors)

Create matrix (with same size vectors)

For each image find best match in matrix

Change matrix node and surrounding nodes

Reduce neighborhood size and change factor

Re-run for a number of epochs

Technique: Image Data

Reduction

Features

– e.g. grayscale histogram

Storage

Setting up the Map

2D Array of matrix nodes

Same feature vectors

Initialized to random values

Mapping begins

Finding the Best Match

Euclidean distance between the image (input) vectors and the matrix vectors

D mV 0 iV 0 2 mV 1 iV 1 2 ... mV n 1 iV n 1 2

Technique: Mapping

Find best match and change that node

Technique: Mapping

Change the surrounding nodes

Technique: Mapping

Change the surrounding nodes

Making Changes

After best match is found change the matrix node is made to be more like the input node

e.g. matrixVector[0] = 50, imageVector[0] = 30, cf = 0.9

matrixVector[0] = 50 – 0.9(50 – 30) = 32

e.g. matrixVector[0] = 30, imageVector[0] = 50, cf = 0.9

matrixVector[0] = 30 – 0.9(30 – 50) = 48

matrixVector i matrixVector i cf matrixVector i imageVector i

Finding the Neighbors

Inverse parabola

Filters for neighbors

Also determines change factor

Change Over Time

Inverse parabola determines neighborhood and change factor

yx2

Mt Where M=matrix dimension and t = time

Sample Neighborhood

Best Matching unit is set to position (1,1) with t = 1

0.8 0.9 0.8 0.5 0.0 -0.7 0.9 1.0 0.9 0.6 0.1 -0.6 0.8 0.9 0.8 0.5 0.0 -0.7 0.5 0.6 0.5 0.2 -0.3 -1.0 0.0 0.1 0.0 -0.3 -0.8 -1.5-0.7 -0.6 -0.7 -1.0 -1.5 -2.2

Sample Neighborhood

Best Matching unit is set to position (1,1) with t = 0.3

0.1 0.2 0.1 -0.2 -0.7 -1.4 0.2 0.3 0.2 -0.1 -0.6 -1.3 0.1 0.2 0.1 -0.2 -0.7 -1.4-0.2 -0.1 -0.2 -0.5 -1.0 -1.7-0.7 -0.6 -0.7 -1.0 -1.5 -2.2-1.4 -1.3 -1.4 -1.7 -2.2 -2.9

End Results

Images with similar feature vectors should be grouped into the same matrix cells.

Surrounding cells should also contain images that are similar.

Conclusions

SOM results compared to human categorized results

Determining closeness of match

Average percentages

– Grayscale: 41.46% – Quantized color: 55.14%– Grayscale with texture: 50.01%– Quantized with texture: 56.03%