Vision:Man Vs. Machine

By Jason Pele

Vision Taken for Granted

Much about human vision remains unclear

How does the brain work with the eye to form perceptions and interpretations of what we see?

Billions of neural cell connections within the brain present a large complexity problem

Yet human vision is fallible. Illusions and ambiguities are encountered all the time

Computer Vision

Development of a theoretical and algorithmic base for which useful information about the world can be extracted and analyzed from images.

Goal it to comprise a computer system that is closely modeled after the human visual system

Image formation, low and high level processing and then 3-D description/interpretation

Developing CV based on HVS

Scene/Image

Computer Man/Eyes

Processing in eyes/brainProcessing (MATLAB)

High Level Processing (interpretation) High Level Brain functions

3-D scene

description

Low Level Processing: What a computer sees, and ways to analyze it

Matrix of pixels where values usually represent the

intensity of image at that point

In MATLAB, there are different image types, usually

grayscale and RGB are used where RGB is a NxNx3

array.

Simple feature computations: intensity, color, edges

Organization: grouping of pixels, regions, lines

Higher feature computations: patterns, textures,

geometrical shape descriptions

RGB image (GREG) and histogram

GREG

0 50 100 150 200 2500

1000

2000

3000

4000

5000

6000

N

u

m

b

e

r

o

f

p

i

x

e

l

s

Intensity or RGB value

Greg histogram

Intensity profile

GREG

0 20 40 60 80 100 120 140 160 1800

20

40

60

80

100

120

140

Distance along profile

I

n

t

e

n

s

i

t

y

v

a

l

u

e

s

Intensity profile along Greg's scales

0 100 200 300 400 500 600 70080

85

90

95

100

105

110

115

120

125

130

Distance along profile

I

n

t

e

n

s

i

t

y

v

a

l

u

e

s

Profile along ground (plastic container)

RGB image (Samurai) and histogram

Samurai on a flat plane

0 50 100 150 200 2500

1000

2000

3000

4000

5000

6000

7000

Intensity or RGB values

N

u

m

b

e

r

o

f

P

i

x

e

l

s

Samurai Histogram

Intensity Profile

Samurai on a flat plane

0 100 200 300 400 500 6000

50

100

150

200

250

Distance along profile

Samurai

I

n

t

e

n

s

i

t

y

v

a

l

u

e

s

0 100 200 300 400 500 600 700130

140

150

160

170

180

190

200

Distance along profile

Edge detection

Using the Sobel method: Spatial gradient

measurement using two 3x3 convolutional kernels

Perceptions or Hypotheses

How much of what we perceive is based on prior

knowledge or experience?

Viewing the 3D World

We can get a lot of information from a 2-D

image, but what about distances of objects,

curvatures, shapes, etc.

Humans are equipped with binocular vision

(two eyes)

Computer vision must then utilize multiple

images at multiple angles from multiple

calibrated cameras.

Homography: 2D to 2D mapping

Maps points in an image plane as seen from

one camera to points in the same image

plane as seen from a different camera.

Useful for determining positioning of objects

in a picture

Introduces the problem in CV known as point

correspondence that becomes even more

difficult in a 3D to 2D mapping

Samurai on a flat planeSamurai at an angle

Match points

in MATLAB

using

CPSELECT

function

Samurai at an angle mapped to flat plane

MAT in a flat plane MAT at an angle

MAT at angle mapped to flat plane

Summary

CV is difficult because we dont have a concrete model (HVS) to base it on.

We must take an array of numbers as an input and come up with ways to describe textures, colors, depth, objects, and even more complex interpretations such as action and situation

The computer can see the input image/scene in many different ways, the key is to understand how to combine these representations to reach the goal of having a machine or computer view the world as a human does

Other relative topics

Biological vision

Image and signal processing

Neurophysiology

Psychology

Psychophysics

Other Cognitive sciences

The list goes on.

Sources and further reading/research

Bennamoun, M. & Mamic, G.J. Object Recognition Fundamentals and Case Studies. Britain: Springer-Verlag, 2002.

Gregory, Richard. Eye and Brain. New Jersey: Princeton University Press, 1997.

Levine, Martin. Vision in Man and Machine. New York: McGraw-Hill Inc, 1985.

Nalwa, Vishvjit. A Guided Tour of Computer Vision. New York: Addison-Wesley Publishing Company & AT&T, 1993.