cse 415 -- (c) s. tanimoto, 2008 image understanding i 1 image understanding 1 outline: saccade art:...

CSE 415 -- (c) S. Tanimoto, 2008 Image Understanding I

1

Image Understanding 1Outline:

Saccade Art: What's going on?Vision and IntelligenceMotivating applications and ideasHuman vision and illusionsImage representation: Sampling, Quantization, ThresholdingStereo vision as an AI problem Stereograms, Geometry of stereograms, Computing correspondencesLetting cues vote for hypotheses: Polar representation of a line, Hough transform


2

Saccade Art

A great example is at the San Francisco Exploratorium.

An online example is athttp://www.cs.washington.edu/research/metip/SaccArt/

Block off all but one stripe. Can you still see the effect? How few stripes are enough for you?

Is it an image? Is there any image?


3

Is Vision Part of Intelligence?

25% of the brain by volume is concerned with vision. This is Brodmann area 17, part of the striate (visual) cortex.

image from: http://en.wikipedia.org/wiki/Primary_Visual_Cortex


4

Vision Requires Intelligence1. The image is usually missing relevant information.

2. The gaps must be filled in by making inferences using knowledge and context.

3. How are these inferences made? At many levels: At the retinal level, relevant structure is extracted from "receptive fields". The brain sends expectations to the eyes. An intermediate level theory is the Marr "Primal Sketch". High level: object-recognition.

4. Active vision is the use of motion, including eye movements, to gather scene data – the eyes must be controlled by the brain.

5. "Visual thinking" includes spatial reasoning, navigation, pattern recognition, associative memory based on visual aspects, and perception itself.

6. Specific phenomena that reflect on visual intelligence include: unawareness of blind spots, visual illusions, pareidolia, hallucination.


5

MotivationAllow computer and robots to read books.

Allow mobile robots to navigate using vision.

Support applications in industrial inspection, medical image analysis, security and surveillance, and remote sensing of the environment.

Permit computers to recognize users’ faces, fingerprints, and to track them in various environments.

Provide prostheses for the blind.

Develop artistic intelligence.


6

Human Vision25% of brain volume is allocated to visual perception.

Human vision is a parallel & distributed system, involving 2 eyes, retinal processing, and multiple layers of processing in the striate cortex.

Most humans are trichromats and they perceive color in a 3-D color space (except for bichromats and monochromats).

Vision provides a high-bandwidth input mechanism... “a picture is worth 1000 words.”


7

The Human Eye


8

Retina:Cross

section(a) schematic

(b) photo


9

Densities of Rods and Cones


10

Visual Pathway


11

Visual Illusions

Help us understand

• the limits of human perception

• the processes of perception

• ways to produce effects in art and architecture

• possible approaches to artificial perception


12

Visual IllusionsThey provide insights about the nature of the human visual system, helping us understand how it works.

Mueller-Lyer illusion


13

Herman Grid Illusion


14

Herman Grid Illusion (dark on light)


15

Subjective Contour (Triangle)


16


17

?


18

Dalmation Illusion

Camouflage vs Acute Perception

Hyperacute perception: Hallucination

Pareidolia: Attributing significance to patterns perceived in random arrangements

(UFO, visions in the twilight, etc)

From http://www.gifford.co.uk/~principia/Illusions


19

Perception: Stimulus + Expectation

Image understanding is the interpretation of visual stimuli using context and knowledge.

IU by computer normally begins with digital images from a camera.


20

Image UnderstandingOutline:

Saccade Art: What's going on?Vision and IntelligenceMotivating applications and ideasHuman vision and illusionsImage representation: Sampling, Quantization, ThresholdingStereo vision as an AI problem Stereograms, Geometry of stereograms, Computing correspondencesLetting cues vote for hypotheses: Polar representation of a line, Hough transform


21

Image RepresentationSampling: Number and density of “pixel” measurementsQuantization: Number of levels permitted in pixel values.


22

Image Representation (cont.)Sampling: e.g., 4 by 4, square grid, 1 pixel/cmQuantization: e.g., binary, {0, 1}, 0 = black, 1 = white.

1

0

11

1

11

00

00

0

0 0 0 0


23

Aliasing due to Under-sampling

Here the apparent frequency is about 1/5 the true frequency.


24

Shannon/Nyquist SamplingA band is a range of frequency values. (But sometimes it's defined as a range of wavelengths.)

A signal that is bandlimited to band B has no frequency components outside of B.

Now, we'll assume B = [0 , fmax]

Theorem:

If a continuous signal z(t) is bandlimited to B, then it is possible to sample it at a frequency of fs > 2 fmax such that z(t) can be perfectly reconstructed from the samples.

2 fmax is called the Nyquist rate.

fs / 2 is called the Nyquist frequency, and depends on fs.


25

Minimal SamplingLet P be an oscillating pattern in an image.

To capture P in a sampled representation, you need

(1) The rest of the image to be bandlimited to P's frequency, and you need either

(2a) two samples per cycle and luck (the phase must be right),

or

(2b) more than two samples per cycle.


26

QuantizationCapturing a wide dynamic range of brightness levels or colors requires fine quantization. Common is 256 levels of each of red, green and blue.

Segmentation is simplified by having a small number of levels -- provided foreground and background pixels are reliably distinguished by their dark or light value.

Grayscale thresholding is typically to used to reduce the number of quantization levels to 2.


27

Vision as Inferring Information from Clues

Deriving 3D structure from 2D info requires additional information: e.g., constraints.

Deriving global descriptions from local data requires information fusion, i.e., inference.


28

Stereo Vision as an AI Problem

Projection from 3 dimension to 2 loses information.

With 2 projections, we can gain back some of that information.

Recovering the missing information is an inference problem.

The missing information is constrained by knowledge about the real world and assumptions about the scene.

The use of knowledge and assumptions to make inferences is a standard approach in artificial intelligence.


29

Stereo and Stereograms

A stereogram can help us understand what information is required by a human to make convincing inferences about depth.

This can provide a model for a stereo image understanding system.


30

StereogramsTwo-view stereograms:1. spatially separated left-eye/right-eye pair (including virtual-reality goggles)2. superimposed, with separation using color filters.3. superimposed, with temporal shuttering.4. superimposed, with separation using polarizing filters.

Single-view stereograms:1. Magic-eye pictures with depth-modulated carrier.2. Wallpaper offering depth effects due to its periodicity.


31

Geometry of Stereograms


32

Computing Correspondence

Approach 1: Extract features and find a consistent matching of features in each view.

Approach 2: Directly compute a disparity map, performing local correlations of the views.


33

Processing Incomplete and Uncertain Evidence:

How it's sometimes handled in image understanding

Case study: the Hough Transform

(rhymes with "rough France dorm")


34

Inferring Trends via Voting Methods

The classical Hough Transform identifies prominent lines in a scene by letting each edge point vote for the line(s) it is on.

Voting methods can do well under noisy conditions.

Votes are tallied in an array of accumulators, indexed by theta and rho (polar parameters of a line).

ρ = x cos θ + y sin θ.


35

Letting a Point Vote for all the Lines that Pass Through It


36

Hough Transform: Polar representation

ρ = x cos θ + y sin θ.

ρ

θ

(x, y)

(0, 0)


37

Hough Transform (Cont.)

nondirectional, unweighted Hough Transform:

H(θ,ρ) = Σ Σ f(x,y) δ(x cos θ + y sin θ - ρ).

δ(z) = 1 if | z | < 1 0 otherwise


38

H.T. Peak Detection

After vote accumulation:

Apply smoothing to suppress non-dominant peaks.

Extract peaks.

Trace lines in image space to determine endpoints.

cse 415 -- (c) s. tanimoto, 2008 image understanding i 1 image understanding 1 outline: saccade art:...

Documents

image understanding

illusions image representation

cortex slide

medical image analysis

visual intelligence

active vision

ideas human vision

human eye slide