physiology of vision: a swift overview pixels to percepts a. efros, cmu, spring 2011 some figures...

Physiology of Vision: a swift overview

Pixels to PerceptsA. Efros, CMU, Spring 2011

Some figures from Steve Palmer

ThalamusLGN

Striatecortex(V1)

DorsalStream

Parietalvisualcortex

Temporalvisualcortex

Eye Opticnerve

Extrastriatecortex

VentralStream

© Stephen E. Palmer, 2002

Understanding the BrainUnderstanding the Brain

Anatomy: The biological study of thephysical structure of organisms.

Physiology: The biological study of thefunctional structure of organisms.

Anatomy versus Physiology


The Brain is tricky business

• Aristotle thought it’s for cooling the blood• Localized or distributed?

Phrenology

How bad Machine Learning got started…

Localized or Distributed?

Evidence from patients with partial brain damage• Lots of useful data from soldiers in the Russo-Japanese war

But also evidence for distributed nature of processing• E.g. [Lashley] showed “graceful degradation” of memory

performance in rats


The Visual SystemThe Visual System

Both eye and brain are required for functional vision

Two kinds of blindness: Normal blindness (eye dysfunction) Cortical blindness (brain dysfunction)



The Visual SystemThe Visual System

Eyes register optical information Pathways to occipital cortex

“What” pathway to temporal cortex “Where” pathway to parietal cortexConvergence on frontal cortex

Two pathways from V1



Pathways to the BrainPathways to the Brain

Anatomy of Pathway to Visual Cortex



Pathways to the BrainPathways to the Brain

The Lateral Geniculate Nucleus (LGN)

Waystation in ThalamusProjections from both eyesSix layersProjects to cortical area V1



Visual CortexVisual Cortex

“Unfolded” view ofvisual areas in the macaque cortex(sizes not to scale).

Map of Visual Areas in Cortex




Evidence from lesionsof monkey cortex

What/Where Pathways

PrimaryVisualCortex

Dorsal Pathway("Where" System)

Ventral Pathway("What" System)

A

TemporalOccipital

Parietal

Frontal

Ventral Lesion

Dorsal Lesion

Object Discrimination Landmark Discrimination

B C



Visual agnosiaVisual agnosia: Inability to identify objects and/or people Caused by damage to inferior (lower) temporal lobe Disruption of the “what” pathway

What/Where PathwaysEvidence from Neuropsychology

Visual neglectVisual neglect: Inability to see objects in the left visual field Caused by damage to right parietal lobe Disruption of the “where” pathway




Visual FeaturesVisual Features Color Shape Depth Motion

Feature-based Pathways Hypothesis

Featural PathwaysFeatural Pathways Separate neural pathwaysin which different features are processed. RETINA

LGN

V1

V2 MT

ColorForm

DepthMotion

ColorForm

DepthMotion

Color Form DepthMotion

Color Form Depth Motion



The Gross SummaryThe Gross Summary

The visual system is composed of many interactive functional parts:

Eye (optics of image formation)Retina (light transduction)LGN (waystation?)Area V1 (hypercolumns)Higher cortical areas (features)Cortical pathways (what/where)


Image Formation

Digital Camera

The Eye

Film

Monocular Visual Field: 160 deg (w) X 135 deg (h)Binocular Visual Field: 200 deg (w) X 135 deg (h)

The Eye is a camera

The human eye is a camera!• Iris - colored annulus with radial muscles

• Pupil - the hole (aperture) whose size is controlled by the iris

• What’s the “film”?– photoreceptor cells (rods and cones) in the retina

The Retina

Cross-section of eye

Ganglion cell layer

Bipolar cell layer

Receptor layer

Pigmentedepithelium

Ganglion axons

Cross section of retina

Retina up-close

Light


Cones cone-shaped less sensitive operate in high light color vision

Two types of light-sensitive receptors

cone

rod

Rods rod-shaped highly sensitive operate at night gray-scale vision

Rod / Cone sensitivity

The famous sock-matching problem…


Distribution of Rods and Cones.

0

150,000

100,000

50,000

020 40 60 8020406080

Visual Angle (degrees from fovea)

Rods

Cones Cones

Rods

FoveaBlindSpot

# R

ecep

tors

/mm

2

Night Sky: why are there more stars off-center?

Electromagnetic Spectrum

http://www.yorku.ca/eye/photopik.htm

Human Luminance Sensitivity Function

Why do we see light of these wavelengths?


.

0 1000 2000 3000

En

erg

y

Wavelength (nm)

400 700

700 C

2000 C

5000 C

10000 C

VisibleRegion

…because that’s where theSun radiates EM energy

Visible Light

The Physics of Light

Any patch of light can be completely describedphysically by its spectrum: the number of photons (per time unit) at each wavelength 400 - 700 nm.

400 500 600 700

Wavelength (nm.)

# Photons(per ms.)



.

# P

ho

ton

s

D. Normal Daylight

Wavelength (nm.)

B. Gallium Phosphide Crystal

400 500 600 700

# P

ho

ton

s

Wavelength (nm.)

A. Ruby Laser

400 500 600 700

400 500 600 700

# P

ho

ton

s

C. Tungsten Lightbulb

400 500 600 700

# P

ho

ton

s

Some examples of the spectra of light sources



Some examples of the reflectance spectra of surfaces

Wavelength (nm)

% P

hoto

ns R

efle

cted

Red

400 700

Yellow

400 700

Blue

400 700

Purple

400 700


The Psychophysical Correspondence

There is no simple functional description for the perceivedcolor of all lights under all viewing conditions, but …...

A helpful constraint: Consider only physical spectra with normal distributions

area

Wavelength (nm.)

# Photons

400 700500 600

mean

variance



Mean Hue

yellowgreenblue

# P

hoto

ns

Wavelength



Variance Saturation

Wavelength

high

medium

low

hi.

med.

low# P

hoto

ns



Area Brightness#

Pho

tons

Wavelength

B. Area Lightness

bright

dark



.

400 450 500 550 600 650

RE

LAT

IVE

AB

SO

RB

AN

CE

(%

)

WAVELENGTH (nm.)

100

50

440

S

530 560 nm.

M L

Three kinds of cones:

Physiology of Color Vision

• Why are M and L cones so close?

Retinal Processing


Single Cell Recording

Microelectrode

Amplifier

Time

Electrical response(action potentials)

mV


Single Cell Recording


Retinal Receptive Fields

Receptive field structure in ganglion cells:On-center Off-surround

Stimulus condition Electrical response

Time

Response


Receptive field structure in ganglion cells:On-center Off-surround

Stimulus condition Electrical response

Time

Response



RF of On-center Off-surround cells

Receptive FieldNeural Response

Center

Surround

On Off

Response Profile

on-center

off-surround

Horizontal Position

FiringRate



RF of Off-center On-surround cells

Receptive Field

Horizontal Position

on-surround

off-center

Response Profile

FiringRate



Center

Surround

On Off

Neural Response

Surround

Center

Receptive field structure in bipolar cells

Light



Receptive field structure in bipolar cells

Receptors

Bipolar Cell

A. WIRING DIAGRAM

HorizontalCells

Direct excitatory component (D)

B. RECEPTIVE FIELD PROFILES

LIGHT

Direct Path

Indirect Path

Indirectinhibitory

component (I)

D + I





aka:Primary visual cortexStriate cortexBrodman’s area 17

Cortical Area V1

ThalamusLGN

Striatecortex(V1)

DorsalStream

Parietalvisualcortex

Temporalvisualcortex

Eye Opticnerve

Extrastriatecortex

VentralStream

Cortical Receptive Fields

Single-cell recording from visual cortex

David Hubel & Thorston Wiesel© Stephen E. Palmer, 2002


Single-cell recording from visual cortex

TimeTime



Three classes of cells in V1

Simple cells

Complex cells

Hypercomplex cells



Simple Cells: “Line Detectors”

A. Light Line Detector

Horizontal Position

FiringRate

B. Dark Line Detector

Horizontal Position

FiringRate



Simple Cells: “Edge Detectors”

C. Dark-to-light Edge Detector

Horizontal Position

FiringRate

D. Light-to-dark Edge Detector

Horizontal Position

FiringRate



Constructing a line detector

Receptive Fields

Retina LGN

Center-Surround Cells

Simple Cell

CorticalArea V1



Complex Cells

STIMULUS NEURAL RESPONSE

Time

00o



Complex Cells


Time

060o



Complex Cells


Time

090o



Complex Cells


Time

0120o



Constructing a Complex Cell

Simple Cells

Cortical Area V1

Complex CellReceptive Fields

Retina



Hypercomplex Cells

Time




Time


“End-stopped” Simple Cells



Constructing a Hypercomplex Cell

Receptive Fields

RETINA CORTICAL AREA V1

Complex Cell End-stopped Cell


Mapping from Retina to V1Mapping from Retina to V1

Why edges?

physiology of vision: a swift overview pixels to percepts a. efros, cmu, spring 2011 some figures...

Documents

steve palmer stephen

visual cortex stephen

pathway stephen

physiology stephen

rats stephen

cortical area v1 stephen

visual cortexevidence

visual systemboth eye