the brain from retina to extrastriate cortex. neural processing responsible for vision...
TRANSCRIPT
![Page 1: The Brain from retina to extrastriate cortex. Neural processing responsible for vision photoreceptors retina –bipolar and horizontal cells –ganglion cells](https://reader035.vdocuments.us/reader035/viewer/2022062718/56649e845503460f94b85a59/html5/thumbnails/1.jpg)
The Brain
from retina to extrastriate cortex
![Page 2: The Brain from retina to extrastriate cortex. Neural processing responsible for vision photoreceptors retina –bipolar and horizontal cells –ganglion cells](https://reader035.vdocuments.us/reader035/viewer/2022062718/56649e845503460f94b85a59/html5/thumbnails/2.jpg)
Neural processing responsible for vision
• photoreceptors• retina
– bipolar and horizontal cells– ganglion cells (optic nerve)
• optic nerves• optic chiasma (X)• lateral geniculate body• striate cortex• extrastriate cortex
![Page 3: The Brain from retina to extrastriate cortex. Neural processing responsible for vision photoreceptors retina –bipolar and horizontal cells –ganglion cells](https://reader035.vdocuments.us/reader035/viewer/2022062718/56649e845503460f94b85a59/html5/thumbnails/3.jpg)
Photoreceptors
Ganglion cells
Light
Lateral inhibition
• Edge detection and contrast enhancement
• Bipolar, Horizontal and Ganglion cells
![Page 4: The Brain from retina to extrastriate cortex. Neural processing responsible for vision photoreceptors retina –bipolar and horizontal cells –ganglion cells](https://reader035.vdocuments.us/reader035/viewer/2022062718/56649e845503460f94b85a59/html5/thumbnails/4.jpg)
1000 0
100
Lateral inhibition
• If no activity in neighboring photoreceptors,
output = output of photoreceptor
![Page 5: The Brain from retina to extrastriate cortex. Neural processing responsible for vision photoreceptors retina –bipolar and horizontal cells –ganglion cells](https://reader035.vdocuments.us/reader035/viewer/2022062718/56649e845503460f94b85a59/html5/thumbnails/5.jpg)
100100 100
0
Lateral inhibition
• if activity in neighboring photoreceptors,– output is decreased, possibly absent
![Page 6: The Brain from retina to extrastriate cortex. Neural processing responsible for vision photoreceptors retina –bipolar and horizontal cells –ganglion cells](https://reader035.vdocuments.us/reader035/viewer/2022062718/56649e845503460f94b85a59/html5/thumbnails/6.jpg)
100100 100
0
(-.5) (-.5)
-50 -50
+
(1.0)
100
200200 200
0
(-.5) (-.5)
-100 -100
+
(1.0)
200
Lateral inhibition via addition and negative weights
![Page 7: The Brain from retina to extrastriate cortex. Neural processing responsible for vision photoreceptors retina –bipolar and horizontal cells –ganglion cells](https://reader035.vdocuments.us/reader035/viewer/2022062718/56649e845503460f94b85a59/html5/thumbnails/7.jpg)
cornea
crystallinelens
retina: photoreceptors = rods + cones
opticnerve
Optic nerve
• axons of the ganglion cells– 1 million optic nerves– 120 million photoreceptors
![Page 8: The Brain from retina to extrastriate cortex. Neural processing responsible for vision photoreceptors retina –bipolar and horizontal cells –ganglion cells](https://reader035.vdocuments.us/reader035/viewer/2022062718/56649e845503460f94b85a59/html5/thumbnails/8.jpg)
From light to vision
Lateral Geniculate Nucleus (LGN)
StriateCortexGeniculo-Striate Pathway
![Page 9: The Brain from retina to extrastriate cortex. Neural processing responsible for vision photoreceptors retina –bipolar and horizontal cells –ganglion cells](https://reader035.vdocuments.us/reader035/viewer/2022062718/56649e845503460f94b85a59/html5/thumbnails/9.jpg)
(LGN)
StriateCortex
Striate cortex(primary visual centre)
• Neurons are edge detectorsfires when an edge of a particular orientation is present
![Page 10: The Brain from retina to extrastriate cortex. Neural processing responsible for vision photoreceptors retina –bipolar and horizontal cells –ganglion cells](https://reader035.vdocuments.us/reader035/viewer/2022062718/56649e845503460f94b85a59/html5/thumbnails/10.jpg)
(LGN)
StriateCortex
Striate cortex(primary visual centre)
• Neurons are edge detectorsfires when an edge of a particular orientation is present
frequent output
vertical bar
![Page 11: The Brain from retina to extrastriate cortex. Neural processing responsible for vision photoreceptors retina –bipolar and horizontal cells –ganglion cells](https://reader035.vdocuments.us/reader035/viewer/2022062718/56649e845503460f94b85a59/html5/thumbnails/11.jpg)
(LGN)
StriateCortex
Striate cortex(primary visual centre)
• Neurons are edge detectorsfires when an edge of a particular orientation is present
infrequent output
diagonal bar
![Page 12: The Brain from retina to extrastriate cortex. Neural processing responsible for vision photoreceptors retina –bipolar and horizontal cells –ganglion cells](https://reader035.vdocuments.us/reader035/viewer/2022062718/56649e845503460f94b85a59/html5/thumbnails/12.jpg)
Edge detection
• each cell “tuned” to particular orientation– vertical– horizontal– diagonal
• cats: only horizontal and vertical• humans: 10 degree steps• edges at particular orientations and positions
![Page 13: The Brain from retina to extrastriate cortex. Neural processing responsible for vision photoreceptors retina –bipolar and horizontal cells –ganglion cells](https://reader035.vdocuments.us/reader035/viewer/2022062718/56649e845503460f94b85a59/html5/thumbnails/13.jpg)
Extrastriate cortex(beyond the striate cortex)
V1
![Page 14: The Brain from retina to extrastriate cortex. Neural processing responsible for vision photoreceptors retina –bipolar and horizontal cells –ganglion cells](https://reader035.vdocuments.us/reader035/viewer/2022062718/56649e845503460f94b85a59/html5/thumbnails/14.jpg)
Extrastriate cortex
• Each area handles separate aspect of visual analysis– “V1-V2 complex”: Map for edges– V3: Map for form and local movement– V4: Map for colour– V5: Map for global motion
• Each is a visual module– connects to other areas– operates largely independently
![Page 15: The Brain from retina to extrastriate cortex. Neural processing responsible for vision photoreceptors retina –bipolar and horizontal cells –ganglion cells](https://reader035.vdocuments.us/reader035/viewer/2022062718/56649e845503460f94b85a59/html5/thumbnails/15.jpg)
Douglas A. Lyon, Ph.D.Chair, Computer Engineering Dept. Fairfield
University, CT, USA
[email protected], http://www.DocJava.com
Copyright 2002 © DocJava, Inc.
![Page 16: The Brain from retina to extrastriate cortex. Neural processing responsible for vision photoreceptors retina –bipolar and horizontal cells –ganglion cells](https://reader035.vdocuments.us/reader035/viewer/2022062718/56649e845503460f94b85a59/html5/thumbnails/16.jpg)
Background
• It is easy to find a bad edge!• We know a good edge when we see it!
![Page 17: The Brain from retina to extrastriate cortex. Neural processing responsible for vision photoreceptors retina –bipolar and horizontal cells –ganglion cells](https://reader035.vdocuments.us/reader035/viewer/2022062718/56649e845503460f94b85a59/html5/thumbnails/17.jpg)
The Problem
• Given an expert + an image.
• The expert places markers on a good edge.
• Find a way to connect the markers.
![Page 18: The Brain from retina to extrastriate cortex. Neural processing responsible for vision photoreceptors retina –bipolar and horizontal cells –ganglion cells](https://reader035.vdocuments.us/reader035/viewer/2022062718/56649e845503460f94b85a59/html5/thumbnails/18.jpg)
Motivation
• Experts find/define good edges
• Knowledge is hard to encode.
![Page 19: The Brain from retina to extrastriate cortex. Neural processing responsible for vision photoreceptors retina –bipolar and horizontal cells –ganglion cells](https://reader035.vdocuments.us/reader035/viewer/2022062718/56649e845503460f94b85a59/html5/thumbnails/19.jpg)
Approach
• Mark an important edge
• Pixels=graph nodes
• Search in pixel space using a heuristic
• A* is faster than DP
![Page 20: The Brain from retina to extrastriate cortex. Neural processing responsible for vision photoreceptors retina –bipolar and horizontal cells –ganglion cells](https://reader035.vdocuments.us/reader035/viewer/2022062718/56649e845503460f94b85a59/html5/thumbnails/20.jpg)
Assumptions
• User is a domain expert
• Knowledge rep=heuristics+markers
![Page 21: The Brain from retina to extrastriate cortex. Neural processing responsible for vision photoreceptors retina –bipolar and horizontal cells –ganglion cells](https://reader035.vdocuments.us/reader035/viewer/2022062718/56649e845503460f94b85a59/html5/thumbnails/21.jpg)
Applications
• Photo interpretation
• Path planning (source+destination)
• Medical imaging
![Page 22: The Brain from retina to extrastriate cortex. Neural processing responsible for vision photoreceptors retina –bipolar and horizontal cells –ganglion cells](https://reader035.vdocuments.us/reader035/viewer/2022062718/56649e845503460f94b85a59/html5/thumbnails/22.jpg)
Photo Interpretation
![Page 23: The Brain from retina to extrastriate cortex. Neural processing responsible for vision photoreceptors retina –bipolar and horizontal cells –ganglion cells](https://reader035.vdocuments.us/reader035/viewer/2022062718/56649e845503460f94b85a59/html5/thumbnails/23.jpg)
Echocardiogram
•3D-multi-scale analysis
![Page 24: The Brain from retina to extrastriate cortex. Neural processing responsible for vision photoreceptors retina –bipolar and horizontal cells –ganglion cells](https://reader035.vdocuments.us/reader035/viewer/2022062718/56649e845503460f94b85a59/html5/thumbnails/24.jpg)
Path Plans, the idea
![Page 25: The Brain from retina to extrastriate cortex. Neural processing responsible for vision photoreceptors retina –bipolar and horizontal cells –ganglion cells](https://reader035.vdocuments.us/reader035/viewer/2022062718/56649e845503460f94b85a59/html5/thumbnails/25.jpg)
Path Planning-pre proc.•hist+thresh
•Dil+Skel
![Page 26: The Brain from retina to extrastriate cortex. Neural processing responsible for vision photoreceptors retina –bipolar and horizontal cells –ganglion cells](https://reader035.vdocuments.us/reader035/viewer/2022062718/56649e845503460f94b85a59/html5/thumbnails/26.jpg)
Path Planning - Search
![Page 27: The Brain from retina to extrastriate cortex. Neural processing responsible for vision photoreceptors retina –bipolar and horizontal cells –ganglion cells](https://reader035.vdocuments.us/reader035/viewer/2022062718/56649e845503460f94b85a59/html5/thumbnails/27.jpg)
The Idea
• The mind selects from filter banks
• The mind tunes the filters
![Page 28: The Brain from retina to extrastriate cortex. Neural processing responsible for vision photoreceptors retina –bipolar and horizontal cells –ganglion cells](https://reader035.vdocuments.us/reader035/viewer/2022062718/56649e845503460f94b85a59/html5/thumbnails/28.jpg)
Gabor filter w/threshold
• The Strong edge is the wrong edge!
![Page 29: The Brain from retina to extrastriate cortex. Neural processing responsible for vision photoreceptors retina –bipolar and horizontal cells –ganglion cells](https://reader035.vdocuments.us/reader035/viewer/2022062718/56649e845503460f94b85a59/html5/thumbnails/29.jpg)
Canny Edge Detector
![Page 30: The Brain from retina to extrastriate cortex. Neural processing responsible for vision photoreceptors retina –bipolar and horizontal cells –ganglion cells](https://reader035.vdocuments.us/reader035/viewer/2022062718/56649e845503460f94b85a59/html5/thumbnails/30.jpg)
Mehrotra and Zhang
![Page 31: The Brain from retina to extrastriate cortex. Neural processing responsible for vision photoreceptors retina –bipolar and horizontal cells –ganglion cells](https://reader035.vdocuments.us/reader035/viewer/2022062718/56649e845503460f94b85a59/html5/thumbnails/31.jpg)
![Page 32: The Brain from retina to extrastriate cortex. Neural processing responsible for vision photoreceptors retina –bipolar and horizontal cells –ganglion cells](https://reader035.vdocuments.us/reader035/viewer/2022062718/56649e845503460f94b85a59/html5/thumbnails/32.jpg)
![Page 33: The Brain from retina to extrastriate cortex. Neural processing responsible for vision photoreceptors retina –bipolar and horizontal cells –ganglion cells](https://reader035.vdocuments.us/reader035/viewer/2022062718/56649e845503460f94b85a59/html5/thumbnails/33.jpg)
Sub bands for 3x3 Robinson
![Page 34: The Brain from retina to extrastriate cortex. Neural processing responsible for vision photoreceptors retina –bipolar and horizontal cells –ganglion cells](https://reader035.vdocuments.us/reader035/viewer/2022062718/56649e845503460f94b85a59/html5/thumbnails/34.jpg)
Sub Bands 7x7 Gabor
![Page 35: The Brain from retina to extrastriate cortex. Neural processing responsible for vision photoreceptors retina –bipolar and horizontal cells –ganglion cells](https://reader035.vdocuments.us/reader035/viewer/2022062718/56649e845503460f94b85a59/html5/thumbnails/35.jpg)
Gabor-biologically motivated
![Page 36: The Brain from retina to extrastriate cortex. Neural processing responsible for vision photoreceptors retina –bipolar and horizontal cells –ganglion cells](https://reader035.vdocuments.us/reader035/viewer/2022062718/56649e845503460f94b85a59/html5/thumbnails/36.jpg)
Log filters=prefilter+laplacian
2 1
2 2 ex 2 y 2
2 2
1
4 1 x2 y2
2 2
ex 2 y 2
2 2
2 f (x, y) 2 f
x2 2 f
y2
![Page 37: The Brain from retina to extrastriate cortex. Neural processing responsible for vision photoreceptors retina –bipolar and horizontal cells –ganglion cells](https://reader035.vdocuments.us/reader035/viewer/2022062718/56649e845503460f94b85a59/html5/thumbnails/37.jpg)
Mexican Hat (LoG Kernel)
![Page 38: The Brain from retina to extrastriate cortex. Neural processing responsible for vision photoreceptors retina –bipolar and horizontal cells –ganglion cells](https://reader035.vdocuments.us/reader035/viewer/2022062718/56649e845503460f94b85a59/html5/thumbnails/38.jpg)
The Log kernel
![Page 39: The Brain from retina to extrastriate cortex. Neural processing responsible for vision photoreceptors retina –bipolar and horizontal cells –ganglion cells](https://reader035.vdocuments.us/reader035/viewer/2022062718/56649e845503460f94b85a59/html5/thumbnails/39.jpg)
Oriented Filters are steerable
![Page 40: The Brain from retina to extrastriate cortex. Neural processing responsible for vision photoreceptors retina –bipolar and horizontal cells –ganglion cells](https://reader035.vdocuments.us/reader035/viewer/2022062718/56649e845503460f94b85a59/html5/thumbnails/40.jpg)
Changing Scale at 0 Degrees
![Page 41: The Brain from retina to extrastriate cortex. Neural processing responsible for vision photoreceptors retina –bipolar and horizontal cells –ganglion cells](https://reader035.vdocuments.us/reader035/viewer/2022062718/56649e845503460f94b85a59/html5/thumbnails/41.jpg)
Changing Phase at 0 degrees
![Page 42: The Brain from retina to extrastriate cortex. Neural processing responsible for vision photoreceptors retina –bipolar and horizontal cells –ganglion cells](https://reader035.vdocuments.us/reader035/viewer/2022062718/56649e845503460f94b85a59/html5/thumbnails/42.jpg)
Summary
• Heuristics+markers= knowledge• Low-level image processing still needed• Global optimization is not appropriate for
all applications• Sometimes we only want a single, good
edge