![Page 1: Vision - Computer Science and Engineeringweb.cse.ohio-state.edu/~wang.77/teaching/cis730/Vision.pdf · Vision From image to visual description An interdisciplinary field • Computational](https://reader035.vdocuments.us/reader035/viewer/2022070811/5f0acff67e708231d42d76a0/html5/thumbnails/1.jpg)
Wang: Suvery of Artificial Intelligence
1
Vision
From image to visual description
An interdisciplinary field
• Computational vision (computer/robot vision)
• Visual physiology
• Receptive fields• Orientation and other feature detectors• Retinotopic organization
• Visual perception
![Page 2: Vision - Computer Science and Engineeringweb.cse.ohio-state.edu/~wang.77/teaching/cis730/Vision.pdf · Vision From image to visual description An interdisciplinary field • Computational](https://reader035.vdocuments.us/reader035/viewer/2022070811/5f0acff67e708231d42d76a0/html5/thumbnails/2.jpg)
Wang: Suvery of Artificial Intelligence
2
Three Levels of Analysis (Marr)
1. Computational Theory
Goals of computation, Appropriateness of the goal,General strategies
2. Representation/Algorithm
• How to represent the input and the output• Algorithms for transforming from one representation to
another
3. Implementation
How can the representation and algorithm be realizedPhysically (architecture, hardware)?
![Page 3: Vision - Computer Science and Engineeringweb.cse.ohio-state.edu/~wang.77/teaching/cis730/Vision.pdf · Vision From image to visual description An interdisciplinary field • Computational](https://reader035.vdocuments.us/reader035/viewer/2022070811/5f0acff67e708231d42d76a0/html5/thumbnails/3.jpg)
Wang: Suvery of Artificial Intelligence
3
Image Example
A photograph
Closer look
![Page 4: Vision - Computer Science and Engineeringweb.cse.ohio-state.edu/~wang.77/teaching/cis730/Vision.pdf · Vision From image to visual description An interdisciplinary field • Computational](https://reader035.vdocuments.us/reader035/viewer/2022070811/5f0acff67e708231d42d76a0/html5/thumbnails/4.jpg)
Wang: Suvery of Artificial Intelligence
4
Marrian Representational Framework
Input: Images as intensity distribution
• Primal Sketch
Intensity changes,geometrical distribution of line segments
• 2 12 -D Sketch
Orientation and rough depth of visual surfaces, and thecontours of discontinuities between surfaces, in a viewer-centered coordinate frame
• 3-D Model
Shapes and their spatial organization in an object-centeredcoordinate frame
![Page 5: Vision - Computer Science and Engineeringweb.cse.ohio-state.edu/~wang.77/teaching/cis730/Vision.pdf · Vision From image to visual description An interdisciplinary field • Computational](https://reader035.vdocuments.us/reader035/viewer/2022070811/5f0acff67e708231d42d76a0/html5/thumbnails/5.jpg)
Wang: Suvery of Artificial Intelligence
5
Primal Sketch
Lines and contours from gray level images
• Edge Detection by finding discontinuities
• Edge segments are lined up and glued together
• Similar curves are combined to form contours
![Page 6: Vision - Computer Science and Engineeringweb.cse.ohio-state.edu/~wang.77/teaching/cis730/Vision.pdf · Vision From image to visual description An interdisciplinary field • Computational](https://reader035.vdocuments.us/reader035/viewer/2022070811/5f0acff67e708231d42d76a0/html5/thumbnails/6.jpg)
Wang: Suvery of Artificial Intelligence
6
Edge Detection
• Different kinds of edges
![Page 7: Vision - Computer Science and Engineeringweb.cse.ohio-state.edu/~wang.77/teaching/cis730/Vision.pdf · Vision From image to visual description An interdisciplinary field • Computational](https://reader035.vdocuments.us/reader035/viewer/2022070811/5f0acff67e708231d42d76a0/html5/thumbnails/7.jpg)
Wang: Suvery of Artificial Intelligence
7
Edge Detection (cont.)
• One-dimensional (1D) edges
![Page 8: Vision - Computer Science and Engineeringweb.cse.ohio-state.edu/~wang.77/teaching/cis730/Vision.pdf · Vision From image to visual description An interdisciplinary field • Computational](https://reader035.vdocuments.us/reader035/viewer/2022070811/5f0acff67e708231d42d76a0/html5/thumbnails/8.jpg)
Wang: Suvery of Artificial Intelligence
8
Edge Detection (cont.)
• Convolution with linear filters: h = f * g
• 1D
• Continuous
h x f u g x u du( ) ( ) ( )= −−∞
+∞∫
• Discrete
h x f u g x u( ) ( ) ( )= −−∞
+∞
∑
• 2D - straightforward extension
![Page 9: Vision - Computer Science and Engineeringweb.cse.ohio-state.edu/~wang.77/teaching/cis730/Vision.pdf · Vision From image to visual description An interdisciplinary field • Computational](https://reader035.vdocuments.us/reader035/viewer/2022070811/5f0acff67e708231d42d76a0/html5/thumbnails/9.jpg)
Wang: Suvery of Artificial Intelligence
9
Edge Detection (cont.)
• Canny edge detector
• Smoothing by convolving with a Gaussian
G x ex
σσ
πσ( ) =
−1
2
2
22
• Differentiate with the equality: ( f * g)' = f * g'And
′ = −−
G xx
ex
σσ
πσ( )
2 32
2
2
• Overall three-step algorithm
• Convolve an image I with ′Gσ to get R• Find the absolute value ||R||• Mark those peaks in ||R|| that are above a threshold
Tn, in order to reduce noise effects. Results are edgeelements
![Page 10: Vision - Computer Science and Engineeringweb.cse.ohio-state.edu/~wang.77/teaching/cis730/Vision.pdf · Vision From image to visual description An interdisciplinary field • Computational](https://reader035.vdocuments.us/reader035/viewer/2022070811/5f0acff67e708231d42d76a0/html5/thumbnails/10.jpg)
Wang: Suvery of Artificial Intelligence
10
Smoothing Examples
y
x
W
© 1998 Morgan Kaufmann Publishers 2D Gaussian
(a) Original image (b) Width of Gaussian = 2 pixels
(c) Width of Gaussian = 4 pixels (d) Width of Gaussian = 8 pixels
© 1998 Morgan Kaufman Publishers
Smoothing with different Gaussian widths
![Page 11: Vision - Computer Science and Engineeringweb.cse.ohio-state.edu/~wang.77/teaching/cis730/Vision.pdf · Vision From image to visual description An interdisciplinary field • Computational](https://reader035.vdocuments.us/reader035/viewer/2022070811/5f0acff67e708231d42d76a0/html5/thumbnails/11.jpg)
Wang: Suvery of Artificial Intelligence
11
Edge Detection (cont.)
• LOG: Laplacian of Gaussian
• Smoothing using a Gaussian kernel• Edge detection using the Laplacian operator• Edge elements are zero crossings
2D Gaussian and LOG function
G x y ex y
σσ
πσ( , )
( )
=− +
12 2
2
2 2
2
LOG x yx y
G x yσ σ∂∂
∂∂
( , ) ( ) ( , )= +2
2
2
2
![Page 12: Vision - Computer Science and Engineeringweb.cse.ohio-state.edu/~wang.77/teaching/cis730/Vision.pdf · Vision From image to visual description An interdisciplinary field • Computational](https://reader035.vdocuments.us/reader035/viewer/2022070811/5f0acff67e708231d42d76a0/html5/thumbnails/12.jpg)
Wang: Suvery of Artificial Intelligence
12
LOG - 1D
“Positive” partof window
“Negative” partof window
© 1998 Morgan Kaufman Publishers A step edge
1 2 3 4 5 6
0.5
1
1.5
2
1 2 3 4 5 6
0.20.40.60.8
1
1 2 3 4 5 6
–1
–0.5
0.5
1
Image intensity, I(x)
dI/dx
d2I/dx2
© 1998 Morgan Kaufman Publishers LOG operation
![Page 13: Vision - Computer Science and Engineeringweb.cse.ohio-state.edu/~wang.77/teaching/cis730/Vision.pdf · Vision From image to visual description An interdisciplinary field • Computational](https://reader035.vdocuments.us/reader035/viewer/2022070811/5f0acff67e708231d42d76a0/html5/thumbnails/13.jpg)
Wang: Suvery of Artificial Intelligence
13
LOG - A 2D Function
W
y
x
+
–
© 1998 Morgan Kaufman Publishers
Laplacian of a 2D Gaussian
• 2D LOG examples
![Page 14: Vision - Computer Science and Engineeringweb.cse.ohio-state.edu/~wang.77/teaching/cis730/Vision.pdf · Vision From image to visual description An interdisciplinary field • Computational](https://reader035.vdocuments.us/reader035/viewer/2022070811/5f0acff67e708231d42d76a0/html5/thumbnails/14.jpg)
Wang: Suvery of Artificial Intelligence
14
Region Growing for Surface Detection
Input Image (unclassified pixels)=>
Disjoint Regions of Pixels
• Goal
Let X be the set of all pixels in the input image
Partition X into subsets (R1, R2, ..., Rn) such that
• The sets are disjoint• The union of the Ri's is X
• Similarity Measure
H Ri( ) =
TRUE if is homogeneous, or Max( ) – Min( ) <
FALSE if otherwise
R R Ri i i θ
![Page 15: Vision - Computer Science and Engineeringweb.cse.ohio-state.edu/~wang.77/teaching/cis730/Vision.pdf · Vision From image to visual description An interdisciplinary field • Computational](https://reader035.vdocuments.us/reader035/viewer/2022070811/5f0acff67e708231d42d76a0/html5/thumbnails/15.jpg)
Wang: Suvery of Artificial Intelligence
15
Region Growing for Surface Detection
• Some Algorithms
• Merging - Bottom-up method. Starting with allindividual pixels
• Splitting - Top-down method. Starting with the entireimage
• A Sample Example (region growing)
1. Choose all pixels as seeds, Si, i = 1, 2, ..., N. N is thenumber of pixels. Each seed forms an initial region
2. Pick up two unchecked neighboring regions, check thesimilarity criterion
• If yes, merge• If no, mark them as separate regions
3. Goto Step 2 until all regions are marked
![Page 16: Vision - Computer Science and Engineeringweb.cse.ohio-state.edu/~wang.77/teaching/cis730/Vision.pdf · Vision From image to visual description An interdisciplinary field • Computational](https://reader035.vdocuments.us/reader035/viewer/2022070811/5f0acff67e708231d42d76a0/html5/thumbnails/16.jpg)
Wang: Suvery of Artificial Intelligence
16
3-D Model (or Model-based Vision)
Describing Shapes in Object-centered Coordinates
• Organization of 3-D shapes in a hierarchy
• Cylinder-based representation: Components, spatialrelations, and relative sizes
• Derive 3-D model
- Based on previous analysis (primal sketch, 2 12 D sketch)
- Recognition as matching between the representedimage and catalogued (stored) models
![Page 17: Vision - Computer Science and Engineeringweb.cse.ohio-state.edu/~wang.77/teaching/cis730/Vision.pdf · Vision From image to visual description An interdisciplinary field • Computational](https://reader035.vdocuments.us/reader035/viewer/2022070811/5f0acff67e708231d42d76a0/html5/thumbnails/17.jpg)
Wang: Suvery of Artificial Intelligence
17
Vision
Summary
1. Computationally expensive, thus high demand onparallel algorithms
2. Computational vision is difficult
- Low level vision (primal sketch): There exist goodalgorithms.
- Medium level vision (2 12 - D sketch): There are
some reasonable algorithms.
- High level vision (3-D model): Invariant patternrecognition and image understanding (Sceneinterpretation), Few good algorithms.
These levels are intertwined together, making the wholeproblem most challenging.
3. Currently active field