![Page 1: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/1.jpg)
Edge and Texture
VBM 686 – Bilgisayarlı Görü
Pinar Duygulu
Hacettepe University
![Page 2: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/2.jpg)
Filters for features
• Previously, thinking of filtering as a way to remove or reduce noise
• Now, consider how filters will allow us to abstract higher-level “features”.
– Map raw pixels to an intermediate representation that will be used for subsequent processing
– Goal: reduce amount of data, discard redundancy, preserve what’s useful
Source: Darrell, Berkeley
![Page 3: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/3.jpg)
Edge detection• Goal: Identify sudden
changes (discontinuities) in an image– Intuitively, most semantic and
shape information from the image can be encoded in the edges
– More compact than pixels
• Ideal: artist’s line drawing (but artist is also using object-level knowledge)
Source: D. LoweSource: Hays, Brown
![Page 4: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/4.jpg)
Edge detection
• Goal: map image from 2d array of pixels to a set of curves or line segments or contours.
• Why?
• Main idea: look for strong gradients, post-process
Figure from J. Shotton et al., PAMI 2007
Source: Darrell, Berkeley
![Page 5: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/5.jpg)
Why do we care about edges?
• Extract information, recognize objects
• Recover geometry and viewpoint
Vanishingpoint
Vanishingline
Vanishingpoint
Vertical vanishingpoint
(at infinity)
Source: Hays, Brown
![Page 6: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/6.jpg)
Origin of Edges
• Edges are caused by a variety of factors
depth discontinuity
surface color discontinuity
illumination discontinuity
surface normal discontinuity
Source: Steve SeitzSource: Hays, Brown
![Page 7: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/7.jpg)
What can cause an edge?
Depth discontinuity: object boundary
Change in surface orientation: shape
Cast shadows
Reflectance change: appearance information, texture
Source: Darrell, Berkeley
![Page 8: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/8.jpg)
Contrast and invariance
Source: Darrell, Berkeley
![Page 9: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/9.jpg)
Closeup of edges
Source: D. HoiemSource: Hays, Brown
![Page 10: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/10.jpg)
Closeup of edges
Source: D. HoiemSource: Hays, Brown
![Page 11: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/11.jpg)
Closeup of edges
Source: D. HoiemSource: Hays, Brown
![Page 12: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/12.jpg)
Closeup of edges
Source: D. HoiemSource: Hays, Brown
![Page 13: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/13.jpg)
Characterizing edges• An edge is a place of rapid change in the
image intensity function
imageintensity function
(along horizontal scanline) first derivative
edges correspond toextrema of derivative
Source: Hays, Brown
![Page 14: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/14.jpg)
What is an edge?
adapted from Larry Davis, University of Maryland
![Page 15: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/15.jpg)
General Strategy
• Determine image gradient
• Mask points where gradient is particularly large with respect to neighbors (ideally curves of such points)
![Page 16: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/16.jpg)
What is Gradient?
adapted from Michael Black, Brown University
![Page 17: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/17.jpg)
What is Gradient?
adapted from Michael Black, Brown University
![Page 18: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/18.jpg)
What is Gradient?
adapted from Michael Black, Brown University
![Page 19: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/19.jpg)
2D edge detection
adapted from Michael Black, Brown University
![Page 20: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/20.jpg)
Differentiation and convolution
For 2D function, f(x,y), the partial derivative is:
For discrete data, we can approximate using finite differences:
To implement above as convolution, what would be the associated filter?
),(),(lim
),(
0
yxfyxf
x
yxf
1
),(),1(),( yxfyxf
x
yxf
Source: Darrell, Berkeley
![Page 21: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/21.jpg)
CS554 Computer Vision ©Pinar Duygulu
Finite Differences
adapted from Michael Black, Brown University
Partial Derivatives
![Page 22: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/22.jpg)
Partial derivatives of an image
Which shows changes with respect to x?
-1 1
1 -1
or
?-1 1
x
yxf
),(
y
yxf
),(
(showing flipped filters)Source: Darrell, Berkeley
![Page 23: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/23.jpg)
Assorted finite difference filters
>> My = fspecial(‘sobel’);
>> outim = imfilter(double(im), My);
>> imagesc(outim);
>> colormap gray;
Source: Darrell, Berkeley
![Page 24: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/24.jpg)
Image gradientThe gradient of an image:
The gradient points in the direction of most rapid change in intensity
The gradient direction (orientation of edge normal) is given by:
The edge strength is given by the gradient magnitude
Slide credit S. SeitzSource: Darrell, Berkeley
![Page 25: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/25.jpg)
Intensity profile
Source: D. HoiemSource: Hays, Brown
![Page 26: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/26.jpg)
With a little Gaussian noise
Gradient
Source: D. HoiemSource: Hays, Brown
![Page 27: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/27.jpg)
Effects of noise• Consider a single row or column of the image
– Plotting intensity as a function of position gives a signal
Where is the edge?Source: S. Seitz
Source: Hays, Brown
![Page 28: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/28.jpg)
Effects of noise
• Difference filters respond strongly to noise
– Image noise results in pixels that look very different from their neighbors
– Generally, the larger the noise the stronger the response
• What can we do about it?
Source: D. ForsythSource: Hays, Brown
![Page 29: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/29.jpg)
Solution: smooth first
• To find edges, look for peaks in )( gfdx
d
f
g
f * g
)( gfdx
d
Source: S. SeitzSource: Hays, Brown
![Page 30: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/30.jpg)
• Differentiation is convolution, and convolution is associative:
• This saves us one operation:
gdx
dfgf
dx
d )(
Derivative theorem of convolution
gdx
df
f
gdx
d
Source: S. SeitzSource: Hays, Brown
![Page 31: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/31.jpg)
Derivative of Gaussian filter
* [1 -1] =
Source: Hays, Brown
![Page 32: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/32.jpg)
Derivative of Gaussian filters
x-direction y-direction
Source: L. LazebnikSource: Darrell, Berkeley
![Page 33: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/33.jpg)
Second derivative
adapted from Martial Hebert, CMU
•Another way
to detect an
extremal first
derivative is
to look for a
zero second
derivative
![Page 34: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/34.jpg)
Step edge
adapted from Michael Black, Brown University
![Page 35: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/35.jpg)
Second derivative
adapted from Michael Black, Brown University
![Page 36: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/36.jpg)
Sobel Operator
![Page 37: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/37.jpg)
Laplacian of Gaussian (LOG)
• Bad idea to apply a Laplacian without smoothing
– smooth with Gaussian, apply Laplacian
– this is the same as filtering with a Laplacian of Gaussian filter
![Page 38: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/38.jpg)
Laplacian of Gaussian (LOG)
adapted from Martial Hebert, CMU
![Page 39: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/39.jpg)
Laplacian of Gaussian
Consider
Laplacian of Gaussianoperator
Where is the edge? Zero-crossings of bottom graphSource: Darrell, Berkeley
![Page 40: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/40.jpg)
2D edge detection filters
• is the Laplacian operator:
Laplacian of Gaussian
Gaussian derivative of Gaussian
Source: Darrell, Berkeley
![Page 41: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/41.jpg)
Smoothing with a Gaussian
Recall: parameter σ is the “scale” / “width” / “spread” of the Gaussian kernel, and controls the amount of smoothing.
…
Source: Darrell, Berkeley
![Page 42: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/42.jpg)
Effect of σ on derivatives
The apparent structures differ depending on Gaussian’s scale parameter.
Larger values: larger scale edges detectedSmaller values: finer features detected
σ = 1 pixel σ = 3 pixels
Source: Darrell, Berkeley
![Page 43: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/43.jpg)
So, what scale to choose?It depends what we’re looking for.
Too fine of a scale…can’t see the forest for the trees.Too coarse of a scale…can’t tell the maple grain from the cherry.Source: Darrell, Berkeley
![Page 44: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/44.jpg)
Thresholding
• Choose a threshold value t
• Set any pixels less than t to zero (off)
• Set any pixels greater than or equal to t to one (on)
Source: Darrell, Berkeley
![Page 45: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/45.jpg)
Original image
Source: Darrell, Berkeley
![Page 46: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/46.jpg)
Gradient magnitude image
Source: Darrell, Berkeley
![Page 47: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/47.jpg)
Thresholding gradient with a lower threshold
Source: Darrell, Berkeley
![Page 48: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/48.jpg)
Thresholding gradient with a higher threshold
Source: Darrell, Berkeley
![Page 49: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/49.jpg)
• Smoothed derivative removes noise, but blurs edge. Also finds edges at different “scales”.
1 pixel 3 pixels 7 pixels
Tradeoff between smoothing and localization
Source: D. ForsythSource: Hays, Brown
![Page 50: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/50.jpg)
Designing an edge detector• Criteria for a good edge detector:
– Good detection: the optimal detector should find all real edges, ignoring noise or other artifacts
– Good localization• the edges detected must be as close as possible to
the true edges• the detector must return one point only for each
true edge point
• Cues of edge detection– Differences in color, intensity, or texture across the
boundary– Continuity and closure– High-level knowledge
Source: L. Fei-FeiSource: Hays, Brown
![Page 51: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/51.jpg)
Canny edge detector• This is probably the most widely used edge
detector in computer vision
• Theoretical model: step-edges corrupted by additive Gaussian noise
• Canny has shown that the first derivative of the Gaussian closely approximates the operator that optimizes the product of signal-to-noise ratio and localization
J. Canny, A Computational Approach To Edge Detection, IEEE Trans. Pattern Analysis and Machine Intelligence, 8:679-714, 1986.
Source: L. Fei-FeiSource: Hays, Brown
![Page 52: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/52.jpg)
Example
original image (Lena)
Source: Hays, Brown
![Page 53: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/53.jpg)
Derivative of Gaussian filter
x-direction y-direction
Source: Hays, Brown
![Page 54: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/54.jpg)
The Canny edge detector
original image (Lena)
Source: Darrell, Berkeley
![Page 55: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/55.jpg)
Compute Gradients (DoG)
X-Derivative of Gaussian Y-Derivative of Gaussian Gradient Magnitude
Source: Hays, Brown
![Page 56: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/56.jpg)
The Canny edge detector
norm of the gradient
Source: Darrell, Berkeley
![Page 57: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/57.jpg)
The Canny edge detector
thresholding
Source: Darrell, Berkeley
![Page 58: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/58.jpg)
The Canny edge detector
thresholding
How to turn these thick regions of the gradient into curves?
Source: Darrell, Berkeley
![Page 59: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/59.jpg)
Gradient based edge detection
adapted from David Forsyth, UC Berkeley
We wish to mark points along the curve where the magnitude is biggest.
We can do this by looking for a maximum along a slice normal to the curve
(non-maximum suppression). These points should form a curve. There are
then two algorithmic issues: at which point is the maximum, and where is the
next one?
![Page 60: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/60.jpg)
Non-maximum suppression
Check if pixel is local maximum along gradient direction, select single max across width of the edge– requires checking interpolated pixels p and r
Source: Darrell, Berkeley
![Page 61: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/61.jpg)
Non-Maximum Suppression Algorithm
![Page 62: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/62.jpg)
Non-Maximum Suppression Algorithm
adapted from Martial Hebert, CMU
![Page 63: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/63.jpg)
Get Orientation at Each Pixel
• Threshold at minimum level
• Get orientation theta = atan2(gy, gx)
Source: Hays, Brown
![Page 64: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/64.jpg)
Non-maximum suppression for each orientation
At q, we have a maximum if the value is larger than those at both p and at r. Interpolate to get these values.
Source: D. ForsythSource: Hays, Brown
![Page 65: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/65.jpg)
Before Non-max Suppression
Source: Hays, Brown
![Page 66: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/66.jpg)
After non-max suppression
Source: Hays, Brown
![Page 67: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/67.jpg)
The Canny edge detector
thinning
(non-maximum suppression)
Problem: pixels along this edge didn’t survive the thresholding
Source: Darrell, Berkeley
![Page 68: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/68.jpg)
Hysteresis thresholding
• Check that maximum value of gradient value is sufficiently large
– drop-outs? use hysteresis
• use a high threshold to start edge curves and a low threshold to continue them.
Source: S. SeitzSource: Darrell, Berkeley
![Page 69: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/69.jpg)
Hysteresis Thresholding
adapted from Martial Hebert, CMU
.
•Idea : use a high threshold to start edge curves and a low threshold to
continue them
•Start with a high threshold (strong edge)
•Apply another threshold which is lower than the higher one
•Look for the other edges which are higher than the lower
threshold and connected to the strong edges
•Add them to the edge list
![Page 70: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/70.jpg)
Hysteresis Thresholding
adapted from Martial Hebert, CMU
![Page 71: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/71.jpg)
Hysteresis thresholding• Threshold at low/high levels to get weak/strong edge pixels
• Do connected components, starting from strong edge pixels
Source: Hays, Brown
![Page 72: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/72.jpg)
Hysteresis thresholding
original image
high threshold(strong edges)
low threshold(weak edges)
hysteresis threshold
Source: L. Fei-FeiSource: Darrell, Berkeley
![Page 73: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/73.jpg)
Final Canny Edges
Source: Hays, Brown
![Page 74: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/74.jpg)
Canny edge detector
1. Filter image with x, y derivatives of Gaussian
2. Find magnitude and orientation of gradient
3. Non-maximum suppression:– Thin multi-pixel wide “ridges” down to single pixel width
4. Thresholding and linking (hysteresis):– Define two thresholds: low and high
– Use the high threshold to start edge curves and the low threshold to continue them
• MATLAB: edge(image, ‘canny’)
Source: D. Lowe, L. Fei-FeiSource: Hays, Brown
![Page 75: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/75.jpg)
Example
adapted from Martial Hebert, CMU
![Page 76: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/76.jpg)
Thresholding
adapted from Martial Hebert, CMU
Different thresholds
Applied to gradient value
![Page 77: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/77.jpg)
Non-local Maxima Suppression
adapted from Martial Hebert, CMU
![Page 78: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/78.jpg)
Hysteresis Thresholding
adapted from Martial Hebert, CMU
![Page 79: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/79.jpg)
LOG Operator
adapted from Martial Hebert, CMU
![Page 80: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/80.jpg)
LOG Operator
adapted from Martial Hebert, CMU
![Page 81: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/81.jpg)
Effect of sigma – Detection vs. Localization
adapted from Martial Hebert, CMU
![Page 82: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/82.jpg)
Effect of sigma
adapted from Martial Hebert, CMU
![Page 83: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/83.jpg)
Canny’s Edge detection
adapted from Martial Hebert, CMU
Canny takes two factors into account in designing the edge detector
A model of the kind of edges to be detected
A quantitative definition of the performance this edge detector is supposed to have
Several criteria can be chosen to characterize the performance of an edge detector
Good detection, i.e. robustness to noise
Good localization
Uniqueness to response
![Page 84: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/84.jpg)
Canny’s Edge detection
adapted from Martial Hebert, CMU
![Page 85: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/85.jpg)
Effect of (Gaussian kernel spread/size)
Canny with Canny with original
The choice of depends on desired behavior• large detects large scale edges
• small detects fine features
Source: S. SeitzSource: Hays, Brown
![Page 86: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/86.jpg)
Object boundaries vs. edges
Background Texture Shadows
Source: Darrell, Berkeley
![Page 87: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/87.jpg)
Edge detection is just the beginning…
Berkeley segmentation database:http://www.eecs.berkeley.edu/Research/Projects/CS/vision/grouping/segbench/
image human segmentation gradient magnitude
Source: L. Lazebnik
Much more on segmentation later in term…
Source: Darrell, Berkeley
![Page 88: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/88.jpg)
Texture
Easy to recognize, hard to define
![Page 89: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/89.jpg)
Texture
Whether an effect is a texture or not depends on the scale at which it is viewed
A leaf that occupies most of the image –object
Foliage of a tree - texture
![Page 90: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/90.jpg)
Image Segmentation
Source: ForsythSource: Hays, Brown
![Page 91: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/91.jpg)
Texture Synthesis
Efros & Freeman
![Page 92: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/92.jpg)
Shape from Texture
Texture looks same at different points on a surface
deformation is a cue to the shape of the surface
Adapted from David Forsyth, UC Berkeley
![Page 93: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/93.jpg)
Texture and Material
http://www-cvr.ai.uiuc.edu/ponce_grp/data/texture_database/samples/Source: Hays, Brown
![Page 94: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/94.jpg)
Texture and Orientation
http://www-cvr.ai.uiuc.edu/ponce_grp/data/texture_database/samples/Source: Hays, Brown
![Page 95: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/95.jpg)
Texture and Scale
http://www-cvr.ai.uiuc.edu/ponce_grp/data/texture_database/samples/Source: Hays, Brown
![Page 96: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/96.jpg)
What is texture?
Regular or stochastic patterns caused by bumps, grooves, and/or markings
Source: Hays, Brown
![Page 97: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/97.jpg)
Pre-attentive Texture Discrimination
Adapted from Bill Freeman, MIT
Same or different textures?
![Page 98: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/98.jpg)
Textons
Image textures generally consists of organized patterns of
regular sub-elements
One natural way to represent texture is to find the textons and
then describe the way in which they are laid out
It generally required a human to look at the texture in
order to decide what those fundamental units were...
Adapted from Bill Freeman, MIT
![Page 99: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/99.jpg)
Malik & Perona, CVPR 1989
A set of texture examples used in experiments with human subjects to tell how easily
various types of textures can be discriminated
![Page 100: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/100.jpg)
Bergen and Adelson, Nature
1988
Adapted from Bill Freeman, MIT
![Page 101: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/101.jpg)
Representing Texture
• Instead of looking for patterns at the level of arrow-heads or triangles, look for simpler pattern elements (e.g. dots and bars)
• Easy : Find patterns by filtering the image
• Remember : There is a strong response to the filter when the image pattern in a neighborhood looks similar to the filter kernel
• Represent a texture in terms of the response of a collection of filters in different scales and orientations
![Page 102: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/102.jpg)
How can we represent texture?
• Compute responses of blobs and edges at various orientations and scales
Source: Hays, Brown
![Page 103: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/103.jpg)
Overcomplete representation: filter banks
LM Filter Bank
Code for filter banks: www.robots.ox.ac.uk/~vgg/research/texclass/filters.html
Source: Hays, Brown
![Page 104: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/104.jpg)
Filter banks
• Process image with each filter and keep responses (or squared/abs responses)
Source: Hays, Brown
![Page 105: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/105.jpg)
How can we represent texture?
• Measure responses of blobs and edges at various orientations and scales
• Idea 1: Record simple statistics (e.g., mean, std.) of absolute filter responses
Source: Hays, Brown
![Page 106: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/106.jpg)
Can you match the texture to the response?
Mean abs responses
FiltersA
B
C
1
2
3
Source: Hays, Brown
![Page 107: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/107.jpg)
Texture Segmentation
Squared – to count black-to-white pixels same as white-to-black pixels
Smooth – equivalent to estimating the mean of the squared filter outputs in some window
Pass to classifier – to describe the texture
![Page 108: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/108.jpg)
Choice of statistics
Adapted from David Forsyth, UC Berkeley
• What statistics should be collected
• mean of the squared filter output
– Differentiates spoty windows from stripey windows
– (Malik and Perona)
• Use mean and standard deviation
– Texture matching
– (Ma and Manjunath)
![Page 109: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/109.jpg)
Texture Similarity
Ma & Manjunath, CVPR 1996
![Page 110: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/110.jpg)
Application – Satellite Images
Ma & Manjunath, CVPR 1996
![Page 111: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/111.jpg)
Representing texture• Idea 2: take vectors of filter responses at each pixel and
cluster them, then take histograms.
Source: Hays, Brown
![Page 112: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/112.jpg)
CS554 Computer Vision ©Pinar Duygulu
The Goal of Texture Analysis
Adapted from Bill Freeman, MIT
![Page 113: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/113.jpg)
CS554 Computer Vision ©Pinar Duygulu
The Goal of Texture Synthesis
Adapted from Bill Freeman, MIT
![Page 114: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/114.jpg)
Homage to Shannon
![Page 115: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/115.jpg)
Hole Filling
![Page 116: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/116.jpg)
Political Texture Synthesis!
![Page 117: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/117.jpg)
+ =
Application: Texture Transfer
• Try to explain one object with bits and pieces of another object:
Same as texture synthesis, except an additional constraint:1. Consistency of texture
2. Similarity to the image being “explained”
![Page 118: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/118.jpg)
=+
![Page 119: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/119.jpg)
CS554 Computer Vision ©Pinar Duygulu
Texture Transfer
Adapted from Bill Freeman, MIT
![Page 120: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/120.jpg)
CS554 Computer Vision ©Pinar Duygulu
Texture Transfer
Adapted from Bill Freeman, MIT
![Page 121: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/121.jpg)
![Page 122: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/122.jpg)
Image Analogies
• Solution:
![Page 123: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/123.jpg)
![Page 124: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/124.jpg)
![Page 125: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/125.jpg)
Texture by Numbers
![Page 126: Edge and Texture - web.cs.hacettepe.edu.trpinar/courses/VBM686/lectures/edg… · Source: Darrell, BerkeleyToo coarse of a scale…cant tell the maple grain from the cherry. Thresholding](https://reader033.vdocuments.us/reader033/viewer/2022042917/5f5a151bba7ffa2af66dda35/html5/thumbnails/126.jpg)
CS554 Computer Vision ©Pinar Duygulu
Texture Synthesis
Adapted from David Forsyth, UC Berkeley