image processing - cs.utexas.edu...image processing an image processing operation typically defines...
TRANSCRIPT
![Page 1: Image processing - cs.utexas.edu...Image processing An image processing operation typically defines a new image g in terms of an existing image f. The simplest operations are those](https://reader033.vdocuments.us/reader033/viewer/2022050804/603885b639dafc120b7e950f/html5/thumbnails/1.jpg)
University of Texas at Austin CS384G - Computer Graphics Fall 2010 Don Fussell
Image processing
![Page 2: Image processing - cs.utexas.edu...Image processing An image processing operation typically defines a new image g in terms of an existing image f. The simplest operations are those](https://reader033.vdocuments.us/reader033/viewer/2022050804/603885b639dafc120b7e950f/html5/thumbnails/2.jpg)
University of Texas at Austin CS384G - Computer Graphics Fall 2010 Don Fussell 2
Reading
Jain, Kasturi, Schunck, Machine Vision.McGraw-Hill, 1995. Sections 4.2-4.4,4.5(intro), 4.5.5, 4.5.6, 5.1-5.4.
![Page 3: Image processing - cs.utexas.edu...Image processing An image processing operation typically defines a new image g in terms of an existing image f. The simplest operations are those](https://reader033.vdocuments.us/reader033/viewer/2022050804/603885b639dafc120b7e950f/html5/thumbnails/3.jpg)
University of Texas at Austin CS384G - Computer Graphics Fall 2010 Don Fussell 3
Image processingAn image processing operation typically defines a newimage g in terms of an existing image f.The simplest operations are those that transform each pixelin isolation. These pixel-to-pixel operations can bewritten:
Examples: threshold, RGB grayscaleNote: a typical choice for mapping to grayscale is to applythe YIQ television matrix and keep the Y.!
g(x,y) = t( f (x,y))
!
Y
I
Q
"
#
$ $ $
%
&
' ' '
=
0.299 0.587 0.114
0.596 (0.275 (0.321
0.212 (0.523 0.311
"
#
$ $ $
%
&
' ' '
R
G
B
"
#
$ $ $
%
&
' ' '
![Page 4: Image processing - cs.utexas.edu...Image processing An image processing operation typically defines a new image g in terms of an existing image f. The simplest operations are those](https://reader033.vdocuments.us/reader033/viewer/2022050804/603885b639dafc120b7e950f/html5/thumbnails/4.jpg)
University of Texas at Austin CS384G - Computer Graphics Fall 2010 Don Fussell 4
Pixel movement
Some operations preserve intensities, but movepixels around in the image
Examples: many amusing warps of images
[Show image sequence.]
% %( , ) ( ( , ), ( , ))g x y f x x y y x y=
![Page 5: Image processing - cs.utexas.edu...Image processing An image processing operation typically defines a new image g in terms of an existing image f. The simplest operations are those](https://reader033.vdocuments.us/reader033/viewer/2022050804/603885b639dafc120b7e950f/html5/thumbnails/5.jpg)
University of Texas at Austin CS384G - Computer Graphics Fall 2010 Don Fussell 5
Noise Image processing is also useful for noise reduction and edge
enhancement. We will focus on these applications for the remainder ofthe lecture…
Common types of noise: Salt and pepper noise:
contains randomoccurrences of black andwhite pixels
Impulse noise: containsrandom occurrences ofwhite pixels
Gaussian noise:variations in intensity drawnfrom a Gaussian normaldistribution
![Page 6: Image processing - cs.utexas.edu...Image processing An image processing operation typically defines a new image g in terms of an existing image f. The simplest operations are those](https://reader033.vdocuments.us/reader033/viewer/2022050804/603885b639dafc120b7e950f/html5/thumbnails/6.jpg)
University of Texas at Austin CS384G - Computer Graphics Fall 2010 Don Fussell 6
Ideal noise reduction
![Page 7: Image processing - cs.utexas.edu...Image processing An image processing operation typically defines a new image g in terms of an existing image f. The simplest operations are those](https://reader033.vdocuments.us/reader033/viewer/2022050804/603885b639dafc120b7e950f/html5/thumbnails/7.jpg)
University of Texas at Austin CS384G - Computer Graphics Fall 2010 Don Fussell 7
Ideal noise reduction
![Page 8: Image processing - cs.utexas.edu...Image processing An image processing operation typically defines a new image g in terms of an existing image f. The simplest operations are those](https://reader033.vdocuments.us/reader033/viewer/2022050804/603885b639dafc120b7e950f/html5/thumbnails/8.jpg)
University of Texas at Austin CS384G - Computer Graphics Fall 2010 Don Fussell 8
Practical noise reduction
How can we “smooth” away noise in a single image?
Is there a more abstract way to represent this sort ofoperation? Of course there is!
![Page 9: Image processing - cs.utexas.edu...Image processing An image processing operation typically defines a new image g in terms of an existing image f. The simplest operations are those](https://reader033.vdocuments.us/reader033/viewer/2022050804/603885b639dafc120b7e950f/html5/thumbnails/9.jpg)
University of Texas at Austin CS384G - Computer Graphics Fall 2010 Don Fussell 9
Discrete convolution
For a digital signal, we define discrete convolution as:
where
!
g[i] = f [i]" h[i]
= f [ # i ]h[i $ # i ]# i
%
= f [ # i ]) h [ # i $ i]
# i
%
!
) h [i] = h["i]
![Page 10: Image processing - cs.utexas.edu...Image processing An image processing operation typically defines a new image g in terms of an existing image f. The simplest operations are those](https://reader033.vdocuments.us/reader033/viewer/2022050804/603885b639dafc120b7e950f/html5/thumbnails/10.jpg)
University of Texas at Austin CS384G - Computer Graphics Fall 2010 Don Fussell 10
Discrete convolution in 2D
Similarly, discrete convolution in 2D becomes:
where
!
g[i, j] = f [i, j]" h[i, j]
= f [ # i , # j ]h[i $ # i , j $ # j ]# j
%# i
%
= f [ # i , # j ]) h [ # i $ i, # j $ j]
# j
%# i
%
!
) h [i, j] = h["i," j]
![Page 11: Image processing - cs.utexas.edu...Image processing An image processing operation typically defines a new image g in terms of an existing image f. The simplest operations are those](https://reader033.vdocuments.us/reader033/viewer/2022050804/603885b639dafc120b7e950f/html5/thumbnails/11.jpg)
University of Texas at Austin CS384G - Computer Graphics Fall 2010 Don Fussell 11
Convolution representation
Since f and h are defined over finite regions, we can writethem out in two-dimensional arrays:
Note: This is not matrix multiplication!Q: What happens at the edges?
![Page 12: Image processing - cs.utexas.edu...Image processing An image processing operation typically defines a new image g in terms of an existing image f. The simplest operations are those](https://reader033.vdocuments.us/reader033/viewer/2022050804/603885b639dafc120b7e950f/html5/thumbnails/12.jpg)
University of Texas at Austin CS384G - Computer Graphics Fall 2010 Don Fussell 12
Mean filters
How can we represent our noise-reducingaveraging filter as a convolution diagram(know as a mean filter)?
![Page 13: Image processing - cs.utexas.edu...Image processing An image processing operation typically defines a new image g in terms of an existing image f. The simplest operations are those](https://reader033.vdocuments.us/reader033/viewer/2022050804/603885b639dafc120b7e950f/html5/thumbnails/13.jpg)
University of Texas at Austin CS384G - Computer Graphics Fall 2010 Don Fussell 13
Effect of mean filters
![Page 14: Image processing - cs.utexas.edu...Image processing An image processing operation typically defines a new image g in terms of an existing image f. The simplest operations are those](https://reader033.vdocuments.us/reader033/viewer/2022050804/603885b639dafc120b7e950f/html5/thumbnails/14.jpg)
University of Texas at Austin CS384G - Computer Graphics Fall 2010 Don Fussell 14
Gaussian filters
Gaussian filters weigh pixels based on their distance fromthe center of the convolution filter. In particular:
This does a decent job of blurring noise while preservingfeatures of the image.What parameter controls the width of the Gaussian?What happens to the image as the Gaussian filter kernelgets wider?What is the constant C? What should we set it to?
2 2 2( ) /(2 )
[ , ]i j
eh i j
C
!" +
=
![Page 15: Image processing - cs.utexas.edu...Image processing An image processing operation typically defines a new image g in terms of an existing image f. The simplest operations are those](https://reader033.vdocuments.us/reader033/viewer/2022050804/603885b639dafc120b7e950f/html5/thumbnails/15.jpg)
University of Texas at Austin CS384G - Computer Graphics Fall 2010 Don Fussell 15
Effect of Gaussian filters
![Page 16: Image processing - cs.utexas.edu...Image processing An image processing operation typically defines a new image g in terms of an existing image f. The simplest operations are those](https://reader033.vdocuments.us/reader033/viewer/2022050804/603885b639dafc120b7e950f/html5/thumbnails/16.jpg)
University of Texas at Austin CS384G - Computer Graphics Fall 2010 Don Fussell 16
Median filters
A median filter operates over an mxmregion by selecting the median intensity inthe region.What advantage does a median filter haveover a mean filter?Is a median filter a kind of convolution?
![Page 17: Image processing - cs.utexas.edu...Image processing An image processing operation typically defines a new image g in terms of an existing image f. The simplest operations are those](https://reader033.vdocuments.us/reader033/viewer/2022050804/603885b639dafc120b7e950f/html5/thumbnails/17.jpg)
University of Texas at Austin CS384G - Computer Graphics Fall 2010 Don Fussell 17
Effect of median filters
![Page 18: Image processing - cs.utexas.edu...Image processing An image processing operation typically defines a new image g in terms of an existing image f. The simplest operations are those](https://reader033.vdocuments.us/reader033/viewer/2022050804/603885b639dafc120b7e950f/html5/thumbnails/18.jpg)
University of Texas at Austin CS384G - Computer Graphics Fall 2010 Don Fussell 18
Comparison: Gaussian noise
![Page 19: Image processing - cs.utexas.edu...Image processing An image processing operation typically defines a new image g in terms of an existing image f. The simplest operations are those](https://reader033.vdocuments.us/reader033/viewer/2022050804/603885b639dafc120b7e950f/html5/thumbnails/19.jpg)
University of Texas at Austin CS384G - Computer Graphics Fall 2010 Don Fussell 19
Comparison: salt and pepper noise
![Page 20: Image processing - cs.utexas.edu...Image processing An image processing operation typically defines a new image g in terms of an existing image f. The simplest operations are those](https://reader033.vdocuments.us/reader033/viewer/2022050804/603885b639dafc120b7e950f/html5/thumbnails/20.jpg)
University of Texas at Austin CS384G - Computer Graphics Fall 2010 Don Fussell 20
Edge detection
One of the most important uses of imageprocessing is edge detection:
Really easy for humansReally difficult for computers
Fundamental in computer visionImportant in many graphics applications
![Page 21: Image processing - cs.utexas.edu...Image processing An image processing operation typically defines a new image g in terms of an existing image f. The simplest operations are those](https://reader033.vdocuments.us/reader033/viewer/2022050804/603885b639dafc120b7e950f/html5/thumbnails/21.jpg)
University of Texas at Austin CS384G - Computer Graphics Fall 2010 Don Fussell 21
What is an edge?
Q: How might you detect an edge in 1D?
![Page 22: Image processing - cs.utexas.edu...Image processing An image processing operation typically defines a new image g in terms of an existing image f. The simplest operations are those](https://reader033.vdocuments.us/reader033/viewer/2022050804/603885b639dafc120b7e950f/html5/thumbnails/22.jpg)
University of Texas at Austin CS384G - Computer Graphics Fall 2010 Don Fussell 22
Gradients
The gradient is the 2D equivalent of the derivative:
Properties of the gradientIt’s a vectorPoints in the direction of maximum increase of fMagnitude is rate of increase
How can we approximate the gradient in a discrete image?
( , ) ,f f
f x yx y
! "# #$ = % &' # # (
![Page 23: Image processing - cs.utexas.edu...Image processing An image processing operation typically defines a new image g in terms of an existing image f. The simplest operations are those](https://reader033.vdocuments.us/reader033/viewer/2022050804/603885b639dafc120b7e950f/html5/thumbnails/23.jpg)
University of Texas at Austin CS384G - Computer Graphics Fall 2010 Don Fussell 23
Less than ideal edges
![Page 24: Image processing - cs.utexas.edu...Image processing An image processing operation typically defines a new image g in terms of an existing image f. The simplest operations are those](https://reader033.vdocuments.us/reader033/viewer/2022050804/603885b639dafc120b7e950f/html5/thumbnails/24.jpg)
University of Texas at Austin CS384G - Computer Graphics Fall 2010 Don Fussell 24
Steps in edge detection
Edge detection algorithms typically proceedin three or four steps:
Filtering: cut down on noiseEnhancement: amplify the difference betweenedges and non-edgesDetection: use a threshold operationLocalization (optional): estimate geometry ofedges beyond pixels
![Page 25: Image processing - cs.utexas.edu...Image processing An image processing operation typically defines a new image g in terms of an existing image f. The simplest operations are those](https://reader033.vdocuments.us/reader033/viewer/2022050804/603885b639dafc120b7e950f/html5/thumbnails/25.jpg)
University of Texas at Austin CS384G - Computer Graphics Fall 2010 Don Fussell 25
Edge enhancementA popular gradient magnitude computation is the Sobeloperator:
We can then compute the magnitude of the vector (sx, sy).
1 0 1
2 0 2
1 0 1
1 2 1
0 0 0
1 2 1
x
y
s
s
!" #$ %
= !$ %$ %!& '
" #$ %
= $ %$ %! ! !& '
![Page 26: Image processing - cs.utexas.edu...Image processing An image processing operation typically defines a new image g in terms of an existing image f. The simplest operations are those](https://reader033.vdocuments.us/reader033/viewer/2022050804/603885b639dafc120b7e950f/html5/thumbnails/26.jpg)
University of Texas at Austin CS384G - Computer Graphics Fall 2010 Don Fussell 26
Results of Sobel edge detection
![Page 27: Image processing - cs.utexas.edu...Image processing An image processing operation typically defines a new image g in terms of an existing image f. The simplest operations are those](https://reader033.vdocuments.us/reader033/viewer/2022050804/603885b639dafc120b7e950f/html5/thumbnails/27.jpg)
University of Texas at Austin CS384G - Computer Graphics Fall 2010 Don Fussell 27
Second derivative operators
The Sobel operator can produce thick edges. Ideally, we’re looking forinfinitely thin boundaries.
An alternative approach is to look for local extrema in the firstderivative: places where the change in the gradient is highest.
Q: A peak in the first derivative corresponds to what in the secondderivative?
Q: How might we write this as a convolution filter?
![Page 28: Image processing - cs.utexas.edu...Image processing An image processing operation typically defines a new image g in terms of an existing image f. The simplest operations are those](https://reader033.vdocuments.us/reader033/viewer/2022050804/603885b639dafc120b7e950f/html5/thumbnails/28.jpg)
University of Texas at Austin CS384G - Computer Graphics Fall 2010 Don Fussell 28
Localization with the Laplacian
An equivalent measure of the second derivative in 2D isthe Laplacian:
Using the same arguments we used to compute thegradient filters, we can derive a Laplacian filter to be:
Zero crossings of this filter correspond to positions ofmaximum gradient. These zero crossings can be used tolocalize edges.
2 22
2 2( , )
f ff x y
x y
! !" = +
! !
2
0 1 0
1 4 1
0 1 0
! "# $% = &# $# $' (
![Page 29: Image processing - cs.utexas.edu...Image processing An image processing operation typically defines a new image g in terms of an existing image f. The simplest operations are those](https://reader033.vdocuments.us/reader033/viewer/2022050804/603885b639dafc120b7e950f/html5/thumbnails/29.jpg)
University of Texas at Austin CS384G - Computer Graphics Fall 2010 Don Fussell 29
Localization with the Laplacian
Original Smoothed
Laplacian (+128)
![Page 30: Image processing - cs.utexas.edu...Image processing An image processing operation typically defines a new image g in terms of an existing image f. The simplest operations are those](https://reader033.vdocuments.us/reader033/viewer/2022050804/603885b639dafc120b7e950f/html5/thumbnails/30.jpg)
University of Texas at Austin CS384G - Computer Graphics Fall 2010 Don Fussell 30
Marching squares
We can convert these signed values intoedge contours using a “marching squares”technique:
![Page 31: Image processing - cs.utexas.edu...Image processing An image processing operation typically defines a new image g in terms of an existing image f. The simplest operations are those](https://reader033.vdocuments.us/reader033/viewer/2022050804/603885b639dafc120b7e950f/html5/thumbnails/31.jpg)
University of Texas at Austin CS384G - Computer Graphics Fall 2010 Don Fussell 31
Sharpening with the Laplacian
Original Laplacian (+128)
Original + Laplacian Original - LaplacianWhy does the sign make a difference?How can you write each filter that makes each bottom image?
![Page 32: Image processing - cs.utexas.edu...Image processing An image processing operation typically defines a new image g in terms of an existing image f. The simplest operations are those](https://reader033.vdocuments.us/reader033/viewer/2022050804/603885b639dafc120b7e950f/html5/thumbnails/32.jpg)
University of Texas at Austin CS384G - Computer Graphics Fall 2010 Don Fussell 32
Spectral impact of sharpeningWe can look at the impact of sharpening on the Fourier spectrum:
2
0 1 0
1 5 1
0 1 0
!
"# $% &" ' = " "% &% &"( )
Spatial domain Frequency domain
![Page 33: Image processing - cs.utexas.edu...Image processing An image processing operation typically defines a new image g in terms of an existing image f. The simplest operations are those](https://reader033.vdocuments.us/reader033/viewer/2022050804/603885b639dafc120b7e950f/html5/thumbnails/33.jpg)
University of Texas at Austin CS384G - Computer Graphics Fall 2010 Don Fussell 33
Summary
What you should take away from this lecture:The meanings of all the boldfaced terms.How noise reduction is doneHow discrete convolution filtering worksThe effect of mean, Gaussian, and median filtersWhat an image gradient is and how it can be computedHow edge detection is doneWhat the Laplacian image is and how it is used in either edgedetection or image sharpening
![Page 34: Image processing - cs.utexas.edu...Image processing An image processing operation typically defines a new image g in terms of an existing image f. The simplest operations are those](https://reader033.vdocuments.us/reader033/viewer/2022050804/603885b639dafc120b7e950f/html5/thumbnails/34.jpg)
University of Texas at Austin CS384G - Computer Graphics Fall 2010 Don Fussell 34
Next time: Affine Transformations
Topic:How do we represent the rotations, translations, etc.
needed to build a complex scene from simpler objects?Read:
• Watt, Section 1.1.
Optional:• Foley, et al, Chapter 5.1-5.5.• David F. Rogers and J. Alan Adams,
Mathematical Elements for Computer Graphics, 2ndEd., McGraw-Hill, New York, 1990, Chapter 2.