cs292 computational vision and language image processing and transform
TRANSCRIPT
![Page 1: CS292 Computational Vision and Language Image processing and transform](https://reader030.vdocuments.us/reader030/viewer/2022032702/56649cf35503460f949c11a7/html5/thumbnails/1.jpg)
CS292 Computational Vision and
LanguageImage processing and transform
![Page 2: CS292 Computational Vision and Language Image processing and transform](https://reader030.vdocuments.us/reader030/viewer/2022032702/56649cf35503460f949c11a7/html5/thumbnails/2.jpg)
Image processing and transform
Objectives:
• To enhance features that are meaningful to applications
• Obtain key representations (silent features) of image content
![Page 3: CS292 Computational Vision and Language Image processing and transform](https://reader030.vdocuments.us/reader030/viewer/2022032702/56649cf35503460f949c11a7/html5/thumbnails/3.jpg)
Classification of Image Transforms
• Point transforms– modify individual pixels– modify pixels’ locations
• Local transforms– output derived from neighbourhood
• Global transforms– whole image contributes to each output value
![Page 4: CS292 Computational Vision and Language Image processing and transform](https://reader030.vdocuments.us/reader030/viewer/2022032702/56649cf35503460f949c11a7/html5/thumbnails/4.jpg)
Point Transforms
• Manipulating individual pixel values– Brightness adjustment– Contrast adjustment
• Histogram manipulation– equalisation
• Image magnification
![Page 5: CS292 Computational Vision and Language Image processing and transform](https://reader030.vdocuments.us/reader030/viewer/2022032702/56649cf35503460f949c11a7/html5/thumbnails/5.jpg)
Monadic, Point-by-point Operators
• Monadic point-to-point operator
![Page 6: CS292 Computational Vision and Language Image processing and transform](https://reader030.vdocuments.us/reader030/viewer/2022032702/56649cf35503460f949c11a7/html5/thumbnails/6.jpg)
Brightness Adjustment
Add a constant to all values
g(x,y) = f(x,y) + k
(k = 50)
![Page 7: CS292 Computational Vision and Language Image processing and transform](https://reader030.vdocuments.us/reader030/viewer/2022032702/56649cf35503460f949c11a7/html5/thumbnails/7.jpg)
Intensity Shift
g(x,y) <=
Where k is a user defined variable
0 a(x,y)+k < 0
f(x,y) +k 0<=f(x,y) +k <=W
W W<f(x,y) +k
![Page 8: CS292 Computational Vision and Language Image processing and transform](https://reader030.vdocuments.us/reader030/viewer/2022032702/56649cf35503460f949c11a7/html5/thumbnails/8.jpg)
Contrast Adjustment
Scale all values by a constant
g(x,y) = a* f(x,y)
(a = 1.5)
![Page 9: CS292 Computational Vision and Language Image processing and transform](https://reader030.vdocuments.us/reader030/viewer/2022032702/56649cf35503460f949c11a7/html5/thumbnails/9.jpg)
General expression for brightness and contrast modification
g(x,y) = a * f(x,y) +bIf we do not want to specify a gain and a bias, but
would rather map a particular range of grey levels, [f1,f2], onto a new range, [g1,g2]. This form of mapping is accomplished using
(g(x,y)-g1)/(f(x,y)-f1)=(g2-g1)/(f2-f1)i.e.
g(x,y) = g1 +((g2-g1)/(f2-f1))[f(x,y)-f1]
![Page 10: CS292 Computational Vision and Language Image processing and transform](https://reader030.vdocuments.us/reader030/viewer/2022032702/56649cf35503460f949c11a7/html5/thumbnails/10.jpg)
linear mapping
![Page 11: CS292 Computational Vision and Language Image processing and transform](https://reader030.vdocuments.us/reader030/viewer/2022032702/56649cf35503460f949c11a7/html5/thumbnails/11.jpg)
Linear and non-linear mapping• If ‘a’ is a constant, then it is a linear mapping;• When ‘a’ is a function, then it is a non-linear
mapping• Non-linear mapping functions have a useful
properties: the gain, ‘a’, applied to input grey level, can vary. Thus the way in which contrast is modified depends on the input grey level.
• If a range of grey level is mapped to a wider ranger of grey level, the contrast is enhanced
• If a range of grey level is mapped to a narrower range of grey level, the contrast is reduced.
![Page 12: CS292 Computational Vision and Language Image processing and transform](https://reader030.vdocuments.us/reader030/viewer/2022032702/56649cf35503460f949c11a7/html5/thumbnails/12.jpg)
Non-linear mapping
•Generally, logarithmic mapping is for to enhance details in the darker regions of the image, at the expense of detail in the brighter regions.
•Exponential mapping has a reverse effect, contrast in the brighter parts of an image is increased at the expense of contrast in the darker parts.
![Page 13: CS292 Computational Vision and Language Image processing and transform](https://reader030.vdocuments.us/reader030/viewer/2022032702/56649cf35503460f949c11a7/html5/thumbnails/13.jpg)
Image Histogram
• Measure frequency of occurrence of each grey/colour value
Grey Value Histogram
0
2000
4000
6000
8000
10000
12000
1 16 31 46 61 76 91 106
121
136
151
166
181
196
211
226
241
256
Grey Value
Fre
qu
ency
![Page 14: CS292 Computational Vision and Language Image processing and transform](https://reader030.vdocuments.us/reader030/viewer/2022032702/56649cf35503460f949c11a7/html5/thumbnails/14.jpg)
Calculation of an image histogram
Create an array histogram with 2b elements
for all grey levels, i, dohistogram[i] = 0
end for
for all pixel coordinates, x and y, do
Increment histogram [f(x,y)] by 1
end for
![Page 15: CS292 Computational Vision and Language Image processing and transform](https://reader030.vdocuments.us/reader030/viewer/2022032702/56649cf35503460f949c11a7/html5/thumbnails/15.jpg)
Histogram Manipulation
• Modify distribution of grey values to achieve some effect
![Page 16: CS292 Computational Vision and Language Image processing and transform](https://reader030.vdocuments.us/reader030/viewer/2022032702/56649cf35503460f949c11a7/html5/thumbnails/16.jpg)
Cumulative Histogram
• Records the cumulative frequency distribution of grey levels in an image.
• A cumulative histogram is a mapping that counts the cumulative number of pixels in all of the bins up to the specified bin. That is, the cumulative histogram Hk of a histogram hk is defined as:
• (k’ can start from 0, if the range is from 0-255)
![Page 17: CS292 Computational Vision and Language Image processing and transform](https://reader030.vdocuments.us/reader030/viewer/2022032702/56649cf35503460f949c11a7/html5/thumbnails/17.jpg)
![Page 18: CS292 Computational Vision and Language Image processing and transform](https://reader030.vdocuments.us/reader030/viewer/2022032702/56649cf35503460f949c11a7/html5/thumbnails/18.jpg)
Histogram Equalisation
• Histogram equalisation is based on the argument that the image’s appearance will be improved if the distribution of pixels over the available grey level is even.
• A non-linear mapping of grey level, specific to that image, that will yield an optimal improvement in contrast.
• Redistributes grey levels in an attempt to flatten the frequency distribution.
• More grey levels are allocated where there are most pixels, fewer grey levels where there are fewer pixels.
• This tends to increase contrast in the most heavily populated regions of the histogram, and often reveals previously hidden detail.
![Page 19: CS292 Computational Vision and Language Image processing and transform](https://reader030.vdocuments.us/reader030/viewer/2022032702/56649cf35503460f949c11a7/html5/thumbnails/19.jpg)
Histogram Equalisation• If we are to increase contrast for the most frequently
occurring grey levels and reduce contrast in the less popular part of the grey level range, then we need a mapping function which has a steep slope (a>1) at grey levels that occur frequently, and a gentle slope (a<1) at unpopular grey levels.
• The cumulative histogram of the image has these properties.
• The mapping function we need is obtained simply by rescaling the cumulative histogram so that its values lie in the range 0-255.
![Page 20: CS292 Computational Vision and Language Image processing and transform](https://reader030.vdocuments.us/reader030/viewer/2022032702/56649cf35503460f949c11a7/html5/thumbnails/20.jpg)
Calculating histogram equalisation
Compute a scaling factor, a=255/number of pixels
Calculate histogram (see previous algorithm)
c[0] = a*histogram[0]
for all remaining grey levels, i, do
c[i] = c[i-1]+a*histogram[i]
end for
for all pixel coordinates, x and y, do
g(x,y) = c[f(x,y)]
end for
![Page 21: CS292 Computational Vision and Language Image processing and transform](https://reader030.vdocuments.us/reader030/viewer/2022032702/56649cf35503460f949c11a7/html5/thumbnails/21.jpg)
Equalisation/Adaptive Equalisation
• Specifically to make histogram uniform
Grey Value Histogram
0
2000
4000
6000
8000
10000
12000
1 16 31 46 61 76 91 106
121
136
151
166
181
196
211
226
241
256
Grey Value
Fre
qu
ency
![Page 22: CS292 Computational Vision and Language Image processing and transform](https://reader030.vdocuments.us/reader030/viewer/2022032702/56649cf35503460f949c11a7/html5/thumbnails/22.jpg)
Threshold
• This is an important function, which converts a grey scale image to a binary format. Unfortunately, it is often difficult, or even impossible to find satisfactory values for the user defined integer threshold value
• Transform grey/colour image to binary
if f(x, y) > T output = W (or 1)
else 0• How to find T?
![Page 23: CS292 Computational Vision and Language Image processing and transform](https://reader030.vdocuments.us/reader030/viewer/2022032702/56649cf35503460f949c11a7/html5/thumbnails/23.jpg)
Threshold
• C(x,y) <=
for (int y=0; y<height; y++) for(int x=0; x<width; x++)
if (input.getxy(x,y) < threshold)output.setxy(x,y, BLACK);
elseoutput.setxy(x,y,WHITE);
W a(x,y)>=threshold
0 otherwise
![Page 24: CS292 Computational Vision and Language Image processing and transform](https://reader030.vdocuments.us/reader030/viewer/2022032702/56649cf35503460f949c11a7/html5/thumbnails/24.jpg)
Threshold Value
• Manual– User defines a threshold
• P-Tile – if we know the proportion of the image is occupied by the object, define the threshold as that grey level which has the correct proportion
• Mode - Threshold at the minimum between the histogram’s peaks
• Other automatic methods
![Page 25: CS292 Computational Vision and Language Image processing and transform](https://reader030.vdocuments.us/reader030/viewer/2022032702/56649cf35503460f949c11a7/html5/thumbnails/25.jpg)
Image Magnification
• Reducing– new value is weighted sum of nearest neighbours
– new value equals nearest neighbour
• Enlarging– new value is weighted sum of nearest neighbours
– add noise to obscure pixelation
(such operations will be practised in the labs)
![Page 26: CS292 Computational Vision and Language Image processing and transform](https://reader030.vdocuments.us/reader030/viewer/2022032702/56649cf35503460f949c11a7/html5/thumbnails/26.jpg)
Local Transforms
• Consider neighbourhood information
• Convolution
• Applications– smoothing– sharpening– Matching
• Very useful, but computationally costly
![Page 27: CS292 Computational Vision and Language Image processing and transform](https://reader030.vdocuments.us/reader030/viewer/2022032702/56649cf35503460f949c11a7/html5/thumbnails/27.jpg)
Local Operators
• The concept behind local operators is that the intensities of several pixels are combined together in order to calculate the intensity of just one pixel. Amongst the simplest of the local operators are those which use a set of nine pixels arranged in a 3 x 3 square region. It computes the value for one pixel on the basis of the intensities within a region containing 3 x 3 pixels. Other local operators employ larger windows.
![Page 28: CS292 Computational Vision and Language Image processing and transform](https://reader030.vdocuments.us/reader030/viewer/2022032702/56649cf35503460f949c11a7/html5/thumbnails/28.jpg)
Local Operators
![Page 29: CS292 Computational Vision and Language Image processing and transform](https://reader030.vdocuments.us/reader030/viewer/2022032702/56649cf35503460f949c11a7/html5/thumbnails/29.jpg)
A sub-image
(x-1, y-1) (x, y-1) (x+1, y-1)
(x-1, y) (x,y) (x+1,y)
(x-1,y+1) (x, y+1) (x+1, y+1)
![Page 30: CS292 Computational Vision and Language Image processing and transform](https://reader030.vdocuments.us/reader030/viewer/2022032702/56649cf35503460f949c11a7/html5/thumbnails/30.jpg)
Convolution Definition
Place template on image
Multiply overlapping values in image and template
Sum products and normalise
(Templates usually small)
x yyx,tycx,rgcr,g'
![Page 31: CS292 Computational Vision and Language Image processing and transform](https://reader030.vdocuments.us/reader030/viewer/2022032702/56649cf35503460f949c11a7/html5/thumbnails/31.jpg)
ExampleImage Template Result
1 1 11 2 11 1 1
… . . . . . ...… 3 5 7 4 4 …… 4 5 8 5 4 …… 4 6 9 6 4 …… 4 6 9 5 3 …… 4 5 8 5 4 …… . . . . . ...
… . . . . . ...… . . . . . ...… . 6 6 6 . …… . 6 7 6 . …… . 6 7 6 . …… . . . . . ...… . . . . . ...
Divide by template sum
![Page 32: CS292 Computational Vision and Language Image processing and transform](https://reader030.vdocuments.us/reader030/viewer/2022032702/56649cf35503460f949c11a7/html5/thumbnails/32.jpg)
Separable Templates
• Convolve with n x n template– n2 multiplications and additions
• Convolve with two n x 1 templates– 2n multiplications and additions
![Page 33: CS292 Computational Vision and Language Image processing and transform](https://reader030.vdocuments.us/reader030/viewer/2022032702/56649cf35503460f949c11a7/html5/thumbnails/33.jpg)
Example
• Laplacian template
• Separated kernels
0 –1 0-1 4 –1 0 –1 0
-1 2 -1 -12-1
![Page 34: CS292 Computational Vision and Language Image processing and transform](https://reader030.vdocuments.us/reader030/viewer/2022032702/56649cf35503460f949c11a7/html5/thumbnails/34.jpg)
Applications
• Usefulness of convolution is the effects generated by changing templates– Smoothing
• Noise reduction
– Sharpening• Edge enhancement
![Page 35: CS292 Computational Vision and Language Image processing and transform](https://reader030.vdocuments.us/reader030/viewer/2022032702/56649cf35503460f949c11a7/html5/thumbnails/35.jpg)
Smoothing
• Aim is to reduce noise• What is “noise”?
– Noise is deviation of a value from its expected value
• How is it reduced– Addition– Adaptively– Weighted
![Page 36: CS292 Computational Vision and Language Image processing and transform](https://reader030.vdocuments.us/reader030/viewer/2022032702/56649cf35503460f949c11a7/html5/thumbnails/36.jpg)
Original Smoothed
Median Smoothing
![Page 37: CS292 Computational Vision and Language Image processing and transform](https://reader030.vdocuments.us/reader030/viewer/2022032702/56649cf35503460f949c11a7/html5/thumbnails/37.jpg)
Sharpening
• What is it?– Enhancing discontinuities– Edge detection
• Why do it?– Perceptually important– Computationally important
![Page 38: CS292 Computational Vision and Language Image processing and transform](https://reader030.vdocuments.us/reader030/viewer/2022032702/56649cf35503460f949c11a7/html5/thumbnails/38.jpg)
Edge Definition
•An edge is a significant local change in image intensity.
•Significant: in relation to the proportional change in image intensity across the edge
•Local: Neighbourhood relationship matters
![Page 39: CS292 Computational Vision and Language Image processing and transform](https://reader030.vdocuments.us/reader030/viewer/2022032702/56649cf35503460f949c11a7/html5/thumbnails/39.jpg)
![Page 40: CS292 Computational Vision and Language Image processing and transform](https://reader030.vdocuments.us/reader030/viewer/2022032702/56649cf35503460f949c11a7/html5/thumbnails/40.jpg)
Algorithms for Edge Detection
• Algorithms for detecting edges - edge detectors– differentiation based
• estimated the derivatives of the image intensity function, the idea being that large image derivatives reflect abrupt intensity changes.
– Model based• determine whether the intensities in a small area
conform to some model for the edges that we have assumed.
![Page 41: CS292 Computational Vision and Language Image processing and transform](https://reader030.vdocuments.us/reader030/viewer/2022032702/56649cf35503460f949c11a7/html5/thumbnails/41.jpg)
First Derivative, Gradient Edge Detection
• If an edge is a discontinuity
• Can detect it by differencing
![Page 42: CS292 Computational Vision and Language Image processing and transform](https://reader030.vdocuments.us/reader030/viewer/2022032702/56649cf35503460f949c11a7/html5/thumbnails/42.jpg)
Roberts Cross Edge Detector
• Simplest edge detector
-1 0
0 1
0 -1
1 0
![Page 43: CS292 Computational Vision and Language Image processing and transform](https://reader030.vdocuments.us/reader030/viewer/2022032702/56649cf35503460f949c11a7/html5/thumbnails/43.jpg)
Prewitt/Sobel Edge Detector-1 -1 -1
0 0 0
1 1 1
-1 0 1
-1 0 1
-1 0 1
![Page 44: CS292 Computational Vision and Language Image processing and transform](https://reader030.vdocuments.us/reader030/viewer/2022032702/56649cf35503460f949c11a7/html5/thumbnails/44.jpg)
Edge Detection
• Combine horizontal and vertical edge estimates
h
vandvhMag tan 122
![Page 45: CS292 Computational Vision and Language Image processing and transform](https://reader030.vdocuments.us/reader030/viewer/2022032702/56649cf35503460f949c11a7/html5/thumbnails/45.jpg)
Example Results
![Page 46: CS292 Computational Vision and Language Image processing and transform](https://reader030.vdocuments.us/reader030/viewer/2022032702/56649cf35503460f949c11a7/html5/thumbnails/46.jpg)
Example Results
![Page 47: CS292 Computational Vision and Language Image processing and transform](https://reader030.vdocuments.us/reader030/viewer/2022032702/56649cf35503460f949c11a7/html5/thumbnails/47.jpg)
Global Transforms
• Computing a new value for a pixel using the whole image as input
• Cosine and Sine transforms• Fourier transform
– Frequency domain processing
• Hough transform• Karhunen-Loeve transform• Wavelet transform
![Page 48: CS292 Computational Vision and Language Image processing and transform](https://reader030.vdocuments.us/reader030/viewer/2022032702/56649cf35503460f949c11a7/html5/thumbnails/48.jpg)
Geometric Transformations
• Definitions– Affine and non-affine transforms
• Applications– Manipulating image shapes
![Page 49: CS292 Computational Vision and Language Image processing and transform](https://reader030.vdocuments.us/reader030/viewer/2022032702/56649cf35503460f949c11a7/html5/thumbnails/49.jpg)
Affine TransformsScale, Shear, Rotate, Translate
Change values of transform matrix elements according to desired effect.
a, e scalingb, d shearing
a, b, d, e rotationc, f translation
=x’y’1[ ] a b c
d e fg h i[ ] x
y1[ ]
Length and areas preserved.
![Page 50: CS292 Computational Vision and Language Image processing and transform](https://reader030.vdocuments.us/reader030/viewer/2022032702/56649cf35503460f949c11a7/html5/thumbnails/50.jpg)
Affine Transform Examples
![Page 51: CS292 Computational Vision and Language Image processing and transform](https://reader030.vdocuments.us/reader030/viewer/2022032702/56649cf35503460f949c11a7/html5/thumbnails/51.jpg)
Warping ExampleAnsell Adams’ Aspens
xyxyxyx 22 2.12.1','
![Page 52: CS292 Computational Vision and Language Image processing and transform](https://reader030.vdocuments.us/reader030/viewer/2022032702/56649cf35503460f949c11a7/html5/thumbnails/52.jpg)
Summary
• Point transforms– scaling, histogram manipulation,thresholding
• Local transforms– edge detection, smoothing
• Global transforms– Fourier, Hough, Principal Component, Wavelet
• Geometrical transforms