digital image processing lecture 8 thresholding with hysteres
Post on 24-Dec-2021
4 Views
Preview:
TRANSCRIPT
1
Maria Magnusson, Computer Vision Lab., Dept. of Electrical Engineering, Linkรถping University
Digital Image Processing Lecture 8 Thresholding with hysteres Matched filters and pattern recognition
Different types of correlation Pattern recognition with correlation Point and blob detection with Laplacian or matched filter
The Structure tensor and the Harris operator Gonzales & Woods:
Lacks much material, except from: Correlation in Chapter 12, pp. 915-917, Laplace in Chapter 3, pp. 179 Point detection in Chapter 10, pp. 701-708 Harris operator in Chapter 11, pp. 869-875
p. 1
Thresholding with hysteres Useful when... Threshold T1 keeps only parts of the object Threshold T2 keeps parts of the background
a: Gray scaleimage
b: Thresholdedat T1
c: Thresholdedat T2
d: Result
Image b is dilated with the condition that expansion is not allowed outside the objects in image c.
In G&W: Only as a part of Cannyโs edge (Chapter 10)
Lab 6
p. 2
Flowchart for thresholding with hysteres
c:=(a>T2)
b:=d mult c
b = e? The final resultis image byesno
b:=(a>T1)
Load image a
d:=dilate(b)
e:=b
Not in G&W!
Lab 6
p. 3
Correlation is the same as convolution with a 180o rotated filter. (The convolution procedure without the 180o rotation of the filter.)
Correlation:
Convolution:
๐ โ ๐ ๐ฅ, ๐ฆ ๐ ๐ผ, ๐ฝ โ ๐ ๐ฅ ๐ผ, ๐ฆ ๐ฝ โ๐๐ผ๐๐ฝ
๐ ๐ ๐ฅ, ๐ฆ ๐ ๐ผ, ๐ฝ โ ๐ ๐ฅ ๐ผ, ๐ฆ ๐ฝ โ๐๐ผ๐๐ฝ
2D Convolution and correlation
Also called cross correlation, which definition contains the expectation value E( ), when it is defined for stochastic signals.
p. 4
2
The convolutionresult for comparison
๐ โ ๐ ๐ฅ, ๐ฆ ๐ ๐ผ, ๐ฝ โ ๐ ๐ฅ ๐ผ, ๐ฆ ๐ฝ
๐ ๐ ๐ฅ, ๐ฆ ๐ ๐ผ, ๐ฝ โ ๐ ๐ฅ ๐ผ, ๐ฆ ๐ฝ
๐ ๐ฅ, ๐ฆ ๐ ๐ฅ, ๐ฆNote the symmetryand that there is amax in the origin
00 -2
2
1
000
0 00 -2
2
1
000
000
0 0
0
2 -4
-2-200 0
-4 2
00
0 0
0
4
400 0
4 -4
0
0
1
9 -8
2D discreteconvolution and correlation
(12-9)
p. 5
The proofs are a nice exercise!
Correlation does not commutate:
Convolution commutates:๐ โ ๐ ๐ฅ, ๐ฆ ๐ โ ๐ ๐ฅ, ๐ฆ
๐ ๐ ๐ฅ, ๐ฆ ๐ ๐ ๐ฅ, ๐ฆ
Convolution in the Fourier domain:โฑ ๐ โ ๐ ๐ฅ, ๐ฆ ๐น ๐ข, ๐ฃ โ ๐บ ๐ข, ๐ฃ
Correlation in the Fourier domain, f and g real:โฑ ๐โโ๐ ๐ฅ, ๐ฆ ๐นโ ๐ข, ๐ฃ โ ๐บ ๐ข, ๐ฃ
Theorems for convolutionand correlation
Table 4.4
Why is the Sobel filters in Mariaโs slides rotated 180ยบ
compared to G&W and previously Wikipedia?
Maria convolve with the Sobel filters, whereas
G&W correlate with the Sobel filters. From G&W,
2013: โThis imprecise terminology [in the
literature] is a frequent source of confusionโ.
Maria strongly believes in using convolution since then the Sobel filtering process corresponds to what is performed in a
linear system.
p. 6
Correlation in the Fourier domain:โฑ ๐โโ๐ ๐ฅ, ๐ฆ ๐น ๐ข, ๐ฃ โ ๐บ ๐ข, ๐ฃ
Computation in the spatial or the Fourier domain Convolution/correlation with a filter consisting of n x n
pixels performed in the spatial domain would require n2 x 5122 = n2 106 /4 multiplications and (n2-1) 106 /4 additions. This suggests that correlation with filters smaller than about 7 x 7 pixels is preferably executed in the spatial domain and not in Fourier domain.
It can be calculated that convolution/correlation on a 512 x 512 image costs 10 million multiplications and 15 million additions, provided that the Fourier transform is implemented with FFT.
We will check this on the next 3 slides!We presuppose that a 2D FFT of an NxN real image
takes N2logN2 multiplications and 1.5N2logN2 additions.
p. 7 Computation inthe spatial domain
p. 8
๐ ยท ๐ ๐ ยท 512 ๐ ยท
ยท 10 ๐ 7
12 milj. MULT
๐ 1 ยท ๐ ๐ 1 ยท 512๐ 7 12 milj. ADD
3
๐ถ ๐ข, ๐ฃ ๐ดโ ๐ข, ๐ฃ โ ๐ต ๐ข, ๐ฃ
Computation in the Fourier domain
0 0 00 00 0 0
A(u,v) A*(u,v)2DFFT conj.
can be pre-computed
zero-paddedpattern a(x,y) C(u,v) c(x,y)
2DIFFT
Image:b(x,y) B(u,v)
correlationresult2D
FFT
p. 9 Computation in the Fourier domain
p. 10
2D FFT: ๐ log ๐ MULT, 1.5๐ log ๐ ADD
2๐ log ๐ 2 ยท 512 ยท 2 ยท 9 10 milj. MULT
Mult: ๐ 2โ komplex MULT
2D IFFT: ๐ log ๐ MULT, 1.5๐ log ๐ ADD
3๐ log ๐ 3 ยท 512 ยท 2 ยท 9 15 milj. ADD
In total:
Pattern recognitionwith correlation
p. 11
๐ ๐ฅ, ๐ฆ ๐ ๐ผ, ๐ฝ โ ๐ ๐ผ ๐ฅ, ๐ฝ ๐ฆ
๐ ๐ฅ, ๐ฆ โฑ ๐ถ ๐ข, ๐ฃ โฑ ๐ดโ ๐ข, ๐ฃ โ ๐ต ๐ข, ๐ฃ
Disadvantage: A positive pattern and high positive values in the image can provide high correlation results even if the pattern and image are not similar.
๐
๐
๐
Pattern recognition with common correlation
(12-9)
Lab 6
p. 12
4
๐
๐
๐
๐ ๐ฅ, ๐ฆโ โ ๐ ๐ผ, ๐ฝ โ ๐ ๐ผ ๐ฅ, ๐ฝ ๐ฆ
โ โ ๐ ๐ผ, ๐ฝ โ โ ๐ ๐ผ ๐ฅ, ๐ฝ ๐ฆ, โ
Pattern recognition with normalized correlation Lab 6
p. 13 Pattern recognition with normalized correlation
Has maximum value (= 1) when the imageand pattern match perfectly, i.e. when a = b.
Compare with the Cachy-Schwarz's inequality.
๐โ โ ๐ ๐ผ, ๐ฝ โ ๐ ๐ผ ๐ฅ, ๐ฝ ๐ฆ
โ โ ๐ ๐ผ, ๐ฝ โ โ ๐ ๐ผ ๐ฅ, ๐ฝ ๐ฆ, โ
Lab 6
p. 14
๐
๐
๐
๐ ๐ฅ, ๐ฆ ๐ ๐ผ, ๐ฝ ๐ โ ๐ ๐ผ ๐ฅ, ๐ฝ ๐ฆ ๐
๐ ๐ผ, ๐ฝ ๐ โ ๐ ๐ผ ๐ฅ, ๐ฝ ๐ฆ ๐ ๐ผ, ๐ฝ โ ๐ ๐ผ ๐ฅ, ๐ฝ ๐ฆ ๐
โ crosscovariance
Lab 6
Pattern recognition with corre-lation without local DC-level
p. 15
b is the mean in the local neighborhood.
The proofs
are a nice exercise!
๐ ๐ฅ, ๐ฆโ โ ๐ ๐ผ, ๐ฝ ๐ โ ๐ ๐ผ ๐ฅ, ๐ฝ ๐ฆ ๐
โ โ ๐ ๐ผ, ๐ฝ ๐ โ โ โ ๐ ๐ผ ๐ฅ, ๐ฝ ๐ฆ ๐, โ
Pattern recognition with normalized correlation without local DC-level
See example on next slide from G&W 2:nd edition, Fig.12.9.G&W 4:th edition Fig.12.13 contains another example.
Lab 6
(12-10)
p. 16
b is the mean in the local neighborhood.
a is the mean of the pattern.
It is also called the correlation coefficient.
5
In-image Pattern Correlation result
Maximum value here
Pattern recognition with normalized correlation without local DC-level
p. 17
The choice of correlation method depends on what you want
a) 1D pattern b) found by
correlation without local DC-level
c) found by normalized correlation
d) found by normalized correlation without local DC-level
Common correlation, calculated via the Fourier domain:
Phase filter variant 1 divides with the amplitudes in the Fourier domain (โremove amplitude, keep phaseโ):
Suppose that the image b is translated in relation to the pattern a:
The correlation peak then becomes a dirac impulse at the desired position:
๐ ๐ฅ, ๐ฆ โฑ ๐ถ ๐ข, ๐ฃ โฑ ๐ดโ ๐ข, ๐ฃ โ ๐ต ๐ข, ๐ฃ
Pattern recognition with phase filters
p. 19
๐ ๐ฅ, ๐ฆ โฑโ , โ ,โ , โ ,
โฑโ , โ ,
, ,
๐ ๐ฅ, ๐ฆ ๐ ๐ฅ ๐ฅ , ๐ฆ ๐ฆโ
๐ต ๐ข, ๐ฃ ๐ ๐ด ๐ข, ๐ฃ
๐ ๐ฅ, ๐ฆ ๐ฟ ๐ฅ ๐ฅ , ๐ฆ ๐ฆ
Pattern recognition with phase filters Phase filter variant 2 divides with the pattern energy and
multiplies with a Gaussian G(u,v) in in the Fourier domain:
This way, the correlation peak becomes a Gaussian. This phase filter is less sensitive to noise and small
pattern variations.
p. 20
๐ ๐ฅ, ๐ฆ โฑโ , โ , โ ,
โ , โ ,โฑ
โ , โ , โ ,,
โข Our guest lecturer Bertil Grelsson uses a similar filter in his work.โข Similar to the MOSSE correlation filter that can be used for
visual object tracking in images.โข Martin Danelljan (former employee at CVL) uses a similar filter in his PhD thesis about visual tracking. His thesis has been assessed as the best Nordic PhD Thesis 2017/2018.
6
Point detection with a negative Laplacian filter
Fig. 10.4
Can be viewed as a high pass
filter or a correlation
pattern without DC-level.
X-ra
y im
age
of
turb
ine
blad
e
p. 21
Matches a single point
0 000 0
00 01 1 1
11 1
11 11 -1 -1
-1-1 -1
-1-1 -18
Calculate local DC-level
Do not match a smooth surface
/9/9
A negative Laplacian filter also works as line and edge detection
Fig. 10.5
Imag
e
Afte
r filt
erin
g
Abso
lute
val
ues
Posi
tive
valu
es
Note that edges gives one positive line close to one negative
line(Lab1)
p. 22
Alternative NegativeLaplacian filters
Fig. 3.45
Match points and lines with dif-ferent thickness. Also perform various amount of smoothing.
Also respond to edges.
Fig. 10.21
Lab5!
โ sobelx โ sobelx sobely โ sobely
2 4 4 4 24 0 8 0 44 8 24 8 44 0 8 0 42 4 4 4 2
/64
โ ๐ โ
(10.2-21)(10.2-22)
Laplacian of Gaussian
(LoG)
p. 23
The sum of the filter coefficients = 0 Constant areas give zero response.
Symmetric Does not care about direction.
Positive values in the center and negative values around.
What characterize a Negative Laplacian filter?
p. 24
7
๐ ๐ฅ, ๐ฆ ๐โฒ ๐ผ, ๐ฝ โข ๐โฒ ๐ผ ๐ฅ, ๐ฝ ๐ฆ
โโโ ๐ ๐ผ, ๐ฝ โ ๐ ๐ผ ๐ฅ, ๐ฝ ๐ฆ
โโโ ๐ ๐ผ, ๐ฝ โ ๐ ๐ผ ๐ฅ, ๐ฝ ๐ฆ
dรคrโ๐โฒ๐๐
๐๐๐๐ฅ๐๐๐๐ฆ
och ๐โฒ๐๐
๐๐๐๐ฅ๐๐๐๐ฆ
Pattern recognition with correlation of the gradient
Not in G&W!
p. 25
Found insome smart
cameras.Master
thesis at SICK IVP
2007.
Pattern recognition with correlation of the gradient
In the pattern, the vectors may be normalized.
In the image, the vectors can possibly be un-normalized.
p. 26
Not in G&W!
Very computationally
intensive!
Translation-, scaling- and rotation-invariant pattern recognition
p. 27
Methods to speed up the pattern recognition on the previous slide To find the rotation (and possibly
scaling) with a scale pyramid. Make a scale pyramid of the image. Start at a rough scale and find
translation, scale and rotation. Refine the search at finer scales in the
scale pyramid. To find translation and rotation
(next Lecture) Fourier Descriptors (see also PPT-
presentation FourierDescr at home page)
Using eigenvectors
Found insome smart
cameras.Master
thesis at SICK IVP
2007.
p. 28
8
Estimation of local orientation using the
structure tensor The structure tensor is a little 2x2-matrix that can
be computed in every pixel of the image. It contains combinations of 1D derivatives. It gives information about the structure in a
neighborhood of every pixel. It can be used in many different applications. Taught more theoretical in the Computer Vision
course (TSBB15).
Partl
y in
G&W
!H
arris
: pp.
869
-875
p. 29
Lab 6
Test image: The camera man slightly modified The image
contain edges and line-like structures and also an artificial pattern in the upper right part of the image.
p. 30
Estimation of local orientationโข The gradient โf(x,y) can be computed with two convolutions:
gx and gy are derivative filters, e.g. gx= sobelโx and gy = sobelโy, or Gaussian derivatives.
โข A very simple description of local orientaโtion is given by the direction of โf(x,y) :
p. 31
โ๐ ๐ฅ, ๐ฆ๐๐
๐๐ ๐ฅ, ๐ฆ๐๐ฅ
๐๐ ๐ฅ, ๐ฆ๐๐ฆ
๐ โ ๐ ๐ฅ, ๐ฆ๐ โ ๐ ๐ฅ, ๐ฆ
๐งโ๐ ๐ฅ, ๐ฆโ๐ ๐ฅ, ๐ฆ
Estimation of local orientationโข Problem 1: The angle of โf(x,y) changes 180ยฐ across a line.
โข Problem 2: โf(x,y) may be zero, even though there is a wellโdefined orientation.
p. 32
9
The gradient components, fxand fy, as two images Negative and positive values are found close
together in the line-like structures.
p. 33
Estimation of local orientationโข Form the outer product of the gradient with itself:
โข This is a symmetric 2ร2 matrix (a tensor):
โข Note: This is a preliminary stage of T. A neigh-borhood is needed to estimate local orientation. Therefore LP-filtering is necessary to form the true structure tensor T.
p. 34
โ๐ โ๐๐๐ ๐ ๐
๐ ๐ ๐
๐ ๐ ๐๐ ๐๐ ๐
๐ ๐๐ ๐ ๐ ๐ ๐ ๐
๐ ๐ ๐
Estimation of local orientationโข A complex representation of the gradient is:
โข Multiply this with itself and note that it can be written by using the tensor elements:
โข Note that the angle is doubled:
p. 35
๐ณ ๐ ๐๐ abs ๐ณ exp ๐ arg ๐ณ
๐ณ ๐ณ โ ๐ณ abs ๐ณ exp ๐2 โ arg ๐ณ
๐ณ ๐ณ โ ๐ณ ๐ ๐ ๐2 ๐ ๐ ๐ ๐ ๐2๐
Estimation of local orientationโข Problem 1 solved: The angle of z do not change across a line.
โข Solution to Problem 2: Perform lowโpass filtering of the individual elements, see 2โ4 slides ahead.
p. 36
angle for zg angle for z
10
Color coding ofโthe double angle representationโ
angle 0
angle ๐ 2โ
angle ๐
angle 3 ๐ 2โ
p. 37
T11 and T22 before LP-filteringโข Lineโlike objects in the original image give gaps in T11 and T22 (also in T12).
p. 38
LP-filtering As mentioned earlier, a neighborhood is needed
to estimate local orientation. Hence LP-filtering is necessary to form a true structure tensor T.
The LP-filter can be a Gaussian. It is separable and can be performed first in the x-direction and then in the y-direction.
The standard deviation sigma can be adjusted to a suitable size. Matlab example:>> sigma = 2;>> gauss = exp(-0.5*([-9:9]/sigma).^2);>> lp = gauss/sum(gauss);>> imageLP = conv2(image,lp,'same');>> imageLP = conv2(imageLP,lp','same');
p. 39
T11 and T22 after LP-filteringโข The gaps have been filled.
p. 40
11
Re(z)=T11-T22 and Im(z)=2๊T12
p. 41
abs(z) and arg(z) (its angle)
p. 42
Left:abs(z) and arg(z) combined in one image. Right: color coding
p. 43
Information in the eigenvalues Let ฮป1 and ฮป2 be the eigenvalues of T,
where ฮป1 โฅ ฮป2 It can be shown that for local regions
containing only 1D variation ฮป1โฅ0, ฮป2=0 It can be shown that for a local isotropic
region, ฮป1 = ฮป2
p. 44
12
ฮป1- and ฮป2-images
p. 45 Confidence measuresC1, C2 and Cโข From the identities:
โข we can define three confidence measures:
โข For regions containing only 1D variation: C2=0โข For a local isotropic region: C1=0 โข For regions with weak structures: C=โsmallโโข For regions with strong structures: C=โlargeโ
tr ๐ ๐ ๐ ๐ ๐det ๐ ๐ ๐ ๐ ๐ ๐
๐ถ tr ๐ 4 det ๐ ๐ ๐๐ถ 2 det ๐ 2๐ ๐๐ถ ๐ถ ๐ถ tr ๐ 2 det ๐ ๐ ๐
p. 46
C1- and C2-images
p. 47
10 10
To increase the visibility, we can use: sqrt(C1)- and sqrt(C2)-images
p. 48
13
Conclusion about 1D and 2D structures C1, C2 or sqrt(C1), sqrt(C2 ), are slightly better to
discriminate 1D or 2D structures than ฮป1- and ฮป2-images
C=C1+C2 can discriminate weak structures from strong structures
p. 49 Confidence measure:the Harris corner detector Compute the corner response function:
A) Find points with large values of cHarris
(thresholding).
B) Find local maxima points of cHarris
Multiply A) and B) => Harris points!
๐Harris det ๐ ๐ tr ๐ , ๐ 0.05
p. 50
(11-63)
cHarris and original image with overlaid Harris points in red
p. 51
10
Example) Automatic reading of barcodes on packages The first part
of automatic bar-code reading is to detect the positions of the barcodes.
p. 52
14
Sqrt(C1)- and Sqrt(C2)-images
p. 53 Classification of structures:1D and 2D, weak and strong C1 and C2 (or sqrt(C1) and sqrt(C2)) can
discriminate 1D and 2D structures. C=C1+C2 can discriminate weak structures from
strong structures.
p. 54
After classification Strong structures classified in 1D and 2D.
p. 55 1D structures after some morphology: Final result! Apply opening, closing and removal of small
objects and holes.
p. 56
top related