cs223b, jana kosecka image features local, meaningful, detectable parts of the image. line detection...
Post on 20-Dec-2015
213 views
TRANSCRIPT
CS223b, Jana Kosecka
Image Features
Local, meaningful, detectable parts of the image.• Line detection • Corner detectionMotivation• Information content high• Invariant to change of view point, illumination• Reduces computational burden• Uniqueness • Can be tuned to a task at hand
CS223b, Jana Kosecka
Canne Edge Detector
• Edge detection involves 3 steps:– Noise smoothing– Edge enhancement– Edge localization
• J. Canny formalized these steps to design an optimal edge detector
• How to go from derivatives to edges ?
Horizontal edges
Before:
CS223b, Jana Kosecka
• Compute image derivatives • if gradient magnitude > and the value is a local maximum along gradient direction – pixel is an edge candidate
Canny edge detector
gradient magnitudeoriginal image
Edge Detection
CS223b, Jana Kosecka
Algorithm Canny Edge detector
• The input is image I; G is a zero mean Gaussian filter (std = )
– J = I * G (smoothing)– For each pixel (i,j): (edge enhancement)
– Compute the image gradient 1. J(i,j) = (Jx(i,j),Jy(i,j))’
1. Estimate edge strength » es(i,j) = (Jx
2(i,j)+ Jy2(i,j))1/2
– Estimate edge orientation 1. eo(i,j) = arctan(Jx(i,j)/Jy(i,j))
• The output are images Es - Edge Strength - Magnitude• and Edge Orientation Eo
-
CS223b, Jana Kosecka
• Es has large values at edges: Find local maxima
• … but it also may have wide ridges around the local maxima (large values around the edges)
ThTh
CS223b, Jana Kosecka
NONMAX_SUPRESSION
• The inputs are Es & Eo (outputs of CANNY_ENHANCER)• Consider 4 directions D={ 0,45,90,135} wrt x
• For each pixel (i,j) do:• Find the direction dD s.t. d Eo(i,j) (normal to
the edge)
• If {Es(i,j) is smaller than at least one of its neigh. along d} • IN(i,j)=0
• Otherwise, IN(i,j)= Es(i,j)
1. The output is the thinned edge image IN
Edge orientationEdge orientation
CS223b, Jana Kosecka
Thresholding
• Edges are found by thresholding the output of NONMAX_SUPRESSION
• If the threshold is too high:– Very few (none) edges
•High MISDETECTIONS, many gaps• If the threshold is too low:
– Too many (all pixels) edges•High FALSE POSITIVES, many extra edges
CS223b, Jana Kosecka
SOLUTION: Hysteresis Thresholding
Es(i,j)Es(i,j)> H> H
Es(i,j)<Es(i,j)<HH
Es(i,j)Es(i,j)>L>L
Es(i,j)<LEs(i,j)<LEs(i,j)Es(i,j)>L>L
CS223b, Jana Kosecka
Canny Edge Detection (Example)
courtesy of G. Loy
gap is gone
Originalimage
Strongedgesonly
Strong +connectedweak edges
Weakedges
CS223b, Jana Kosecka
First-order derivative filters (1D)
• Sharp changes in gray level of the input image correspond to “peaks” of the first-derivative of the input signal.
F(x)F(x)F ’(x)F ’(x)
xx
CS223b, Jana Kosecka
Second-order derivative filters (1D)
• Peaks of the first-derivative of the input signal, correspond to “zero-crossings” of the second-derivative of the input signal.
F(x)F(x) F ’(x)F ’(x)
xx
F’’(x)F’’(x)
CS223b, Jana Kosecka
NOTE:
• F’’(x)=0 is not enough!– F’(x) = c has F’’(x) = 0, but there is no edge
• The second-derivative must change sign, -- i.e. from (+) to (-) or from (-) to (+)
• The sign transition depends on the intensity change of the image – i.e. from dark to bright or vice versa.
CS223b, Jana Kosecka
Edge Detection (2D)
1D1D 2D2D
I(x)I(x) I(x,y)I(x,y)
dd22I(x)I(x)dxdx22
= 0= 0
xx
yy
||I(x,y)| =(II(x,y)| =(Ix x 22(x,y) + I(x,y) + Iyy
22(x,y))(x,y))1/2 1/2 > Th> Th
tan tan = I = Ixx(x,y)/ I(x,y)/ Iyy(x,y) (x,y)
F(x)F(x)
xx
dI(x)dI(x)dxdx
> Th> Th
22I(x,y) =II(x,y) =Ix x x x (x,y) + I(x,y) + Iyy yy (x,y)=0(x,y)=0
LaplaciaLaplacia
nn
CS223b, Jana Kosecka
Notes about the Laplacian:
• 22I(x,y) is a SCALARI(x,y) is a SCALAR Can be found using a SINGLE maskCan be found using a SINGLE mask Orientation information is lostOrientation information is lost
22I(x,y) is the sum of SECOND-order derivativesI(x,y) is the sum of SECOND-order derivatives– But taking derivatives increases noiseBut taking derivatives increases noise– Very noise sensitive!Very noise sensitive!
• It is always combined with a smoothing It is always combined with a smoothing operation:operation:
CS223b, Jana Kosecka
LOG Filter
• First smooth (Gaussian filter),• Then, find zero-crossings (Laplacian filter):– O(x,y) = 22(I(x,y) * G(x,y))(I(x,y) * G(x,y))
• Using linearity:– O(x,y) = 22G(x,y) * I(x,y)G(x,y) * I(x,y)– This filter is called: “Laplacian of the This filter is called: “Laplacian of the Gaussian” (LOG)Gaussian” (LOG)
CS223b, Jana Kosecka
First Derivative of a Gaussian
2
2
2
2
22
222
2
1)('
xx
ex
xexg−−
−=−=
PositivPositivee NegativeNegative
As a mask, it is also computing a difference (derivative)As a mask, it is also computing a difference (derivative)
CS223b, Jana Kosecka
Second Derivative of a Gaussian
2
2
23
2
)1
()(''
x
ex
xg−
−=
2D2D
““Mexican Hat”Mexican Hat”
CS223b, Jana Kosecka
Finding lines in an image
• Option 1:– Search for the line at every possible position/orientation
– What is the cost of this operation?
• Option 2:– Use a voting scheme: Hough transform
CS223b, Jana Kosecka
Finding lines in an image
• Connection between image (x,y) and Hough (m,b) spaces– A line in the image corresponds to a point in Hough space
– To go from image space to Hough space:• given a set of points (x,y), find all (m,b) such that y = mx + b
x
y
m
b
m0
b0
image space Hough space
CS223b, Jana Kosecka
Finding lines in an image
• Connection between image (x,y) and Hough (m,b) spaces– A line in the image corresponds to a point in Hough space– To go from image space to Hough space:
• given a set of points (x,y), find all (m,b) such that y = mx + b
– What does a point (x0, y0) in the image space map to?
x
y
m
b
image space Hough space
• A: the solutions of b = -x0m + y0
• this is a line in Hough space
x0
y0
CS223b, Jana Kosecka
Hough transform algorithm• Typically use a different
parameterization
– d is the perpendicular distance from the line to the origin
is the angle this perpendicular makes with the x axis
– Why?Idea – keep an accumulator array (Hough space)and let each edge pixel contribute to it Line candidates are the maxima in the accumulatorarray
CS223b, Jana Kosecka
Typical Hough Transform
Basic Hough transform algorithm
1. Initialize H[d, q]=02. For each edge point I[x,y] in the image3. For q = 0 to 180 H[d, q] += 1 where point is now a sinusoid in Hough space Find the value(s) of (d, q) where H[d, q] is maximum
The detected line in the image is given b
What’s the running time (measured in # votes)?
CS223b, Jana Kosecka
Hough Transform for Curves
• The H.T. can be generalized to detect any curve that can be expressed in parametric form:– Y = f(x, a1,a2,…ap)– a1, a2, … ap are the parameters– The parameter space is p-dimensional– The accumulating array is LARGE!
CS223b, Jana Kosecka
x
y
• Edge detection, non-maximum suppression (traditionally Hough Transform – issues of resolution, threshold selection and search for peaks in Hough space)• Connected components on edge pixels with similar orientation - group pixels with common orientation
Non-max suppressed gradient magnitude
Line fitting
CS223b, Jana Kosecka
• Line fitting lines determined from eigenvalues and eigenvectors of A• Candidate line segments - associated line quality
second moment matrixassociated with eachconnected componentv1 - eigenvector of A
Line Fitting
CS223b, Jana Kosecka
Corners contain more edges than lines.
• A point on a line is hard to match.
Corner detection
CS223b, Jana Kosecka
Finding Corners
Intuition:
• Right at corner, gradient is ill defined.
• Near corner, gradient has two different values.
CS223b, Jana Kosecka
Formula for Finding Corners
⎥⎥⎦
⎤
⎢⎢⎣
⎡=
∑∑∑∑
2
2
yyx
yxx
III
IIIC
We look at matrix:
Sum over a small region, the hypothetical corner
Gradient with respect to x, times gradient with respect to y
Matrix is symmetric
CS223b, Jana Kosecka
⎥⎦
⎤⎢⎣
⎡=
⎥⎥⎦
⎤
⎢⎢⎣
⎡=
∑∑∑∑
2
12
2
0
0
λ
λ
yyx
yxx
III
IIIC
First, consider case where:
This means all gradients in neighborhood are:
(k,0) or (0, c) or (0, 0) (or off-diagonals cancel).
What is region like if:
1. l1 = 0?
2. l2 = 0?
3. l1 = 0 and l2 = 0?
4. l1 > 0 and l2 > 0?
CS223b, Jana Kosecka
General Case:
From Linear Algebra, it follows that because C is symmetric:
RRC ⎥⎦
⎤⎢⎣
⎡= −
2
11
00λ
λ
With R a rotation matrix.
So every case is like one on last slide.
CS223b, Jana Kosecka
So, to detect corners
• Filter image.• Compute magnitude of the gradient everywhere.
• We construct C in a window.• Use Linear Algebra to find λ1and λ2.• If they are both big, we have a corner.
CS223b, Jana Kosecka
• Compute eigenvalues of G• If smalest eigenvalue of G is bigger than - mark pixel as candidate feature point
• Alternatively feature quality function (Harris Corner Detector)
Point Feature Extraction
CS223b, Jana Kosecka
% Harris Corner detector - by Kashif Shahzadsigma=2; thresh=0.1; sze=11; disp=0;
% Derivative masksdy = [-1 0 1; -1 0 1; -1 0 1];dx = dy'; %dx is the transpose matrix of dy % Ix and Iy are the horizontal and vertical edges of imageIx = conv2(bw, dx, 'same');Iy = conv2(bw, dy, 'same'); % Calculating the gradient of the image Ix and Iyg = fspecial('gaussian',max(1,fix(6*sigma)), sigma);Ix2 = conv2(Ix.^2, g, 'same'); % Smoothed squared image derivativesIy2 = conv2(Iy.^2, g, 'same');Ixy = conv2(Ix.*Iy, g, 'same'); % My preferred measure according to research papercornerness = (Ix2.*Iy2 - Ixy.^2)./(Ix2 + Iy2 + eps); % We should perform nonmaximal suppression and thresholdmx = ordfilt2(cornerness,sze^2,ones(sze)); % Grey-scale dilatecornerness = (cornerness==mx)&(cornerness>thresh); % Find maxima[rws,cols] = find(cornerness);
clf ; imshow(bw); hold on;p=[cols rws];plot(p(:,1),p(:,2),'or'); title('\bf Harris Corners')