fitting - lunds tekniska högskola · fitting we want to associate a modelwith observed features...
TRANSCRIPT
![Page 1: Fitting - Lunds tekniska högskola · Fitting We want to associate a modelwith observed features [Fig from Marszalek & Schmid, 2007] For example, the model could be a line, a circle,](https://reader033.vdocuments.us/reader033/viewer/2022052008/601d8dea26c2367d62309bb4/html5/thumbnails/1.jpg)
Fitting
Lecture 6
Cristian Sminchisescu
Slide credits: K. Grauman, S. Seitz, S. Lazebnik, D. Forsyth, J. Ponce
![Page 2: Fitting - Lunds tekniska högskola · Fitting We want to associate a modelwith observed features [Fig from Marszalek & Schmid, 2007] For example, the model could be a line, a circle,](https://reader033.vdocuments.us/reader033/viewer/2022052008/601d8dea26c2367d62309bb4/html5/thumbnails/2.jpg)
FittingWe want to associate a model with observed features
[Fig from Marszalek & Schmid, 2007]
For example, the model could be a line, a circle, or an arbitrary shape.We use a model to produce compact representations that emphasizethe relevant image structures.
![Page 3: Fitting - Lunds tekniska högskola · Fitting We want to associate a modelwith observed features [Fig from Marszalek & Schmid, 2007] For example, the model could be a line, a circle,](https://reader033.vdocuments.us/reader033/viewer/2022052008/601d8dea26c2367d62309bb4/html5/thumbnails/3.jpg)
Example: Line fittingWhy fit lines?
Many objects characterized by presence of straight lines
![Page 4: Fitting - Lunds tekniska högskola · Fitting We want to associate a modelwith observed features [Fig from Marszalek & Schmid, 2007] For example, the model could be a line, a circle,](https://reader033.vdocuments.us/reader033/viewer/2022052008/601d8dea26c2367d62309bb4/html5/thumbnails/4.jpg)
![Page 5: Fitting - Lunds tekniska högskola · Fitting We want to associate a modelwith observed features [Fig from Marszalek & Schmid, 2007] For example, the model could be a line, a circle,](https://reader033.vdocuments.us/reader033/viewer/2022052008/601d8dea26c2367d62309bb4/html5/thumbnails/5.jpg)
![Page 6: Fitting - Lunds tekniska högskola · Fitting We want to associate a modelwith observed features [Fig from Marszalek & Schmid, 2007] For example, the model could be a line, a circle,](https://reader033.vdocuments.us/reader033/viewer/2022052008/601d8dea26c2367d62309bb4/html5/thumbnails/6.jpg)
Example: Line fittingWhy fit lines?
Many objects characterized by presence of straight lines
Why aren’t we done just by running edge detection?
![Page 7: Fitting - Lunds tekniska högskola · Fitting We want to associate a modelwith observed features [Fig from Marszalek & Schmid, 2007] For example, the model could be a line, a circle,](https://reader033.vdocuments.us/reader033/viewer/2022052008/601d8dea26c2367d62309bb4/html5/thumbnails/7.jpg)
• Extra edge points (clutter), multiple models:
– which points go with which line, if any?
• Only some parts of each line detected, and some parts are missing:
– how to find a line that bridges missing evidence?
• Noise in measured edge points, orientations:
– how to detect true underlying parameters?
Difficulty of line fitting
![Page 8: Fitting - Lunds tekniska högskola · Fitting We want to associate a modelwith observed features [Fig from Marszalek & Schmid, 2007] For example, the model could be a line, a circle,](https://reader033.vdocuments.us/reader033/viewer/2022052008/601d8dea26c2367d62309bb4/html5/thumbnails/8.jpg)
Voting• It is not easy to check all combinations of features by fitting a
model to each possible subset (but see later RANSAC)
• Voting is a general technique where we let the features vote for all models that are compatible with it
– Cycle through features, cast votes for model parameters.
– Look for model parameters that receive a lot of votes.
• Noise and clutter features will cast votes too, but typically their votes should be inconsistent with the majority of “good” features. Hence they will have less support.
• Ok if some features not observed, as model can span multiple fragments.
![Page 9: Fitting - Lunds tekniska högskola · Fitting We want to associate a modelwith observed features [Fig from Marszalek & Schmid, 2007] For example, the model could be a line, a circle,](https://reader033.vdocuments.us/reader033/viewer/2022052008/601d8dea26c2367d62309bb4/html5/thumbnails/9.jpg)
Hough transform algorithmUsing the polar parameterization:
Basic Hough transform algorithm1. Initialize H[ , ]=02. for each edge point I[x,y] in the image
for = 0 to 180 // some quantization
H[ , ] += 13. Find the value(s) of ( , ) where H[ , ] is maximum4. The detected line in the image is given by
H: accumulator array (votes)
Complexity (in terms of number of votes)?
sincos yx
Source: Steve Seitz
sincos yx
sincos yx
![Page 10: Fitting - Lunds tekniska högskola · Fitting We want to associate a modelwith observed features [Fig from Marszalek & Schmid, 2007] For example, the model could be a line, a circle,](https://reader033.vdocuments.us/reader033/viewer/2022052008/601d8dea26c2367d62309bb4/html5/thumbnails/10.jpg)
tokens
Note that most points in the vote array are very dark, because they get only one vote.
![Page 11: Fitting - Lunds tekniska högskola · Fitting We want to associate a modelwith observed features [Fig from Marszalek & Schmid, 2007] For example, the model could be a line, a circle,](https://reader033.vdocuments.us/reader033/viewer/2022052008/601d8dea26c2367d62309bb4/html5/thumbnails/11.jpg)
tokens votes
What difficulty does this present for an implementation?
![Page 12: Fitting - Lunds tekniska högskola · Fitting We want to associate a modelwith observed features [Fig from Marszalek & Schmid, 2007] For example, the model could be a line, a circle,](https://reader033.vdocuments.us/reader033/viewer/2022052008/601d8dea26c2367d62309bb4/html5/thumbnails/12.jpg)
Here, everything appears to be “noise”, or random edge points, but we still see peaks in the vote space.
tokens votes
![Page 13: Fitting - Lunds tekniska högskola · Fitting We want to associate a modelwith observed features [Fig from Marszalek & Schmid, 2007] For example, the model could be a line, a circle,](https://reader033.vdocuments.us/reader033/viewer/2022052008/601d8dea26c2367d62309bb4/html5/thumbnails/13.jpg)
ExtensionsExtension 1: Use the image gradient
1. same2. for each edge point I[x,y] in the image
= gradient at (x,y)
H[ , ] += 13. same4. same
(Reduces degrees of freedom)
Extension 2• give more votes for stronger edges
Extension 3• change the sampling of (d, ) to give more/less resolution
Extension 4• The same procedure can be used with circles, squares, or any
other shape
sincos yx
![Page 14: Fitting - Lunds tekniska högskola · Fitting We want to associate a modelwith observed features [Fig from Marszalek & Schmid, 2007] For example, the model could be a line, a circle,](https://reader033.vdocuments.us/reader033/viewer/2022052008/601d8dea26c2367d62309bb4/html5/thumbnails/14.jpg)
ExtensionsExtension 1: Use the image gradient
1. same2. for each edge point I[x,y] in the image
compute unique ( , ) based on image gradient at (x,y)H[ , ] += 1
3. same4. same
(Reduces degrees of freedom)
Extension 2• give more votes for stronger edges (use magnitude of gradient)
Extension 3• change the sampling of ( , ) to give more/less resolution
Extension 4• The same procedure can be used with circles, squares, or any
other shape…
![Page 15: Fitting - Lunds tekniska högskola · Fitting We want to associate a modelwith observed features [Fig from Marszalek & Schmid, 2007] For example, the model could be a line, a circle,](https://reader033.vdocuments.us/reader033/viewer/2022052008/601d8dea26c2367d62309bb4/html5/thumbnails/15.jpg)
![Page 16: Fitting - Lunds tekniska högskola · Fitting We want to associate a modelwith observed features [Fig from Marszalek & Schmid, 2007] For example, the model could be a line, a circle,](https://reader033.vdocuments.us/reader033/viewer/2022052008/601d8dea26c2367d62309bb4/html5/thumbnails/16.jpg)
Showing longest segments found
![Page 17: Fitting - Lunds tekniska högskola · Fitting We want to associate a modelwith observed features [Fig from Marszalek & Schmid, 2007] For example, the model could be a line, a circle,](https://reader033.vdocuments.us/reader033/viewer/2022052008/601d8dea26c2367d62309bb4/html5/thumbnails/17.jpg)
Voting: practical tips
• Minimize irrelevant tokens first (take edge points with significant gradient magnitude)
• Choose a good grid / discretization– Too coarse: large votes obtained when too many different lines
correspond to a single bucket– Too fine: miss lines because some points that are not exactly collinear
cast votes for different buckets
• Vote for neighbors, also (smoothing in accumulator array)• To read back which points voted for “winning” peaks, keep
tags on the votes.
![Page 18: Fitting - Lunds tekniska högskola · Fitting We want to associate a modelwith observed features [Fig from Marszalek & Schmid, 2007] For example, the model could be a line, a circle,](https://reader033.vdocuments.us/reader033/viewer/2022052008/601d8dea26c2367d62309bb4/html5/thumbnails/18.jpg)
Hough transform: pros and consPros• All points are processed independently, so can cope with
occlusion• Some robustness to noise: noise points unlikely to contribute
consistently to any single bin• Can detect multiple instances of a model in a single pass
Cons• Complexity of search time increases exponentially with the
number of model parameters • Non‐target shapes can produce spurious peaks in parameter
space• Quantization: hard to pick a good grid size
![Page 19: Fitting - Lunds tekniska högskola · Fitting We want to associate a modelwith observed features [Fig from Marszalek & Schmid, 2007] For example, the model could be a line, a circle,](https://reader033.vdocuments.us/reader033/viewer/2022052008/601d8dea26c2367d62309bb4/html5/thumbnails/19.jpg)
Generalized Hough transformTo detect the model shape in a new image:
• For each edge point
– Index into table with its gradient orientation θ
– Use retrieved r vectors to vote for position of reference point
• Peak in this Hough space is reference point with most supporting edges
Assuming translation is the only transformation here, i.e., orientation and scale are fixed.
![Page 20: Fitting - Lunds tekniska högskola · Fitting We want to associate a modelwith observed features [Fig from Marszalek & Schmid, 2007] For example, the model could be a line, a circle,](https://reader033.vdocuments.us/reader033/viewer/2022052008/601d8dea26c2367d62309bb4/html5/thumbnails/20.jpg)
Example
model shape Source: L. Lazebnik
Say we’ve already stored a table of displacement vectors as a function of edge orientation for this model shape.
![Page 21: Fitting - Lunds tekniska högskola · Fitting We want to associate a modelwith observed features [Fig from Marszalek & Schmid, 2007] For example, the model could be a line, a circle,](https://reader033.vdocuments.us/reader033/viewer/2022052008/601d8dea26c2367d62309bb4/html5/thumbnails/21.jpg)
Example
displacement vectors for model points
Now we want to look at some edge points detected in a newimage, and vote on the position of that shape.
![Page 22: Fitting - Lunds tekniska högskola · Fitting We want to associate a modelwith observed features [Fig from Marszalek & Schmid, 2007] For example, the model could be a line, a circle,](https://reader033.vdocuments.us/reader033/viewer/2022052008/601d8dea26c2367d62309bb4/html5/thumbnails/22.jpg)
Example
range of voting locations for test point
![Page 23: Fitting - Lunds tekniska högskola · Fitting We want to associate a modelwith observed features [Fig from Marszalek & Schmid, 2007] For example, the model could be a line, a circle,](https://reader033.vdocuments.us/reader033/viewer/2022052008/601d8dea26c2367d62309bb4/html5/thumbnails/23.jpg)
Example
range of voting locations for test point
![Page 24: Fitting - Lunds tekniska högskola · Fitting We want to associate a modelwith observed features [Fig from Marszalek & Schmid, 2007] For example, the model could be a line, a circle,](https://reader033.vdocuments.us/reader033/viewer/2022052008/601d8dea26c2367d62309bb4/html5/thumbnails/24.jpg)
Example
votes for points with θ =
![Page 25: Fitting - Lunds tekniska högskola · Fitting We want to associate a modelwith observed features [Fig from Marszalek & Schmid, 2007] For example, the model could be a line, a circle,](https://reader033.vdocuments.us/reader033/viewer/2022052008/601d8dea26c2367d62309bb4/html5/thumbnails/25.jpg)
Example
displacement vectors for model points
![Page 26: Fitting - Lunds tekniska högskola · Fitting We want to associate a modelwith observed features [Fig from Marszalek & Schmid, 2007] For example, the model could be a line, a circle,](https://reader033.vdocuments.us/reader033/viewer/2022052008/601d8dea26c2367d62309bb4/html5/thumbnails/26.jpg)
Example
range of voting locations for test point
![Page 27: Fitting - Lunds tekniska högskola · Fitting We want to associate a modelwith observed features [Fig from Marszalek & Schmid, 2007] For example, the model could be a line, a circle,](https://reader033.vdocuments.us/reader033/viewer/2022052008/601d8dea26c2367d62309bb4/html5/thumbnails/27.jpg)
votes for points with θ =
Example
![Page 28: Fitting - Lunds tekniska högskola · Fitting We want to associate a modelwith observed features [Fig from Marszalek & Schmid, 2007] For example, the model could be a line, a circle,](https://reader033.vdocuments.us/reader033/viewer/2022052008/601d8dea26c2367d62309bb4/html5/thumbnails/28.jpg)
![Page 29: Fitting - Lunds tekniska högskola · Fitting We want to associate a modelwith observed features [Fig from Marszalek & Schmid, 2007] For example, the model could be a line, a circle,](https://reader033.vdocuments.us/reader033/viewer/2022052008/601d8dea26c2367d62309bb4/html5/thumbnails/29.jpg)
![Page 30: Fitting - Lunds tekniska högskola · Fitting We want to associate a modelwith observed features [Fig from Marszalek & Schmid, 2007] For example, the model could be a line, a circle,](https://reader033.vdocuments.us/reader033/viewer/2022052008/601d8dea26c2367d62309bb4/html5/thumbnails/30.jpg)
![Page 31: Fitting - Lunds tekniska högskola · Fitting We want to associate a modelwith observed features [Fig from Marszalek & Schmid, 2007] For example, the model could be a line, a circle,](https://reader033.vdocuments.us/reader033/viewer/2022052008/601d8dea26c2367d62309bb4/html5/thumbnails/31.jpg)
![Page 32: Fitting - Lunds tekniska högskola · Fitting We want to associate a modelwith observed features [Fig from Marszalek & Schmid, 2007] For example, the model could be a line, a circle,](https://reader033.vdocuments.us/reader033/viewer/2022052008/601d8dea26c2367d62309bb4/html5/thumbnails/32.jpg)
Computer Vision ‐ A Modern ApproachSet: Fitting
Slides by D.A. Forsyth
![Page 33: Fitting - Lunds tekniska högskola · Fitting We want to associate a modelwith observed features [Fig from Marszalek & Schmid, 2007] For example, the model could be a line, a circle,](https://reader033.vdocuments.us/reader033/viewer/2022052008/601d8dea26c2367d62309bb4/html5/thumbnails/33.jpg)
Computer Vision ‐ A Modern ApproachSet: Fitting
Slides by D.A. Forsyth
![Page 34: Fitting - Lunds tekniska högskola · Fitting We want to associate a modelwith observed features [Fig from Marszalek & Schmid, 2007] For example, the model could be a line, a circle,](https://reader033.vdocuments.us/reader033/viewer/2022052008/601d8dea26c2367d62309bb4/html5/thumbnails/34.jpg)
Computer Vision ‐ A Modern ApproachSet: Fitting
Slides by D.A. Forsyth
Robustness
• As we have seen, squared error can be a source of bias in the presence of noise points– One fix is EM ‐ we’ll do this shortly– Another is an M‐estimator
• Square nearby, threshold far away
– A third is RANSAC• Search for good points
![Page 35: Fitting - Lunds tekniska högskola · Fitting We want to associate a modelwith observed features [Fig from Marszalek & Schmid, 2007] For example, the model could be a line, a circle,](https://reader033.vdocuments.us/reader033/viewer/2022052008/601d8dea26c2367d62309bb4/html5/thumbnails/35.jpg)
Computer Vision ‐ A Modern ApproachSet: Fitting
Slides by D.A. Forsyth
![Page 36: Fitting - Lunds tekniska högskola · Fitting We want to associate a modelwith observed features [Fig from Marszalek & Schmid, 2007] For example, the model could be a line, a circle,](https://reader033.vdocuments.us/reader033/viewer/2022052008/601d8dea26c2367d62309bb4/html5/thumbnails/36.jpg)
Computer Vision ‐ A Modern ApproachSet: Fitting
Slides by D.A. Forsyth
![Page 37: Fitting - Lunds tekniska högskola · Fitting We want to associate a modelwith observed features [Fig from Marszalek & Schmid, 2007] For example, the model could be a line, a circle,](https://reader033.vdocuments.us/reader033/viewer/2022052008/601d8dea26c2367d62309bb4/html5/thumbnails/37.jpg)
Computer Vision ‐ A Modern ApproachSet: Fitting
Slides by D.A. Forsyth
![Page 38: Fitting - Lunds tekniska högskola · Fitting We want to associate a modelwith observed features [Fig from Marszalek & Schmid, 2007] For example, the model could be a line, a circle,](https://reader033.vdocuments.us/reader033/viewer/2022052008/601d8dea26c2367d62309bb4/html5/thumbnails/38.jpg)
Computer Vision ‐ A Modern ApproachSet: Fitting
Slides by D.A. Forsyth
![Page 39: Fitting - Lunds tekniska högskola · Fitting We want to associate a modelwith observed features [Fig from Marszalek & Schmid, 2007] For example, the model could be a line, a circle,](https://reader033.vdocuments.us/reader033/viewer/2022052008/601d8dea26c2367d62309bb4/html5/thumbnails/39.jpg)
Computer Vision ‐ A Modern ApproachSet: Fitting
Slides by D.A. Forsyth
![Page 40: Fitting - Lunds tekniska högskola · Fitting We want to associate a modelwith observed features [Fig from Marszalek & Schmid, 2007] For example, the model could be a line, a circle,](https://reader033.vdocuments.us/reader033/viewer/2022052008/601d8dea26c2367d62309bb4/html5/thumbnails/40.jpg)
Computer Vision ‐ A Modern ApproachSet: Fitting
Slides by D.A. Forsyth
![Page 41: Fitting - Lunds tekniska högskola · Fitting We want to associate a modelwith observed features [Fig from Marszalek & Schmid, 2007] For example, the model could be a line, a circle,](https://reader033.vdocuments.us/reader033/viewer/2022052008/601d8dea26c2367d62309bb4/html5/thumbnails/41.jpg)
Computer Vision ‐ A Modern ApproachSet: Fitting
Slides by D.A. Forsyth
Too small
![Page 42: Fitting - Lunds tekniska högskola · Fitting We want to associate a modelwith observed features [Fig from Marszalek & Schmid, 2007] For example, the model could be a line, a circle,](https://reader033.vdocuments.us/reader033/viewer/2022052008/601d8dea26c2367d62309bb4/html5/thumbnails/42.jpg)
Computer Vision ‐ A Modern ApproachSet: Fitting
Slides by D.A. Forsyth
Too large
![Page 43: Fitting - Lunds tekniska högskola · Fitting We want to associate a modelwith observed features [Fig from Marszalek & Schmid, 2007] For example, the model could be a line, a circle,](https://reader033.vdocuments.us/reader033/viewer/2022052008/601d8dea26c2367d62309bb4/html5/thumbnails/43.jpg)
Computer Vision ‐ A Modern ApproachSet: Fitting
Slides by D.A. Forsyth
![Page 44: Fitting - Lunds tekniska högskola · Fitting We want to associate a modelwith observed features [Fig from Marszalek & Schmid, 2007] For example, the model could be a line, a circle,](https://reader033.vdocuments.us/reader033/viewer/2022052008/601d8dea26c2367d62309bb4/html5/thumbnails/44.jpg)
![Page 45: Fitting - Lunds tekniska högskola · Fitting We want to associate a modelwith observed features [Fig from Marszalek & Schmid, 2007] For example, the model could be a line, a circle,](https://reader033.vdocuments.us/reader033/viewer/2022052008/601d8dea26c2367d62309bb4/html5/thumbnails/45.jpg)
as the noise increases in a picture without a line, the number of points in the max cell goes up, too