lecture 9: more lines from edges, connected componentsmtappen/cap5415/lecs/lec9.pdf · lecture 9:...

47
Lecture 9: More Lines from Edges, Connected Components CAP 5415: Computer Vision Fall 2010

Upload: others

Post on 23-May-2020

3 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Lecture 9: More Lines from Edges, Connected Componentsmtappen/cap5415/lecs/lec9.pdf · Lecture 9: More Lines from Edges, Connected Components CAP 5415: Computer Vision Fall 2010

Lecture 9: More Lines from Edges, Connected Components

CAP 5415: Computer VisionFall 2010

Page 2: Lecture 9: More Lines from Edges, Connected Componentsmtappen/cap5415/lecs/lec9.pdf · Lecture 9: More Lines from Edges, Connected Components CAP 5415: Computer Vision Fall 2010

From Edges to Lines

We’ve talked about detecting Edges, but how can we extract lines from those edges?

We'll start with a very basic, least squares approach

Page 3: Lecture 9: More Lines from Edges, Connected Componentsmtappen/cap5415/lecs/lec9.pdf · Lecture 9: More Lines from Edges, Connected Components CAP 5415: Computer Vision Fall 2010

Mathematically

See why it's called least squares?

Page 4: Lecture 9: More Lines from Edges, Connected Componentsmtappen/cap5415/lecs/lec9.pdf · Lecture 9: More Lines from Edges, Connected Components CAP 5415: Computer Vision Fall 2010

Review: Line Fitting

Page 5: Lecture 9: More Lines from Edges, Connected Componentsmtappen/cap5415/lecs/lec9.pdf · Lecture 9: More Lines from Edges, Connected Components CAP 5415: Computer Vision Fall 2010

Review

So, we can write the prediction at multiple points as

Page 6: Lecture 9: More Lines from Edges, Connected Componentsmtappen/cap5415/lecs/lec9.pdf · Lecture 9: More Lines from Edges, Connected Components CAP 5415: Computer Vision Fall 2010

Going back

Notice that

Can be rewritten as

(I'm using MATLAB notation for A(:,1))

Page 7: Lecture 9: More Lines from Edges, Connected Componentsmtappen/cap5415/lecs/lec9.pdf · Lecture 9: More Lines from Edges, Connected Components CAP 5415: Computer Vision Fall 2010

Sketch it Out

Page 8: Lecture 9: More Lines from Edges, Connected Componentsmtappen/cap5415/lecs/lec9.pdf · Lecture 9: More Lines from Edges, Connected Components CAP 5415: Computer Vision Fall 2010

Now we can do the same thing again

Notice that

Can be rewritten as

(I'm using MATLAB notation for A(:,2))

Page 9: Lecture 9: More Lines from Edges, Connected Componentsmtappen/cap5415/lecs/lec9.pdf · Lecture 9: More Lines from Edges, Connected Components CAP 5415: Computer Vision Fall 2010

Now, setting everything to zero

Page 10: Lecture 9: More Lines from Edges, Connected Componentsmtappen/cap5415/lecs/lec9.pdf · Lecture 9: More Lines from Edges, Connected Components CAP 5415: Computer Vision Fall 2010

Also called pseudo-inverse

You will often see this appear as

Why all the matrix hassle?This will allow us to easily generalize to

higher-dimensions

Page 11: Lecture 9: More Lines from Edges, Connected Componentsmtappen/cap5415/lecs/lec9.pdf · Lecture 9: More Lines from Edges, Connected Components CAP 5415: Computer Vision Fall 2010

This is also called regression

• Can think of this as trying to predict y from x

• Classification and regression are two fundamental problems in pattern recognition

• What about non-linear regression?

Page 12: Lecture 9: More Lines from Edges, Connected Componentsmtappen/cap5415/lecs/lec9.pdf · Lecture 9: More Lines from Edges, Connected Components CAP 5415: Computer Vision Fall 2010

Non-Linear Regression

• We can use the same trick that we used for classification

• Make our estimate be a linear combination of a non-linear transformation of x

• Possible example:

Page 13: Lecture 9: More Lines from Edges, Connected Componentsmtappen/cap5415/lecs/lec9.pdf · Lecture 9: More Lines from Edges, Connected Components CAP 5415: Computer Vision Fall 2010

Example

(Similar to Example from Bishop's Book)

Page 14: Lecture 9: More Lines from Edges, Connected Componentsmtappen/cap5415/lecs/lec9.pdf · Lecture 9: More Lines from Edges, Connected Components CAP 5415: Computer Vision Fall 2010

What's going on here?

(Similar to Example from Bishop's Book)

Page 15: Lecture 9: More Lines from Edges, Connected Componentsmtappen/cap5415/lecs/lec9.pdf · Lecture 9: More Lines from Edges, Connected Components CAP 5415: Computer Vision Fall 2010

What's going on here?

(Similar to Example from Bishop's Book)

• This is classic over-fitting• Not enough data, too much flexibility in

classifier

Page 16: Lecture 9: More Lines from Edges, Connected Componentsmtappen/cap5415/lecs/lec9.pdf · Lecture 9: More Lines from Edges, Connected Components CAP 5415: Computer Vision Fall 2010

Two Solutions• Solution 1: Model Selection• We want the simplest model that matches

the data well

Page 17: Lecture 9: More Lines from Edges, Connected Componentsmtappen/cap5415/lecs/lec9.pdf · Lecture 9: More Lines from Edges, Connected Components CAP 5415: Computer Vision Fall 2010

Solution #2: Regularization• Smooths out bumps in the curve

Notice that we haveoversmoothed the

Lower-orderpolynomials!

Page 18: Lecture 9: More Lines from Edges, Connected Componentsmtappen/cap5415/lecs/lec9.pdf · Lecture 9: More Lines from Edges, Connected Components CAP 5415: Computer Vision Fall 2010

Mathematically

• We will change the training criterion from

• To

Page 19: Lecture 9: More Lines from Edges, Connected Componentsmtappen/cap5415/lecs/lec9.pdf · Lecture 9: More Lines from Edges, Connected Components CAP 5415: Computer Vision Fall 2010

More generally• This is the general matrix formulation

• Which changes to

Page 20: Lecture 9: More Lines from Edges, Connected Componentsmtappen/cap5415/lecs/lec9.pdf · Lecture 9: More Lines from Edges, Connected Components CAP 5415: Computer Vision Fall 2010

What if we have multiple lines?

Page 21: Lecture 9: More Lines from Edges, Connected Componentsmtappen/cap5415/lecs/lec9.pdf · Lecture 9: More Lines from Edges, Connected Components CAP 5415: Computer Vision Fall 2010

We'll Need to Fit Multiple Lines

• Two Problems:• Problem 1: How many lines?

• We'll assume we know this• Problem 2: Which points go to which lines?

• We'll figure this out as part of the optimization process

Page 22: Lecture 9: More Lines from Edges, Connected Componentsmtappen/cap5415/lecs/lec9.pdf · Lecture 9: More Lines from Edges, Connected Components CAP 5415: Computer Vision Fall 2010

Mathematical Formulation• Again, we will define a cost function

• Note that we are still minimizing a squared error, but we have included a min term

• This says, “I only count the minimum error”• Practical effect- each line only has to cover

part of the data set

Page 23: Lecture 9: More Lines from Edges, Connected Componentsmtappen/cap5415/lecs/lec9.pdf · Lecture 9: More Lines from Edges, Connected Components CAP 5415: Computer Vision Fall 2010

Optimizing ThisNote, this will be very difficult to optimize

So we will introduce a second set of variables

Page 24: Lecture 9: More Lines from Edges, Connected Componentsmtappen/cap5415/lecs/lec9.pdf · Lecture 9: More Lines from Edges, Connected Components CAP 5415: Computer Vision Fall 2010

What this means

z assigns every point to one line

Every entry is either 0 or 1 Only one entry is equal to one

Page 25: Lecture 9: More Lines from Edges, Connected Componentsmtappen/cap5415/lecs/lec9.pdf · Lecture 9: More Lines from Edges, Connected Components CAP 5415: Computer Vision Fall 2010

Optimizing this

• This leads to a natural two-step process• 1. Optimize θ for each line –

• this is the same least-squares formulation that we used before, except we only consider the points assigned to that line

• 2. Optimize z• Set z so that the line is assigned to the line with the

minimum error

Page 26: Lecture 9: More Lines from Edges, Connected Componentsmtappen/cap5415/lecs/lec9.pdf · Lecture 9: More Lines from Edges, Connected Components CAP 5415: Computer Vision Fall 2010

An ExampleWe start with the points unassignedChoose lines randomly

Page 27: Lecture 9: More Lines from Edges, Connected Componentsmtappen/cap5415/lecs/lec9.pdf · Lecture 9: More Lines from Edges, Connected Components CAP 5415: Computer Vision Fall 2010

An Example

Page 28: Lecture 9: More Lines from Edges, Connected Componentsmtappen/cap5415/lecs/lec9.pdf · Lecture 9: More Lines from Edges, Connected Components CAP 5415: Computer Vision Fall 2010

An Example

Page 29: Lecture 9: More Lines from Edges, Connected Componentsmtappen/cap5415/lecs/lec9.pdf · Lecture 9: More Lines from Edges, Connected Components CAP 5415: Computer Vision Fall 2010

An Example

Page 30: Lecture 9: More Lines from Edges, Connected Componentsmtappen/cap5415/lecs/lec9.pdf · Lecture 9: More Lines from Edges, Connected Components CAP 5415: Computer Vision Fall 2010

An Example

Page 31: Lecture 9: More Lines from Edges, Connected Componentsmtappen/cap5415/lecs/lec9.pdf · Lecture 9: More Lines from Edges, Connected Components CAP 5415: Computer Vision Fall 2010

An Example

Page 32: Lecture 9: More Lines from Edges, Connected Componentsmtappen/cap5415/lecs/lec9.pdf · Lecture 9: More Lines from Edges, Connected Components CAP 5415: Computer Vision Fall 2010

An Example

Page 33: Lecture 9: More Lines from Edges, Connected Componentsmtappen/cap5415/lecs/lec9.pdf · Lecture 9: More Lines from Edges, Connected Components CAP 5415: Computer Vision Fall 2010

Fitting Multiple Lines

Note that we did have to know how many lines we had

This is not guaranteed to converge to a global minimum

Similar to the k-means algorithm that we will study soon.

Page 34: Lecture 9: More Lines from Edges, Connected Componentsmtappen/cap5415/lecs/lec9.pdf · Lecture 9: More Lines from Edges, Connected Components CAP 5415: Computer Vision Fall 2010

Another Approach

Basic Idea: Let edge points vote for possible lines

Page 35: Lecture 9: More Lines from Edges, Connected Componentsmtappen/cap5415/lecs/lec9.pdf · Lecture 9: More Lines from Edges, Connected Components CAP 5415: Computer Vision Fall 2010

Hough Transform

• Parameterize lines as

• Every line can be expressed as a pair of values, θ and r

• To find lines, we’ll let each point vote for the possible lines that it could lie on.

Page 36: Lecture 9: More Lines from Edges, Connected Componentsmtappen/cap5415/lecs/lec9.pdf · Lecture 9: More Lines from Edges, Connected Components CAP 5415: Computer Vision Fall 2010

Basic Idea• Discretize the space of θ and r• Each point “votes” for all (θ, r) combinations that result in a line

passing through that point• Combinations corresponding to lines should receive many votes

θ

r(From Slides by Forsyth)

Page 37: Lecture 9: More Lines from Edges, Connected Componentsmtappen/cap5415/lecs/lec9.pdf · Lecture 9: More Lines from Edges, Connected Components CAP 5415: Computer Vision Fall 2010

What if there’s noise?

(From Slides by Forsyth)

Page 38: Lecture 9: More Lines from Edges, Connected Componentsmtappen/cap5415/lecs/lec9.pdf · Lecture 9: More Lines from Edges, Connected Components CAP 5415: Computer Vision Fall 2010

What if there are no lines?

(From Slides by Forsyth)

Page 39: Lecture 9: More Lines from Edges, Connected Componentsmtappen/cap5415/lecs/lec9.pdf · Lecture 9: More Lines from Edges, Connected Components CAP 5415: Computer Vision Fall 2010

The Hough Transform

• Advantages:• Don’t need to know the number of lines

ahead of time• Conceptually Simple• Can be extended to other types of shapes

• Disadvantages:• Noise leads to lots of phantom lines• Have to pick the right quantization

Page 40: Lecture 9: More Lines from Edges, Connected Componentsmtappen/cap5415/lecs/lec9.pdf · Lecture 9: More Lines from Edges, Connected Components CAP 5415: Computer Vision Fall 2010

Voting Paradigm

• This voting paradigm can be extended to very different problems

Page 41: Lecture 9: More Lines from Edges, Connected Componentsmtappen/cap5415/lecs/lec9.pdf · Lecture 9: More Lines from Edges, Connected Components CAP 5415: Computer Vision Fall 2010

Voting

• Can use to find actions

Page 42: Lecture 9: More Lines from Edges, Connected Componentsmtappen/cap5415/lecs/lec9.pdf · Lecture 9: More Lines from Edges, Connected Components CAP 5415: Computer Vision Fall 2010

Overall Pipeline

Page 43: Lecture 9: More Lines from Edges, Connected Componentsmtappen/cap5415/lecs/lec9.pdf · Lecture 9: More Lines from Edges, Connected Components CAP 5415: Computer Vision Fall 2010

Changing Gears

The edge-detection algorithm gives us a binary map.

It would be useful to group these edges into coherent structures.

Page 44: Lecture 9: More Lines from Edges, Connected Componentsmtappen/cap5415/lecs/lec9.pdf · Lecture 9: More Lines from Edges, Connected Components CAP 5415: Computer Vision Fall 2010

Connected Components Algorithm

Divides binary image into groups of connected pixels

First, need to define the neighborhood

Page 45: Lecture 9: More Lines from Edges, Connected Componentsmtappen/cap5415/lecs/lec9.pdf · Lecture 9: More Lines from Edges, Connected Components CAP 5415: Computer Vision Fall 2010

Algorithm is easily explained recursively

Page 46: Lecture 9: More Lines from Edges, Connected Componentsmtappen/cap5415/lecs/lec9.pdf · Lecture 9: More Lines from Edges, Connected Components CAP 5415: Computer Vision Fall 2010

Connected Components

Called bwlabel in MATLAB image processing toolbox

Page 47: Lecture 9: More Lines from Edges, Connected Componentsmtappen/cap5415/lecs/lec9.pdf · Lecture 9: More Lines from Edges, Connected Components CAP 5415: Computer Vision Fall 2010

Why do lines matter?

• Lines tell us about the structure of the world and the orientations of the camera

• In this model, we assume that the image holds lines that align with the world axes

(From Kosecka and Zhang - “Video Compass”)