instructor: mircea nicolescu lecture 13 cs 485 / 685 computer vision

Instructor: Mircea Nicolescu

Lecture 13

CS 485 / 685

Computer Vision

2

Deformable/Active Contours (Snakes)

• Snakes− Goal – find a contour that best approximates the perimeter of an

object.− It is helpful to visualize it as a rubber band of arbitrary shape that is

capable of deforming in time, trying to get as close as possible to the target contour.

− It is applied to the gradient magnitude everywhere in the image, not just to the edge points (e.g., like the Hough transform).

3


• Snakes – cont.

− Snakes do not solve the entire problem of finding contours in images.

− They depend on other mechanisms such as interaction with a user or with some other higher-level computer vision mechanism

1) First, the snake is placed near the image contour of interest.

2) During an iterative process, the snake is attracted towards the target contour by various forces that control the shape and location of the snake within the image.

4


• Approach− It is based on constructing an energy functional which measures the

appropriateness of the contour.− Good solutions correspond to minima of the functional.− The goal is to minimize this functional with respect to the contour

parameters.

• Contour parameterization− The snake is a contour represented

parametrically as:

c(s) = (x(s), y(s))

where x(s) and y(s) are the coordinates along the contour and

s [0,1]

5


• The energy functional

− The energy functional used is a sum of several terms, each corresponding to some force acting on the contour.

− A suitable energy functional is the sum the following three terms:

− The parameters α, β and γ control the relative influence of the corresponding energy terms and can vary along c.

6


• Interpretation of the functional’s terms

− Each energy term serves a different purpose:Eimage: it attracts the contour toward the closest image edge.

Econt: it forces the contour to be continuous.

Ecurv: it forces the contour to be smooth.

− Econt and Ecurv are called internal energy terms.

− Eimage is called external energy term.

7


• The continuity term

− Minimize the first derivative:

− In the discrete case, the contour is approximated by N points p1, p2, . . . , pN and the first derivative is approximated by a finite difference:

− This term tries to minimize the distance between the points, however, it has the effect of causing the contour to shrink.

8


• The continuity term

− A better form for Econt is the following:

where d is the average distance between the points of the snake.

− The new Econt attempts to keep the points at equal distances (i.e, spread them equally along the snake).

9


• The smoothness term

− The purpose of this term is to enforce smoothness and avoid oscillations of the snake by penalizing high contour curvatures.

− Minimize the second derivative (curvature):

− In the discrete case, the curvature can be approximated by the following finite difference:

10


• The edge attraction term

− The purpose of this term is to attract the contour toward the target contour.

− This can be achieved by the following function:

where I is the gradient of the intensity computed at each snake point.

− Note that Eimage becomes very small when the snake points get close to an edge.

11


• Discrete formulation of the problem

− Assumptions− Let I be an image and p’1, ..., p’N the initial locations of the snake (evenly

spaced, chosen close to the contour of interest).

− Problem Statement− Starting from p’1, ..., p’N, find the deformable contour p1, ..., pN which fits

the target contour by minimizing the energy functional:

12


• A greedy algorithm− A greedy algorithm makes locally optimal choices, hoping that the

final solution will be globally optimal.− Step1 (greedy minimization): each point of the snake is moved

within a small neighborhood (e.g., MxM) to the point which minimizes the energy functional

− Step 2 (allow for corners): search for corners (curvature extrema) along the contour; if a corner is found at point pj, set βj to zero.

13


14


15

− It is important to normalize the contribution of each term for correct implementation:

1) For Econt and Ecurv, it is sufficient to divide by the largest value in the neighborhood in which the point can move.

2) Normalize as

where min and max are the minimum and maximum gradient values in the neighborhood.


• Implementation issues

• Comments− Simple approach, low computational requirements− Does not guarantee convergence to the global minimum of the

functional.− Works very well if the initial snake is not too far from the desired

solution.

Feature Descriptors and Matching

Extract affine regions Normalize regionsEliminate rotational

ambiguityCompute appearance

descriptors

SIFT (Lowe ’04)

Descriptors

16

Simplest Approach: Correlation

2 2

2 2

2 2 2 22 1/2 2 1/2

2 2 2 2

( , ) ( , )

( , )

[ ( , )] [ ( , )]

n n

n nk l

n n n n

n n n nk l k l

h k l f i k j l

N i j

h k l f i k j l

17


• Works satisfactorily when matching corresponding regions related mostly by translation.− e.g., stereo pairs, or video sequence assuming small

camera motion

18


• Sensitive to small variations with respect to:− Location− Pose− Scale− Intra-class variability

• Poorly distinctive!

• Need more powerful descriptors!

19

Scale Invariant Feature Transform (SIFT)

16 histograms x 8 orientations = 128 elements vector

1. Take a 16 x16 window around interest point (i.e., at the scale detected).

2. Divide into a 4x4 grid of cells.

3. Compute histogram of image gradients in each cell (8 bins each).

20

Properties of SIFT

• Highly distinctive− A single feature can be correctly matched with high

probability against a large database of features from many images.

• Scale and rotation invariant• Partially invariant to 3D camera viewpoint

− Can tolerate up to about 60 degree out-of-plane rotation

• Partially invariant to changes in illumination• Can be computed fast and efficiently

21

Example

22

SIFT Computation – Steps

(1) Scale-space extrema detection− Extract scale and rotation invariant interest points

(keypoints)

(2) Keypoint localization− Determine location and scale for each interest point− Eliminate “weak” keypoints

(3) Orientation assignment− Assign one or more orientations to each keypoint

(4) Keypoint descriptor− Use local image gradients at the selected scale

D. Lowe, “Distinctive Image Features from Scale-Invariant Keypoints”, International Journal of Computer Vision, 60(2):91-110, 2004.

Cited 9589 times (as of 3/7/2011)23

instructor: mircea nicolescu lecture 13 cs 485 / 685 computer vision

Documents

image contour

contour parameters

target contour

contour parameterization

e image

external energy term

high contour curvatures

continuity term