3d computer vision generalized hough boundaries representations snakes (trucco: 109-113)...

3D Computer VisionGeneralized Hough

BoundariesRepresentationsSnakes (Trucco: 109-113)

Computational Vision / Ioannis Stamos

Generalized Hough TransformDetecting shapes

not described by an analytic equation

CSc 83020 3-D Computer Vision – Ioannis Stamos

CSc 83020 3-D Computer Vision – Ioannis Stamos

FromShree Nayar’sNotes


Find template t in image f

t and f are both gray-scale images

DiscussionCan you search for matches in

Different scale (size) than template?Different orientation than template?


ComparisonTemplate matching

vs.Hough


To MATCH image boundary/region with MODEL

boundary/region, they must represented in the same

manner.•Boundary Representation•Snakes – Extraction of arbitrary contours from image.•Region Representation

Representation IssuesCompact: Easy to Store & Match.Easy to manipulate & compute properties.Captures Object/Model shape.Computationally efficient.

Polylines: concatenation of line segments.

Breakpoint

Matching on the basis of:# of line segmentslengths of line segmentsangle between consecutive segments

ei

Move along the boundaryAt each point find line that fits previous points

(Least Squares)Compute the fit error E=Sum(ei) using previous pointsIf E exceeds threshold, declare breakpoint and start

a new running line fit.

•Draw Straight line between end-points of curve•For every curve point find distance to line.•If distance is less than tolerance level for all points, Exit•Else, pick point that is farthest away and use as breakpoint.

Introduce new segments.•Recursively apply algorithm to new segments.

s ψ

x

y

s

Ψ

1 2

3 4

5 6

x

y

12

3

4

5

6

π

2π

s ψ

x

y

s

Ψ

1 2

3 4

5 6

x

y

12

3

4

5

6

Ψ-s is periodic (2π)Horizontal section in Ψ-s curve

=> straight line in the ImageNon horizontal section in Ψ-s

=> arc in Image

π

2π

s ψ

x

y H(Ψ)

Ψ

π 2π

HISTOGRAMLines

Arcs

s ψ

x

y H(Ψ)

Ψ

π 2π

HISTOGRAM

H(Ψ) shifts as objects rotates.H(Ψ) wraps around

Lines

Arcs

s ψ

x

y

Find: Ψ(s)Define:Φ(s)= Ψ(s)-(2πs)/P

P: Perimeter.2π: Period of Φ(s)

Φ(s) is a Continuous, Periodic function.Fourier Series for Periodic Functions:

Fourier Coefficients:

Pes

k

sikk

2,)( 0

1

0

P

sikk dsesP 0

0)(1

Φk’s capture shape informationMatch shapes by matching Φk’sUse finite number of Φk’s

Fourier DescriptorsInput Shape Power Spectrum Reconstruction

# of coefficients

Φk

Note: Reconstructed shapes are often not closed sinceonly a finite # of Φk’s are used.

1

0

)()(N

iii sBvsX

•Piecewise continuous polynomials used to INTERPOLATE between Data Points.•Smooth, Flexible, Accurate.

s=0

s=1s=2

x0

x1x2

xi

x

s

Spline X(s)

Data Point

We want to find X(s) from points xiCubic Polynomials are popular:

BASIS FUNCTIONS

COEFFICIENTS

N

Bi(s) has limited support (4 spans)

i-2 i-1 i i+1 i+2

Each span (i->i+1) has only 4 non-zero Basis Functions:Bi-1(s), Bi(s), Bi+1(s), Bi+2(s)

i i+1

Bi(s)

Bi-1(s) Bi+2(s)

Bi+1(s)

t=0 t=1t

s:

s:

3-D Computer Vision CSc 83020 – Ioannis Stamos

i i+1

Bi(s)

Bi-1(s) Bi+2(s)

Bi+1(s)

t=0 t=1t

CubicPolynomials

1/6

4/6


i i+1

Bi(s)

Bi-1(s) Bi+2(s)

Bi+1(s)

t=0 t=1t

6/)()(

6/)1333()()(

6/)463()()(

6/)133()()(

332

2321

231

2301

ttCsB

ttttCsB

tttCsB

ttttCsB

i

i

i

i

CubicPolynomials

1/6

4/6


0141

0303

0363

1331

6

1

1)(

)()()()()(

21123

2312110

C

vvvvCttttx

vtCvtCvtCvtCtxT

iiiii

iiiii

If we compute v0, … vN => continuous representation for the curve.

)0(

...

)0()0()0()0(...............................)0(

)0()0()0()0(................)0(

)0()0()0()0()0(

4332211022

3322110011

2312011000

NN xx

vCvCvCvCxx

vCvCvCvCxx

vCvCvCvCxx

NN v

v

v

v

x

x

x

x

...

410......0

0............0

0...1410

0...0141

0...0014

6

1

...2

1

0

2

1

0

We have our N+1 data points:

And in matrix form:

We can solve for the vi’s from the xi’sBoundary condition for closed curves: v0=vN, v1=vN+1.

Tiiiii vvvvCttttx 21123 1)(

So, for any i we can find:

xi xi+1

xi(t)

t

t=0 t=1

Note: Local support.Spline passes through all data points xi.

B-Spline demo: http://www.doc.ic.ac.uk/~dfg/AndysSplineTutorial/BSplines.html

Fitting a curve to an arbitrary shape.Active/Deformable contour or Snake.

Elastic band of arbitrary shape.Located near the image contour.

Attracted towards the target contour.

Stops when Energy Functional is minimized.

[Davatzikos and Prince][Davatzikos and Prince]

Demo using gimp“Intelligent scissors”

User-Visible OptionsInitialization: user-specified, automaticCurve properties: continuity, smoothnessImage features: intensity, edges, corners, …Other forces: hard constraints, springs,

attractors, repulsors, …Scale: local, multiresolution, global

Behind-the-Scenes OptionsFramework: energy minimization, forces

acting on curveCurve representation: ideal curve, sampled,

spline, implicit functionEvolution method: calculus of variations,

numerical differential equations, local search

Snakes: Active Contour ModelsIntroduced by Kass, Witkin, and TerzopoulosFramework: energy minimization

Bending and stretching curve = more energyGood features = less energyCurve evolves to minimize energy

Also “Deformable Contours”

Snakes Energy EquationParametric representation of curve

Energy functional consists of three terms

)(),()( sysxs v )(),()( sysxs v

)()()(int sss conimg vvv )()()(int sss conimg vvv

Internal Energy

First term is “membrane” term – minimum energy when curve minimizes length(“soap bubble”)

Second term is “thin plate” term – minimum energy when curve is smooth

2)()()()()(22

int sssss sss vvv 2)()()()()(22

int sssss sss vvv

Internal Energy

Control and to vary between extremesSet to 0 at a point to allow cornerSet to 0 everywhere to let curve follow

sharp creases – “strings”

2)()()()()(22

int sssss sss vvv 2)()()()()(22

int sssss sss vvv

Edge AttractionGradient-based:

Laplacian-based:

In both cases, can smooth with GaussianSnakes paper, figures 3 and 4

2),( yxIwimg 2),( yxIwimg

22 ),( yxIwimg 22 ),( yxIwimg

Corner AttractionCan use corner detector we saw last timeAlternatively, let = tan-1 Iy / Ix

and let nbe a unit vector perpendicular to the gradient. Then

Snakes paper, figures 5 and 6

n

wimg

n

wimg

Variants on SnakesBalloons [Cohen 91]

Add inflation force

Helps avoid getting stuck on small features

)(skFinfl n )(skFinfl n

[Cohen 91][Cohen 91]

dsEsEsEs imagecurvcont ))()()((

)(sc

)(),()( sysxs c

Parameterized by arc-length

[Kass, Witkan and Terzopoulos ’87]

Minimize


)(scContinuity – compactness

2

1

2

iicont ds

dE pp

c

)(),()( sysxs c



Minimize



21

2

)( iicont dds

dE pp

c

)(),()( sysxs c



Minimize



)(),()( sysxs c



Minimize

Smoothness

2

11 2 iiicurvE ppp

21

2

)( iicont dds

dE pp

c



)(),()( sysxs c



Minimize

Smoothness

2

11 2 iiicurvE ppp

Edge Attraction Term

2IEimage

21

2

)( iicont dds

dE pp

c

N

iimageicurviconti EEE

1

Problem Statement:

Start with chain of image locations representing initial position of contour.

Find the deformable contour

by minimizing the energy functional:

Nppp ,...,, 21

Nppp ,...,, 21

How can we minimize

i

imageicurviconti EEE

Discrete version: sum over the points.

Greedy minimization:For each point compute the best move over a small area.

How can we minimize

i




Set βj=0 for points of high curvature (corners)

How can we minimize

i




Set βj=0 for points of high curvature (corners)

Stop when a user-specified fraction of points does not move.

Synthetic Results

Real Experiment

Snakes demo: http://www.markschulze.net/snakes/

ScaleIn the simplest snakes algorithm, image

features only attract locallyGreater region of attraction: smooth image

Curve might not follow high-frequency detailMultiresolution processing

Start with smoothed image to attract curveFinish with unsmoothed image to get details

Looking for global minimum vs. local minima

Diffusion-Based MethodsAnother way to attract curve to localized

features: vector flow or diffusion methodsExample:

Find edges using CannyFor each point, compute distance to

nearest edgePush curve along gradient of distance field

Xu and PrinceXu and Prince


Simple SnakeSimple Snake With Gradient Vector FieldWith Gradient Vector Field


1 11 1 1 1 1

1 1 1 1 11 1 1 1

1

Spatial Occupancy Array

•Easy to implement.•Large Storage area.•Can apply set operations (unite/intersect).•Expensive for matching!


Quad TreesEfficient encoding ofSpatial Occupancy Arrayusing Resolution Pyramids.

Black: Fully Occupied.White: Empty.Gray: Partially Occupied.


Quad TreeLevel 0Level 1

Level 2

Level 3

Quad Tree Generation:Start with level 0. If Black or White, Terminate. Else

declare Gray node & expand with four sons.

For each son repeat above step.NW NE

SW SE

3d computer vision generalized hough boundaries representations snakes (trucco: 109-113)...

Documents

straight line

concatenation of line

s curve

starta new running line

new segments

curve point

fourier coefficients

image ft