geometric transformation - university of...

Geometric TransformationGeometric Transformation

• Operations depend on pixel’s Coordinates.

• Context free.

• Independent of pixel values.

( )( ) ',

',

yyxfy

xyxfx

y

x

=→

=→( ) ( )( )yxfyxfIyxI yx ,,,'),( =

(x’,y’)(x,y)

I(x,y) I’(x’,y’)

• Example: Translation

( )( ) 1,

3,

−==′

+==′

yyxfy

xyxfx

y

x

( ) ),(1,3' yxIyxI =−+

(x,y)

I(x,y) I’(x’,y’)

(x’,y’)

Forward MappingForward Mapping

• Forward mapping:

Forward

Mapping

Source Target

( )( ) yyxfy

xyxfx

y

x

′=→

′=→

,

,

• Problems with forward mapping due to sampling:

– Holes (some target pixels are not populated)

– Overlaps (some target pixels assigned few colors)

Forward MappingForward Mapping

Source Target

x=1 x=2 x=1 x=2 x=3 x=4

( ) yyxfy y==′ ,( ) xyxfx x

==′ 2,

TargetSource

x=1 x=2 x=1 x=2

?

( ) yyxfy y==′ ,( ) xyxfx x

==′ 7.0,

• Inverse mapping:

• Each target pixel assigned a single color.

• Color Interpolation is required.

Inverse

Mapping

( )( ) yyxfy

xyxfx

y

x

=′′→′

=′′→′−

−

,

,

1

1

TargetSource

Inverse MappingInverse Mapping

• Example: Scaling along X

– Forward mapping:

– Inverse mapping:

(0,0)

(0,0)

Source Target

Source Target

yyxx =′=′ ;2

yyxx ′=′= ;2/

InterpolationInterpolation

• What happens when a mapping function calculates a

fractional pixel location?

• Interpolation: generates a new pixel by analyzing the

surrounding pixels.

• Good interpolation techniques attempt to find an optimal

balance between three undesirable artifacts: aliasing,

blurring, and edge halos.

9

InterpolationInterpolation

x4 scaling

aliasing blurring halos

Nearest Neighbor InterpolationNearest Neighbor Interpolation

• The assign value is taken from the pixel closest to the

generated location:

• Advantage:

– Fast

• Disadvantage:

– Jagged results

– Aliasing near edges

( ) ( ){ } ( ){ }( )yxfroundyxfroundIyxI yx′′′′=′′′ −− ,,,, 11

Original ImageNearest N.

Interpolation

Original Image

Nearest N.

Interpolation

Bilinear InterpolationBilinear Interpolation

• The assign value is a weighted sum of the four

nearest pixels.

• Each weight is proportional to the distance from each

existing pixel.

Linear InterpolationLinear Interpolation

• Isolating v in the above equation:

xw xe

vw

ve

x

v

we

w

we

w

vv

vv

xx

xx

−−

=−−

( ) we vvv αα −+= 1we

w

xx

xxwhere

−−

=α

NW

NE

SESW

x∆

S

N

V


( )xSWxSES ∆−⋅+∆⋅= 1

( )xNWxNEN ∆−⋅+∆⋅= 1

( )ySyNV ∆−⋅+∆⋅= 1

y∆

NW

NE

SESW

x∆

S

N

V


y∆

• The bilinear interpolation is the best fit low-degree

polynomial of the form:

• The pixel’s boundaries are C0 continuous

(continuous values across boundaries).

ji1

0ji

ij yxayxv ∆∆=∆∆ ∑=,

),(

Bilinear example

z=15 z=7

z=2 z=3

∆x

∆y

1

1.5

2

1

1.5

20

5

10

15

v

∆x ∆y

0

10

Nearest N.Interpolation

Bilinear Interpolation

Bicubic InterpolationBicubic Interpolation

• The assign value is a weighted sum of the 4x4

nearest pixels:

ji3

0ji

ij yxayxv ∆∆=∆∆ ∑=,

),(

How can we find the right coefficients?

• Denote the pixel values Vpq {p,q=0..3}

• The unknown coefficients are aij {i,j=0..3}

21

],[,},..{,,

21yx30qpforyxav ji3

0ji

ijpq −∈∆∆=∆∆=∑=

s

t

• We have a linear system of 16

equations with 16 coefficients.

• The pixel’s boundaries are C1

continuous (continuous

derivatives across boundaries).

N.N Bilinear Bicubic

Bilinear

Bicubic

Applying the TransformationApplying the Transformation

T = …… % 2x2 transformation matrix[r,c] = size(img)

% create array of destination x,y coordinates[X,Y]=meshgrid(1:c,1:r);

% calculate source coordinatessourceCoor = inv(T) * [X(:) Y(:) ] ‘ ;

% calculate nearest neighbor interpolationXs = round(sourceCoor(1,:));Ys = round(sourceCoor(2,:));

indx=find(Xs<1 | Xs>r); %out of range pixelsXs(indx)=1; Ys(indx)=1;

indy=find(Ys<1 | Ys>c); %out of range pixelsXs(indy)=1; Ys(indy)=1;

% calculate new imagenewImage = img((Xs-1).*r+Ys);newImage(indx)=0; newImage(indx)=0; newImage = reshape(newImage,r,c);

Types of linear 2D transformationsTypes of linear 2D transformations

• Rigid (Euclidean) transformation:

– Translation + Rotation (distance preserving).

• Similarity transformation:

– Translation + Rotation + Uniform Scale (angle preserving).

• Affine transformation:

– Translation + Rotation + Scale + Shear (parallelism preserving).

• Projective transformation

– Cross-ratio preserving

• All above transformations are groups where Rigid ⊂

Similarity ⊂ Affine ⊂ Projective

Types of linear 2D transformationsTypes of linear 2D transformations

• All above transformations are groups where Rigid ⊂

Similarity ⊂ Affine ⊂ Projective

Matrix NotationMatrix Notation

• Every location (x,y) is treated as a column vector:

• Coordinate transformation is obtained by

multiplying with a 2x2 matrix?

y

x

+

+=

dycx

byax

y

x

dc

ba

Matrix Notation - ScaleMatrix Notation - Scale

=

y

x

y

x

b

a

'

'

0

0

1

1

• Scale(a,b): (x,y) (ax,by)

• If a or b are negative, we get reflection.

• Inverse: S-1(a,b)=S(1/a,1/b)

=

by

ax

y

x

b

a

0

0

Matrix Notation - ShearMatrix Notation - Shear

• Shear(a,b): (x,y) (x+ay,y+bx)

+

+=

bxy

ayx

y

x

b

a

1

1

a=0.5, b=0

Matrix Notation - RotationMatrix Notation - Rotation

• Rotate(θ):

(x,y) (xcosθ+ysinθ , -xsinθ + ycosθ)

• Inverse: R-1(θ)=RT(θ)=R(-θ)

+−

+=

− θθ

θθθθθθ

cossin

sincos

cossin

sincos

yx

yx

y

x

Matrix Notation - TranslationMatrix Notation - Translation

• Translation(a,b):

• Cannot represent translation using

2x2 matrices.

• Inverse:

+

+→

by

ax

y

x

−′

−′→

′

′

by

ax

y

x

Homogeneous CoordinatesHomogeneous Coordinates

• Homogeneous Coordinates is a mapping from Rn to Rn+1:

• Note: (tx,ty,t) all correspond to the same non-

homogeneous point (x,y). E.g. (2,3,1)≡(6,9,3) ≡(4,6,2).

• Inverse mapping:

( ) ( )yx,,,, =

→W

Y

W

XWYX

),,(),,(),( ttytxWYXyx ≡→

Homogeneous CoordinatesHomogeneous Coordinates

(x,y)

1

(tx,ty,t)

=(x,y,1)

Some 2D TransformationsSome 2D Transformations

• Translation :

• Affine transformation:

• Projective transformation:

+

+

=

=

′

′

′

11100

10

01

y

x

y

x

ty

tx

y

x

t

t

W

Y

X

=

′

′

′

1100

y

x

tdc

tba

W

Y

X

y

x

=

′

′

′

11

y

x

fe

tdc

tba

W

Y

X

y

x

Affine

Hierarchy of Linear 2D TransformationsHierarchy of Linear 2D Transformations

Global Transformations – Image RectificationGlobal Transformations – Image Rectification

Global Transformations – Global WarpingGlobal Transformations – Global Warping

=

′

′

1

y

x

A

1

y

x


points pi points qimatch

=

111

ppp

ppp

A

111

qqq

qqqY

3

y

2

y

1

x3

x2

x1

y

3

y

2

y

1

x3

x2

x1

...

...

...

...


points pi points qimatch

=

−

111

qqq

qqq

A

111

ppp

pppy

3

y

2

y

1

x3

x2

x1

1Y

3

y

2

y

1

x3

x2

x1

...

...

...

...

Inverse Mapping:


Solve for A-1 in terms of the least mean square.

i.e. find A-1 which minimizes:

∑ −−i

ii qAp 2

1 ||||


solution:

( ) 1)(

−= TT

XXXXpinv

=−

111111

321

321

321

321

1...

...

...

...yyy

xxx

Yyy

xxx

qqq

qqq

pinvppp

ppp

A


Alternative representation:

=

′

′

f

e

d

c

b

a

yx

yx

y

x

1000

0100

=

=

′

′

110011

y

x

fdc

eba

y

x

Ay

x

Rearrange:


=

fedcba

qq

qq

p

p

yx

yx

y

x

1000

010011

11

1

1


=

fedcba

qq

qq

qq

qq

qq

qq

p

p

p

p

p

p

yx

yx

yx

yx

yx

yx

y

x

y

x

y

x

1000

0100

1000

0100

1000

0100

33

33

22

22

11

11

3

3

2

2

1

1


=

p

p

p

p

p

p

qq

qq

qq

qq

qq

qq

y

x

y

x

y

x

yx

yx

yx

yx

yx

yx

pinv

f

e

d

c

b

a

3

3

2

2

1

1

33

33

22

22

11

11

1000

0100

1000

0100

1000

0100

'

( ) TTQQQQpinv

1)('

−=

solution:


What about Projective Transformations?

=

′

′

111

y

x

hg

fdc

eba

y

x

Homogeneity must be preserved!

1';

1'

++

++=

++

++=

hygx

fdycxy

hygx

ebyaxx


What about Projective Transformations?

1';

1'

++

++=

++

++=

hygx

fdycxy

hygx

ebyaxx

( )

−−=′

h

g

f

e

d

c

b

a

xyxxyxx ''0100

And similarly for y’

Global Transformations – Image RectificationGlobal Transformations – Image Rectification

So who ARE we?

Local Transformations – Image WarpingLocal Transformations – Image Warping

Demoalex


ps

pd

Area of influence

ps = source pointpd = destination point

ps

pd

Area of influence


ps

pd

Area of influence

ps = source pointpd = destination point

source

destination

ps pd

pd

ps

Image Morphing (Image Metamorphosis)Image Morphing (Image Metamorphosis)

Demo bw

Cross Dissolve (pixel operations)Cross Dissolve (pixel operations)

DestinationImage

SourceImage

( ) ( )]1,0[

1

∈

⋅+⋅−=

t

TtSttI

t

cross dissolve

warp + dissolve

Warping + Cross Dissolve Warping + Cross Dissolve

• Warp source image towards intermediate image.

• Warp destination image towards intermediate image.

• Cross-dissolve the two images by taking the weighted average at each pixel.

time

Cross-dissolve

warping images

source

destination

warp

warp

Cross-dissolve

Cross-dissolve

Image MetamorphosisImage Metamorphosis

• Let S,T be the source and the target images

• Let G(p) be the transformation from S towards T, where

G(0)=I (the identity)

• Let t∈[0,1] the time step to be synthesized

Algorithm:

1. Warp S towards T:

2. Warp T toward S:

3. Cross dissolve:

( ) ( ){ }SptGtS ⋅=

( ) ( )( ) { }TptGtT1

1−⋅−=

( ) ( ) ( ) ( )tTttSttI ⋅+⋅−= 1

t

sourse

target

S(t)=G(t⋅p){S}

T(t)=G((1-t)⋅p)-1{T}

I(t)=(1-t)⋅S(t)+t ⋅(T(t))

Feature Based MorphingFeature Based Morphing

• Morph one shape into another shape

• Use local features to define the geometric warping

P

Q

P’

Q’

• α∈[0,1] is the relative position along the segment (P’,Q’).

• β is the actual perpendicular distance to the segment.

• (u’,v’) is the local coordinates of the segment (P’,Q’):

– u’ is a unit vector parallel to Q’-P’

– v’ is the unit vector perpendicular to Q’-P’

α

P

QRβ

α

P’

Q’

R’β

Source Image Dest Image

uv

u’ v’

( )PQ

PQu

′−′′−′

=′

′−

′=′=⊥′

x

y

u

uuv

One Segment WarpingOne Segment Warping

α

P’

Q’

R’β

u’ v’

( ) ( ) vPRPQ

uPR′⋅′−′=

′−′′⋅′−′

= βα ;

• The point R’ is mapped into (α,β) :

where

'''''' vuPQPR βα +−+=

αP

QRβ

α

P’

Q’

R’β

Source Image Dest Image

uv

u’ v’

Inverse Mapping:

where (u,v) is the local coordinates of the segment (P,Q):

vuPQPR βαβα +−+=),(

( )PQ

PQu

−−

=

−==⊥

x

y

u

uuv

Multiple Segment WarpingMultiple Segment Warping

• In multiple segment warping the point R’ is influenced by multiple segments.

• The influence strength of each segments is proportional to:

– Segment length

– The distance from the point R’

P1

Q1R1

β1

P2

Q2β 2

P1’

Q1’

R’β’1

β’2

P2’

Q2’

R2

• The influence of each segments is:

• The value p∈[0,1] controls the influence of the line length.

• The value a is a small number avoiding division by zero.

• The value b determines how the relative weight diminish as the β

increases

• The final mapping is:

b

i

p

ii

ia

PQW

β+

−=

∑∑

=

k

k

k

kk

W

RW

R

Example:Example:

For more details see:

Thaddeus Beier & Shawn Neely / Feature-Based Image Metamorphosis Siggraph '92

http://www.hammerhead.com/thad/morph.html

Example images from:

http://www.cc.gatech.edu/classes/AY2001/cs4451_spring/projects/Seven/

73

Another Example:Another Example:

Mesh WarpingMesh Warping

From:

http://www.cs.utk.edu/~huangj/CS594F01/imageMorph.ppt

2-Pass Mesh Warping Algorithm2-Pass Mesh Warping Algorithm

The first pass warps the rows of the image:

For each column of the mesh determine the x-coordinates at

which the mesh column crosses each image row.

2-Pass Mesh Warping Algorithm2-Pass Mesh Warping Algorithm

Then, each row of the image is warped individually bylinearly interpolating each segment between the x-coordinates defined by the source mesh to the size of the corresponding segment defined by the x-coordinates of the destination Mesh.

The second pass performs the exact same procedure on

the columns of the image by interpolating the y-coordinates of the meshes.

VidMorph

Fun Morph

geometric transformation - university of...

Documents