Comparison of Image Comparison of Image Registration MethodsRegistration Methods

David GrimmDavid GrimmJoseph HandfieldJoseph Handfield

Mahnaz MohammadiMahnaz MohammadiYushan Zhu Yushan Zhu

March 18, 2004March 18, 2004

Registration task:Registration task: Match two or more images taken:Match two or more images taken:

at different timesat different times with different sensorswith different sensors from different view points from different view points with different filterswith different filters

Outline of this MethodologyOutline of this Methodology Point MappingPoint Mapping Fourier MethodsFourier Methods Geometric Transformation (matrices)Geometric Transformation (matrices) Subimage Processing Pixel CorrelationSubimage Processing Pixel Correlation Multispectral Imaging, an example Multispectral Imaging, an example Evaluation MethodsEvaluation Methods Colorimetric Evaluation Colorimetric Evaluation

Point MappingPoint Mapping Currently widely used (benchmark)Currently widely used (benchmark)

Standard technique for registering images Standard technique for registering images misaligned by an misaligned by an unknownunknown transformation transformation

Requires “control points” to be found in the Requires “control points” to be found in the imagesimages Intrinsic or extrinsicIntrinsic or extrinsic

Can be done either manually or Can be done either manually or automaticallyautomatically

Point Mapping (cont)Point Mapping (cont) Mathematically relates the coordinate Mathematically relates the coordinate

systems of the imagessystems of the images

Higher order equations for more Higher order equations for more complicated transforms are possiblecomplicated transforms are possible

y

x

xbybxybxbybby

yaxayxayaxaax

ε

ε

+′+′+′′+′+′+=

+′+′+′′+′+′+=

...

...2

52

43210

25

243210

Fourier MethodsFourier Methods Very robust for images with correlated Very robust for images with correlated

noisenoise Particularly images taken under differing Particularly images taken under differing

illumination conditionsillumination conditions

Good for images that have been rigidly Good for images that have been rigidly misaligned (rotation, translation, etc.)misaligned (rotation, translation, etc.)

Fourier Methods (cont)Fourier Methods (cont) Phase difference in Fourier Transforms of Phase difference in Fourier Transforms of

2 images correlates to a translation (Shift 2 images correlates to a translation (Shift Theorem)Theorem)

Rotation is a shift in polar coordinatesRotation is a shift in polar coordinates

Geometric RegistrationGeometric Registration This picture gives This picture gives

multiple similar image multiple similar image transformations that transformations that we can encounter in we can encounter in registering an imageregistering an image

There are matrices There are matrices that can register tiles that can register tiles with nonuniformitywith nonuniformity

Transform MatricesTransform Matrices

101 a

Shearx =

101

bSheary =

y

x

SS

Scale0

0=

θθθθ

cossinsincos

, −=yxRot

3D Transform matrix3D Transform matrix

Subimage ProcessingSubimage Processing

Using landmark image patches of the full Using landmark image patches of the full image reduces search data size to register image reduces search data size to register images in a shorter time and increased images in a shorter time and increased accuracy.accuracy.

Edge detection algorithms aid programs in Edge detection algorithms aid programs in automatic subimage selection, picking automatic subimage selection, picking clearly discernable features and matching clearly discernable features and matching correlation values.correlation values.

Subimage ExampleSubimage Example Subimage ↓ Squares of approximate correlation

Evaluation MethodsEvaluation Methods CorrelationCorrelation

ColorimetricColorimetric Only applicable if original scene is availableOnly applicable if original scene is available

CorrelationCorrelation Basic statistical approach to registrationBasic statistical approach to registration Measurement of degree of similarity Measurement of degree of similarity

between imagesbetween images Note:Note: By itself, cross correlation is not a By itself, cross correlation is not a

registration methodregistration method Correlation theorem Correlation theorem

Correlation (Cont.)Correlation (Cont.)

€

Covar iance I,T( )σ Iσ T

=T x,y( ) −μT( ) I x − u,y − v( ) −μ I( )

y∑

x∑

I x − u,y − v( ) −μ I( )2

T x,y( ) −μT( )2

y∑

x∑

y∑

x∑

For a template T and image I:

Cross-correlation function:

€

μ: meanσ : STD

€

C u,v( ) =T x,y( )I x − u,y − v( )

y∑

x∑

I2 x − u,y − v( )y

∑x

∑ ⎡

⎣ ⎢ ⎢

⎤

⎦ ⎥ ⎥

A related measure correlation coefficient:

Correlation theoremCorrelation theorem Fourier transform of the correlation of two Fourier transform of the correlation of two

images is the product of the Fourier images is the product of the Fourier transform of one image and the complex transform of one image and the complex conjugate of the Fourier transform of the conjugate of the Fourier transform of the otherother

€

I x,y( ) o T x,y( ) = 1MN

I* m,n( )T x + m,y + n( )

n= 0

N−1

∑m= 0

M −1

∑I* : Complex Conjugate of I

Correlation theorem (Cont.)Correlation theorem (Cont.) The transformation whose cross-correlation is The transformation whose cross-correlation is

the largest specifies how two images optimally the largest specifies how two images optimally registeredregistered

There is a computational cost with increasing the There is a computational cost with increasing the number of transformationsnumber of transformations

So, measures are often computed on features So, measures are often computed on features instead of the whole imageinstead of the whole image

Noisy images must be pre-filtered before cross-Noisy images must be pre-filtered before cross-correlation ( Matched filter technique)correlation ( Matched filter technique)

Color Multispectral Imaging,Color Multispectral Imaging,An exampleAn example

Colorimetric evaluation as Colorimetric evaluation as measurement of accuracymeasurement of accuracy

Registration of the gray imagesRegistration of the gray images Synthesized sRGB image using gray imagesSynthesized sRGB image using gray images In situ spectral reflectance measurement of the In situ spectral reflectance measurement of the

original imageoriginal image Calculate color-difference between the Calculate color-difference between the

synthesized and the original imagesynthesized and the original image Smaller color-difference, better registrationSmaller color-difference, better registration