static image mosaicing amin charaniya ([email protected]) ee 264: image processing and...
Post on 21-Dec-2015
217 views
TRANSCRIPT
Static Image Mosaicing
Amin Charaniya
EE 264: Image Processing and Reconstruction
Presentation Overview
Problem definition Background
Literature Survey Image transformations
Image Registration Coarse Image registration Transformation Optimization
Image Blending Implementation and Results Conclusions (limitations and enhancements)
The Problem
Q: “Static” ?Ans.: No moving objects in the scene.
+Image 1 Image 2 Mosaiced image
The Solution
Original images
Image Registration /Alignment / Warping Image Blending
Constraints
Scene Static / Dynamic Planar / Non planar (perspective distortion)
Camera Motion Translation (sideways motion) Panning and Tilting (rotation about the Y and X axes) Scaling (zooming, forward / backward motion) General motion
Other Constraints Automated / User input
Background and Literature survey
Barnea & Silverman, 1972 (L1 Norm) Kuglin & Hines, 1975 (Phase Correlation) Mann & Picard, 1994 (Cylindrical projection) Irani & Anandan, 1995 (Static and Dynamic mosaics) Szeliski, 1996 (Transformation optimization) Badra, 1998 (Rotation and Zooming) Peleg and Rousso, 2000 (Adaptive Manifolds, Mosaicing
using strips)
Image transformations
TransformationInputimage
Output
image
w
y
x
w
y
x
876
543
210
'
'
'
mmm
mmm
mmm
100
543
210
mmm
mmm
affineM
Affine transformation
876
543
210
mmm
mmm
mmm
projectiveM
Projective transformation
100
cossin
sincos
y
x
rigid t
t
M
Rigid transformationOriginal
shape
Presentation Overview
Problem definition Background
Literature Survey Image transformations
Image Registration Coarse Image registration Transformation Optimization
Image Blending Implementation and Results Conclusions (limitations and enhancements)
Image Registration
Coarse ImageRegistration
Initial transformation TransformationOptimization
ErrorImproved ?
{Phase Correlation
L1 Norm
User input
Phase Correlation
Kuglin & Hines, 1975 Translation property of Fourier Transform
)(2..00
00),(),( yx yxjyxTFeFyyxxf
1|| 1..1j
TFeFf
2|| 2..2
jTF
eFf
)( 21 jeInverse
transformd(x,y)
maximum
(x0, y0)
Spatial Correlation, L1 Norm
Barnea and Silverman
E(x0,y0) = |f1(x,y) – f2(x- x0, y- y0)|
f1
f2 f2
Spatial correlation techniques User input
Transformation Optimization
Richard Szeliski, “Video Mosaics for Virtual Environments”, 1996. Optimization of initial transformation matrix M, to minimize error. Levenberg-Marquardt non-linear minimization algorithm.
yx
yxfyxfeerror,
221
)),('),(()(minimize
Compute partial derivatives
}7..0{,
km
e
k
bIAm1
)( mMM )()1( tt
Transformation Optimization
Advantages Faster convergence Statistically optimal solution
Limitations Local minimization (need a good initial guess)
Presentation Overview
Problem definition Background
Literature Survey Image transformations
Image Registration Coarse Image registration Transformation Optimization
Image Blending Implementation and Results Conclusions (limitations and enhancements)
Image Blending
Simple averaging Weighted averaging
2/)),('),((),( 21 yxfyxfyxf
),('),(),(),(),( 2211 yxfyxwyxfyxwyxf
Smooth transition (edges, illumination artifacts)
Sample weight function – “hat filter”
0 xmax
2
|2
|1)(
max
max
x
xx
xw
More weight at the center of the image, less at the edges
Image blending
Simple averaging Weighted averaging
Presentation Overview
Problem definition Background
Literature Survey Image transformations
Image Registration Coarse Image registration Transformation Optimization
Image Blending Implementation and Results Conclusions (limitations and enhancements)
Implementation
Implemented using Matlab Source Images
BE 230 lab images (fixed tripod) College 8 images (free hand motion, perpective distortion) East Field House images (free hand motion)
Equipment: Sony DCR-TRV 900 3CCD digital camcorder
Sample results
Sample results
Conclusions/Enhancements
Better automatic coarse registration techniques needed.
Need to handle more general camera motion.
Thanks for listening !!
Questions ?