motion from normal flow
DESCRIPTION
Motion from normal flow. The aperture problem. Depth discontinuities. Optical flow difficulties. Translational Normal Flow. In the case of translation each normal flow vector constrains the location of the FOE to a half-plane. Intersection of half-planes provides FOE. - PowerPoint PPT PresentationTRANSCRIPT
Motion from normal flow
Optical flow difficulties
• The aperture problem • Depth discontinuities
Translational Normal Flow
• In the case of translation each normal flow vector constrains the location of the FOE to ahalf-plane.
• Intersection of half-planes provides FOE.
nu
u Zn
tr
Egoestimation from normal flow
• Idea: choose particular directions: patterns defined on the sign of normal flow along particular orientation fields
• positive depth constraint
• 2 classes of orientation fields: copoint vectors and coaxis vectors
Optical flow and normal flow
Optical flow and normal flow
Coaxis vectorswith respect to axis (A,B,C)
Coaxis vectors
Translational coaxis vectors
Translational coaxis vectors
h passes through FOE and (Af/C, Bf/C), defined by 2 parameters
Rotational coaxis vectors
Rotational coaxis vectors
Rotational coaxis vectors
g passes through AOR and (Af/C, Bf/C), defined by 1 parameter
Combine translation and rotation
Positive + positive positive
Negative + negative negative
Positive + negative don’t know (depends on structure)
Coaxis patterntranslational
rotational
combined
-vectors: Translation
-vectors: Rotation
cossin22
rot
f
r
f
ru
-vectors: Translation and Rotation
UW
VW
alpha beta gamma
Three coaxis vector fields
Copoint vectors
O
copoint vectors
Copoint vectors
defined by point (r,s)
Translational copoint vectors
FOE
AOR
FOE
AOR
FOE
AOR
: Negative
: Positive
: Don't know
FOE
AOR
Translational copoint vectors
k passes through FOE and (r,s) defined by 1 parameter
(r, s)
FOE
Rotational copoint vectors
Rotational copoint vectors
l passes through AOR and (r,s), is defined by 2 parameters
(r, s)
AOR
(r, s)
FOE
(r, s)
AOR
FOE
(r, s)
AOR
translational component rotational component
(a) (b) (c)
Three coaxis vector fields
a,b,c : positive and negative vectors
c,d,e: Fitting of patterns
g: Separation of (coaxis patternh: Separation of (x0, y0) copoint pattern
FOE
AOR
FOE
AOR
FOE
AOR
: Negative
: Positive
: Don't know
FOE
AOR
Opticalillusion
What is the Problem?
• Flow can be accurately estimated in an image patch corresponding to a smooth scene patch,
• But erroneous flow estimates are obtained for image patches corresponding to scene patches containing discontinuities
Image Flow
3D MotionScene structureDiscontinuities
Depth variability constraint
• Errors in motion estimates lead to distortion of the scene estimates.
• The distortion is such that the correct motion gives the “smoothest” (least varying) scene structure.
• Scene depth can be estimated from normal flow measurements:
ntu
nu
)(ˆ
ωˆ1
tr
rotnu
Z
Depth estimation
nununu rottrn
1
Zu
Visual Space Distortion
• Wrong 3D motion gives rise to a rugged (unsmooth) depth function (surface).
• The correct 3D motion leads to the “smoothest” estimated depth.
nutu
ntu
ωδ
ˆ ,ˆ
rottr
trDDZZ
Inverse depth estimates
correct motion
incorrect motion
The error function
• A normal flow measurement:
For an estimate
• The error function to be minimized:
nunu rottrn
1
Zu
2
))ˆ()(ˆ1
()ˆ(
ii
ii n
ZnuW tuωu trrotin
nωuntu )ˆ()ˆ(1
rottrn Zu
ωt ˆ,ˆ
The error function• Estimated normal flow
• The error function to be minimized:
• Global parameters:• Local parameter: locally planar patches:
nωuntu )ˆ()ˆ(ˆ1
ˆ rottrnZ
u
2
nnˆ R i
i uuW
ωt ˆ,ˆZ
cbyaxZ
ˆ1
Error function evaluation
• Given a translation candidate , each local depth can be computed as a linear function of the rotation .
• We obtain a second order function of the rotation; its minimization provides both the rotation and the value of the error function.
t
ω
Is derived from image gradients only
Brightness consistency:
Flow:
Planar patch:
Handling depth discontinuities
• Given a candidate motion, the scene depth can be estimated and further processed to find depth discontinuities.
• Split a region if it corresponds to two depth values separated in space.
The algorithm
• Compute spatio-temporal image derivatives and normal flow.
• Find the direction of translation that minimizes the depth-variability criterion.– Hierarchical search of the 2D space.– Iterative minimization.– Utilize continuity of the solution in time;
usually the motion changes slowly over time.
Divide image into small patches
Search in the 2D space of translations
For each candidate 3D motion, using normal flow
measurements in each patch, compute depth of
the scene.
For each image patch
NO
Depth variation small?
Distinguish between two cases
wrong 3D motionexistence of a
discontinuity at the patch
Use the error Split the patch and repeat the
process
Use the error
YES
The Algorithm
3D reconstruction
• Comparison of the original sequence and the re-projection of the 3D reconstruction.
3D model construction