1 motion analysis using optical flow cis601 longin jan latecki fall 2003 cis dept of temple...
TRANSCRIPT
11
Motion Analysis using Optical flow
CIS601
Longin Jan Latecki
Fall 2003CIS Dept of Temple University
22
Usual input of a motion analysis Usual input of a motion analysis system is a temporal image system is a temporal image sequencesequence
Motion analysis is often connected Motion analysis is often connected with real-time analysiswith real-time analysis
Part 1: Motion Analysis
33
Three main groups of motion analysis problem
• Motion detection: - register any detected motion - single static camera
• Moving object detection and location:
- moving object detection only : motion based segmentation methods
- detection of a moving object, detection of the trajectory of its motion, prediction of its future trajectory: image object matching techniques are often used to solve these tasks (direct matching of image data, matching of object features, specific representative object points (corner etc.),represent moving object as graphs and matching these graphs); another useful method is optical flow
• Derivation of 3D object propertiesDerivation of 3D object properties:: from a set of 2D from a set of 2D projections of acquired at different time instants of object motionprojections of acquired at different time instants of object motion
44
Reflects the image changes due to motion during a time interval Reflects the image changes due to motion during a time interval dt, dt, which is short enough to guarantee small inter-frame motion which is short enough to guarantee small inter-frame motion changeschanges
The immediate objective of optical flow is to determine a The immediate objective of optical flow is to determine a velocity velocity field:field:A 2D representation of a (generally) 3D motion is called a motion A 2D representation of a (generally) 3D motion is called a motion field (velocity field). Whereas each point is assigned a velocity field (velocity field). Whereas each point is assigned a velocity vector corresponding the motion direction, velocity and distance vector corresponding the motion direction, velocity and distance from an observer at an appropriate image locationfrom an observer at an appropriate image location
Based on 2 assumptions:Based on 2 assumptions:
- The observed brightness of any object point is constant over - The observed brightness of any object point is constant over timetime
- Nearby points in the image plane move in a similar - Nearby points in the image plane move in a similar manner(velocity smoothness constraint)manner(velocity smoothness constraint)
Optical flow
55
Optical flow
Motion field is a projection of ‘real’ motion vectors of 3D objects to the image plane.We can only compute an optical flow from time-varying brightnessin the image sequence, which is an approximation of the motion field.
66The program can be obtained here.
77
• Let us suppose we have a continuous image, the image intensity is given by I(x,y,t), where the intensity is now a function of time t, as well as of x and y.
• If this point (x,y) moves to a point (x+dx,y+dy) at time t+dt, the following equation holds, which we transform by Taylor expansion:
Computation Rationale for Optical Flow
0),,(),,(),,(
0
),,(),,(),,(),,(),,(
),,(),,(
dttyxIdytyxIdxtyxI
error
errordttyxIdytyxIdxtyxItyxItyxI
dttdyydxxItyxI
tyx
tyx
88
tT
tT
Idt
dy
dt
dxI
Idt
dy
dt
dxI
t
I
dt
dy
y
I
dt
dx
x
Idt
ttytxdIdt
dI
],[)(
0],[)(
0
0)),(),((
0
Thus, the displacement vector [dx/dt, dy/dt] at pixel (x,y) at frame tis the one for which the spatial derivative of the image intensity Iat point (x,y) is equal to the minus of the temporal derivative of I at (x,y).
99
The aperture problemThe aperture problem
tT I
dt
dy
dt
dxI ],[)(
Only a component of the motion field in the direction of the spatial image gradientcan be computed using this equation.
This component is called the normal component, because the spatial image gradientis normal to the spatial direction along which the image intensity remains constant.See this link for a nice demo movie.
1010
t
I
dt
dy
dt
dx
y
I
x
IT
,,
To compute the displacement vector [dx/dt, dy/dt] at pixel (x,y) at frame twe need at least two equations, but so far we have only one.
We will consider two methods to obtain additional equations.The first method works for color videos, where we have red R, green G, and blue color intensity values:
t
B
dt
dy
dt
dx
y
B
x
B
t
G
dt
dy
dt
dx
y
G
x
G
t
R
dt
dy
dt
dx
y
R
x
R
T
T
T
,,
,,
,,Since this is an
overdetermined system of equations, we have two ways to solve it:
1. Use least squares. 2. Disregard, say blue
color, and use Gauss elimination. See an example Matlab program.
1111
The second method ( from E. Trucco and A. Verri: Introductory Techniques for 3-D Computer Vision)
Based on 2 assumptions:Based on 2 assumptions:
1.1. The observed brightness of any object point is constant over The observed brightness of any object point is constant over timetime
2.2. Nearby points in the image plane move in a similar manner Nearby points in the image plane move in a similar manner (velocity smoothness constraint) implies a constant motion (velocity smoothness constraint) implies a constant motion vector field within vector field within a small patch of the image planea small patch of the image plane
1212
Computation Method
1313
Computation Method
Homework (optional):Implement this algorithm and test on a synthetic and real image sequences.
1414
Optical flow in motion analysis
Motion, as it appears in dynamic images, is usually some Motion, as it appears in dynamic images, is usually some combination of 4 basic elements:combination of 4 basic elements:
(a)Translation at constant distance from the observer.(a)Translation at constant distance from the observer.
---parallel motion vectors---parallel motion vectors
(b)Translation in depth relative to the observer.(b)Translation in depth relative to the observer.
---Vectors having common focus of expansion.---Vectors having common focus of expansion.
(c) Rotation at constant distance from view axis.(c) Rotation at constant distance from view axis.
------concentric motion vectors.concentric motion vectors.
(d) Rotation of planar object perpendicular to the view axis.(d) Rotation of planar object perpendicular to the view axis.
---- ---- vectors starting from straight line segments.vectors starting from straight line segments.