Motion Analysis using Optical flow

CIS750 PresentationStudent: Wan WangProf: Longin Jan Latecki

Spring 2003CIS Dept of Temple

Contents

Brief discussion to Motion AnalysisBrief discussion to Motion Analysis Introduction to optical flowIntroduction to optical flow Application of Detect and tracking Application of Detect and tracking

people in complex scenes using people in complex scenes using optical flowoptical flow

Usual input of a motion analysis Usual input of a motion analysis system is a temporal image system is a temporal image sequencesequence

Motion analysis is often connected Motion analysis is often connected with real-time analysiswith real-time analysis

Part 1: Motion Analysis

Three main groups of motion analysis problem

• Motion detection: - register any detected motion - single static camera

• Moving object detection and location:

- moving object detection only : motion_based segmentation methods

- detection of a moving object, detection of the trajectory of its motion, prediction of its future trajectory: image object_matching techniques are often used to solve these tasks (direct matching of image data, matching of object features, specific representative object points (corner etc.),represent moving object as graphs and mathing these graphs); another useful method is optical flow

• Derivation of 3D object propertiesDerivation of 3D object properties:: from a set of 2D from a set of 2D projections of acquired at different time instants of object motionprojections of acquired at different time instants of object motion

Reflects the image changes due to motion during a time interval Reflects the image changes due to motion during a time interval dt, dt, which is short enough to guarntees small inter-frame motion which is short enough to guarntees small inter-frame motion changeschanges

The immediate objective of optical flow is to determine a The immediate objective of optical flow is to determine a Velocity Velocity field:field:A 2D representation of a (generally) 3D motion is called a A 2D representation of a (generally) 3D motion is called a motion field(velocity field) Whereas each point is assigned motion field(velocity field) Whereas each point is assigned avelocity vector corresponding the motion direction, velocity and avelocity vector corresponding the motion direction, velocity and distance from an observer at an appropriate image locationdistance from an observer at an appropriate image location

Based on 2 assumptions:Based on 2 assumptions:

- The observed brightness of any object point is constant over - The observed brightness of any object point is constant over timetime

- Nearby points in the image plane move in a similar - Nearby points in the image plane move in a similar manner(velocity smoothness constraint)manner(velocity smoothness constraint)

Part2: Optical flow

Optical flow

Eg:http://www.ai.mit.edu/people/lpk/mars/temizer_2001/Optical_Flow/index.html

• Let us suppose we have a continuous image, the image intensity is given by f(x,y,t), where the intensity is now a function of time t, as well as of x and y.

• If this point(x,y) moves to a point (x+dx,y+dy) at time t+dt, the following equation holds:

 

• Taylor expansion of the right side of the equation (1) is

Where fx(x,y,t),fy(x,y,t),ft(x,y,t) denote the partial derivation of f.

And e is the high-order term in Tylor series.

Computation Rationale

Computation Rationale

Assuming that e is negligible, we obtain the next equation:

 

That means

Computation Method

Optical flow in motion analysis

Motion, as it appears in dynamic images, is usually some Motion, as it appears in dynamic images, is usually some combination of 4 basic elements:combination of 4 basic elements:

(a)Translation at constant distance from the observer.(a)Translation at constant distance from the observer.

---parallel motion vectors---parallel motion vectors

(b)Translation in depth relative to the observer.(b)Translation in depth relative to the observer.

---Vectors having common focus of expansion.---Vectors having common focus of expansion.

(c) Rotation at constant distance from view axis.(c) Rotation at constant distance from view axis.

------concentric motion vectors.concentric motion vectors.

(d) Rotation of planar object perpendicular to the view axis.(d) Rotation of planar object perpendicular to the view axis.

---- ---- vectors starting from straight line segments.vectors starting from straight line segments.

Optical flow in motion analysis

• Mutual velocity of an observer and an object

Let mutual velocities be (u,v,w) at direction x,y,z.(z represent the depth) if (x0,y0,z0) is the position at time t0=0.then the position of the same point at time t can be determined as:

• FOE (focus of expansion) determination:

• Distance(depth) determination

• Collision Prediction

Part 3

Experiment of detecting and tracking people in complex scenes using optical flow (by saitama univ)

Demand

• Automatic visual surveillance systems are strongly demanded for various applications. We have several systems commercially available, most of which are based on subtraction between consecutive frames or that between a current image and a stored background image. They can work as expected if environmental conditions do not change, such as indoors.

• However, they cannot work outdoors because there are various disturbances such as changes of lighting and movements of background objects.

By applying two different spatial filters By applying two different spatial filters gg,,hh to the input image , to the input image , the following two constraint equations are derived. the following two constraint equations are derived.

    Two orientation_selective spatial Gaussian filters g, h applied to Two orientation_selective spatial Gaussian filters g, h applied to the original image f(x,y,t): one is sensitive to vertical edges, one is the original image f(x,y,t): one is sensitive to vertical edges, one is to horizental edges.to horizental edges.

(u,v) denotes an optical flow vector and subscript denotes (u,v) denotes an optical flow vector and subscript denotes partial differentiationpartial differentiation

First step: compute the optical flow

segment the flow image into uniform flow regions in a split-segment the flow image into uniform flow regions in a split-and-merge fashion. First, we divide the image into 16 (4 X and-merge fashion. First, we divide the image into 16 (4 X 4) regions, calculating the mean flow vector in each region. 4) regions, calculating the mean flow vector in each region. If the region has any outlier subregions whose flow vectors If the region has any outlier subregions whose flow vectors are different from the mean, the region is further split into 4 are different from the mean, the region is further split into 4 (2 X 2) regions. If the region has no outlier subregion, that (2 X 2) regions. If the region has no outlier subregion, that is, the region has a uniform flow, it will not be split. The is, the region has a uniform flow, it will not be split. The above process is repeated to each region until it becomes above process is repeated to each region until it becomes too small to be splittoo small to be split

Second step: Region Segmentation

We prepare a four-dimensional voting space ( )For We prepare a four-dimensional voting space ( )For each uniform flow region detected in the previous process, each uniform flow region detected in the previous process, we predict a path of the region in a certain time interval of we predict a path of the region in a certain time interval of future. Fig. shows the predicted path(only future. Fig. shows the predicted path(only x-y-tx-y-t are shown). are shown). We assume that the region continues to move in the We assume that the region continues to move in the direction of the mean flow vector ( u,v ) at its speed. We direction of the mean flow vector ( u,v ) at its speed. We approximate each region by an ellipse whose center approximate each region by an ellipse whose center coincides with the region centroid. Every point inside the coincides with the region centroid. Every point inside the ellipse is given weight, according to the two dimensional ellipse is given weight, according to the two dimensional Gaussian as shown in Fig. 3(a). This weight is voted at the Gaussian as shown in Fig. 3(a). This weight is voted at the predicted position (predicted position (x,yx,y) at the time () at the time (tt) in the direction ( ). ) in the direction ( ).

The voted result is compared with a threshold. If there is any The voted result is compared with a threshold. If there is any region whose number of votes is over the threshold, the region whose number of votes is over the threshold, the region is detected as a target. region is detected as a target.

Third step: Predicted Path Voting

ReferenceReference

Image processing, analysis, and Image processing, analysis, and machine visionmachine vision

Detecting and tracking people in complex scenes

http://www-cv.mech.eng.osaka-u.ac.jp/research/tracking_group/iketani/research_e/node1.html