motion tracking. image processing and computer vision: 82 introduction finding how objects have...

Motion Tracking

Image Processing and Computer Vision: 8 2

Introduction Finding how objects have moved in

an image sequence Movement in space Movement in image plane

Camera options Static camera, moving objects Moving camera, moving objects


Contents Acquiring targets

Image differencing Moving edge detector

Following targets Matching Minimum path curvature Model based methods Kalman filtering Condensation Hidden Markov Model


Camera calibration revisited Image to camera co-ordinate

transformation Intrinsic parameters

Camera to world co-ordinate transformation Extrinsic parameters


Camera to Image Co-ordinates Distortionless Camera

If no distortions uniform sampling

Co-ordinates linearly related offset and scale

s yo yyimy

sxoxximx


Camera to Image Co-ordinates Distorting Camera

Periphery is distorted

k2 = 0 is good enough

yxr

rkrkydy

rkrkxdx

222

42

211

42

211


Pinhole Camera

Image

ObjectOpticalcentre

Image and centre, object and centre are similar triangles.

f Z

Z

Yfy

Z

Xfx


World frame

Camera frame

translate androtate

Camera and World Co-ordinates


System architecture

Acquiretarget

Followtarget

Start End


Target acquisition Finding a target to follow

Differencing Moving edge detector


Change and Moving Object Detection Simplest method of detecting

change Compute differences between

Live and background images Adjacent images in a sequence


Image Differencing Differences due to

Moving object overlying static background

Moving object overlying another moving object

Moving object overlying same moving object

Random fluctuation of image data


Demo Inter frame differencing Difference from a background

pFinder


Background image Detecting true differences required

an accurate background Lighting changes? Camera movement?


Background image updates Periodically modify whole

background Will include changes in new

background Systematically incorporate non-

changed portions of image into background BBelse

LBBthenbackgroundLif

titi

tititi ti

1,,

,1,, ,


Critique Can identify changes in the image

data But what do the changes mean?

Need a second layer of processing To recognize changes Optical flow sidesteps this problem...


Target following Observing the positions of an

object or objects in a time sequence of images.

Object matching Minimum path curvature Model based methods


Matching Locate objects in each image Match objects between images Use methods of previous lectures


Minimum path curvature Observations of two objects in three

images


Which is “best” solution? One with overall straightest paths

For each solutionFor each path

Compute total curvature

Sum


Model based tracking Mathematical model of objects’

motions: position, velocity (speed, direction),

acceleration Can predict objects’ positions


System overview

MotionModel

VerifyLocation

PredictLocation

Update?


Newton’s laws

s = position u = velocity a = acceleration

all vector quantities measured in image co-ordinates

tautsts 2

0

21

Simple Motion Model


Prediction Can predict position at time t

knowing Position Velocity Acceleration

At t=0


Uncertainty If some error in a - a or u - u Then error in predicted position -

s

tautsts 20 2

1


Verification Is the object at the predicted

location? Matching

How to decide if object is found Search area

Where to look for object


Object Matching Compare

A small bitmap derived from the object vs.

Small regions of the image Matching?

Measure differences


Search Area: Why? and Where? Uncertainty in knowledge of model

parameters Limited accuracy of measurement Values might change between

measurements Define an area in which object

could be Centred on predicted location, s s


Update the Model? Is the object at the predicted

location? Yes

No change to model No

Model needs updating Kalman filter is a solution


Kalman filter

Mathematically rigorous methods of using uncertain measurements to update a model


Kalman filter notation Relates

Measurements y[k] e.g. positions

System state x[k] Position, velocity of object, etc

Observation matrix H[k] Relates system state to measurements

Evolution matrix A[k] Relates state of system between epochs

Measurement noise n[k] Process noise v[k]


Mathematically

k

ky

ky

kx

kx

ky

kx

kkkk

n

nxHy

0

0

1

0

0

0

0

1

How do observations relate to model?


Prediction of System State Relates system states at epochs k

and k+1

k

kky

kky

kkx

kkx

T

T

kky

kky

kkx

kkx

kkkkkk

v

vxAx

|

|

|

|

1000

100

0010

001

|1ˆ|1ˆ

|1ˆ|1ˆ

||1ˆ


Prediction of Observation From predicted system state,

estimate what observation should occur:

k

kky

kky

kkx

kkx

kky

kkx

kkkkkk

n

nxHy

|1ˆ|1ˆ

|1ˆ|1ˆ

0

0

1

0

0

0

0

1

|1ˆ

|1ˆ

|1ˆ|1ˆ


Updating the Model Predict/estimate a

measurement Make a measurement Predict state of model How does the new

measurement contribute to updating the model?

kk |1ˆ y

1ky kk |1ˆ x


Updating the Model

G is Kalman Gain Derived from A, H, v, n.

kkkkk

kkkkkkk

|1ˆ1|1

|11|1ˆ1|1ˆ

yyy

yGxx

covariancesystem

|||1

|1

|

C

GHCCC

nHHCHCG

kkkkkk

TkkTkk


Example Tracking two corners of a minimum

bounding box Matching using colour Image differencing to locate target


Condensation Tracking So far considered single motions What if movements change?

Bouncing ball Human movements

Use multiple models plus a model selection mechanism


Selection and Tracking

Occur simultaneously Maintain

A distribution of likely object positions plus weights

Predict Select N samples, predict locations

Verify Match predicted locations against image Update distributions


Tracking Using Hidden Markov Models Hidden Markov model describes

States occupied by a system Possible transitions between states Probabilities of transitions How to recognise a transition


Optic Flow Assume each pixel moves but does

not change intensity Pixel at location (x,y) in image 1 is

pixel at (x+x,y+y) in image 2. Optic flow associates displacement

vector with each pixel


t

Iu

y

Iv

x

I

t

I

t

y

y

I

t

x

x

I

tt

Iy

y

Ix

x

I

Ott

Iy

y

Ix

x

ItyxI

ttyyxxItyxI

x

0

0

0

,,

,,,,

2


Aperture Problem

I/ x horizontal difference I/ y vertical difference I/ t difference between imagesOne equation, two unknowns cannot solve equationCould solve for movement

perpendicular to gradient


Solution

Impose smoothness constraint:

Minimise total of sum of squares of velocity component gradients

image y

v

x

vy

u

x

u 2222


EED

EvEuEPD

PEvv

D

PEuu

yx

taverageyaveragex

yaverage

xaverage

222

is a constantIterate over a pair of frames, orover a sequence


Critique Assumptions

Linear variation of intensities Velocity changes smoothly

These are invalid Especially at object boundaries


Area Based Methods Match small regions in image 1

with small regions in image 2 Assume objects move but do not

deform Same formulation as for optical flow,

different areas involved….

tyxIttyyxxI ,,,,


Matching

Compute (x, y) by finding (u,v) that minimises

Then (u,v) = (x, y)

wi

wi

wj

wjtjyixIttjvyiuxIvuD ,,,,2

,


Summary Target acquisition

Image differencing Background model

Target following Matching Minimum path curvature Model based methods Optic flow


This “telephone” has too many shortcomings to be seriously considered as a means of communication. The device is of no value to us.Western Union internal memo, 1876

motion tracking. image processing and computer vision: 82 introduction finding how objects have...

Documents