simultaneous surveillance camera calibration and foot-head homology estimation from human detection...

Simultaneous surveillance camera calibration and foot-head homology estimation from

human detection1

Author : Micusic & Pajdla

Presenter : Shiu, Jia-Hau Advisor : Wang, Sheng-Jyh

1. 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition

Outline

• Introduction• Human Detection• Foot-head homology estimation• Conclusion

Introduction

• This paper uses people to calibrate the camera • Human contour detection (green)• Refined human detection with camera calibration

parameters (blue)• Foot-head homology(o:foot,x:head)

Concept

• Objects are human • Estimate camera parameters by observing a person

standing at several positions

3-D scene 2-D projection Image

System Flow

Sequential Images

Human Detection

Foot-head Homology Estimation

Output

Background

• Shape-based detector(Global search)– Detection rate drop significantly in presence of

occluded humans

• Part-based detector(Local search)

C. Beleznai and H. Bischof. ,“Fast Human Detection in Crowded Scenes by Contour Integration and Local Shape Estimation”, In CVPR,2009.

Background

Left - Shape based : Template matching with head and body

Right - Part based :

Obtain foreground image by background subtraction

Segmentation of detected human

Result : Contour template

Human Detection

• Line edges model a human• Offline: Create around 1000 human contours based

on 3D model and moving and rotating camera

Draw foot-head lines in one image

2-D Image 3-D scene

System Flow

Sequential Images

Human Detection


Output

Background : Camera Model

0

0 c

*[ ]X

*[ ] , u is camera point , X is 3-D point

11

Intrinsic parameters Extrinsic parameters

K= 0 P [ ]

0 0 1

= * = *

c

w

wc

w

x

y

x y

u z K R t

xu

yv z K R t

z

u

v R t

f mx f my

Homography matrix

11 12 13 14

3*4 21 22 23 24

31 32 33 34

11 12 13 14 21 22 23

31 32 33 34

*[ ] *

11 1 1

,

w w w

w w wc

w w w

w w w w w

w w w

x x xu H H H H

y y yv z K R t H H H H H

z z zH H H H

H x H y H z H H x H y H zu v

H x H y H z H

14

31 32 33 34

11 12 13 14 21 22 23 24 31 32 33 34

[ 1 0 0 0 0 - - - -u]* 0

[0 0 0 0 x 1 - - - -v]* 0

, [ ]

w

w w w

w w w w w w

w w w w w w

T

H

H x H y H z H

x y z u x u y u z h

y z v x v y v z h

where h H H H H H H H H H H H H

11 DOF

One pair(2D-3D) of points2 equation

Simple Calibration Example

• Measure 3-D position of special object points in 3-D scene

(0,0,0)

Correspond to camera 2-D point

(0,30,0)

(30,30,40)

(u1,v1)

x

y

z

(u2,v2)


• 1. Camera model : Shifted Homographies• 2. Focal length, Rotation, Translation• 3. Quadratic Eigenvalue Problem(QEP)• 4. Foot-head Homology

Camera model

•

• Extrinsic parameters rotation R and translation t

Camera Parameters

• Assumptions intrinsic parameters – Square pixels – No principal point offset : Image coordinate at center

point (principal point) – No skew : angle of horizon axis and vertical axis = 90’

• Intrinsic parameters K = |f 0 0| |0 f 0| |0 0 1|

x

y

90’

x

y

z

(x1,y1,0)

(x1,y1,z0)

(x2,y2,0)

(x2,y2,z0)

(x3,y3,0)

(x3,y3,z0)

If x1,x2,x3,y1,y2,y3 are knownSix points => 12 equationCompute homography of H

x

y

z

(x1,y1,0)

(x1,y1,z0)

(x2,y2,0)

(x2,y2,z0)

(x3,y3,0)

(x3,y3,z0)

If x1,x2,x3,y1,y2,y3 are unknownHow to find homography of H?

x

y

z

(0,0,0)

(0,0,z0)

(x2,y2,0)

(x2,y2,z0)

(x3,y3,0)

(x3,y3,z0)

(0,0,0) & (0,0,z0) two point are known 4 equation

x1

y1

z

x

y

z

(0,0,0)

(0,0,z0)

(0,0,0)

(0,0,z0)

(x3,y3,0)

(x3,y3,z0)

x1 = x+dx1y1 = y+dy1

x2

y2

z

x1

y1

z

x

y

z

(0,0,0)

(0,0,z0)

(0,0,0)

(0,0,z0)

(0,0,0)

(0,0,z0)



Shifted Homographies

1 2 3 1 2

11 1

points , => 0, 0

3 1 21

1

x dx xu

y dy yv K R t K r r r r dx r dy t

z z

only pick two foot and head x y

uz

v K r r dx r dy t

3+3K unknown, Dof = 3+3K-1

unknown add equationK = 1 6 4K = 2 9 8K = 3 12 12


• The 3D point X = (x, y, z,1) can be simplified assuming x = 0

• ri : is the i-th column of R

• 6+3K unknowns, K : number of detections


•

•

Finding Homographies

• This equation is extended with all the known point correspondences to form this equation:

M contains all the point correspondences h contains h1, h2 and the unknown h3 of the

homographies

[ 1 0 0 0 0 - - - -u]* 0

[0 0 0 0 x 1 - - - -v]* 0w w w w w w

w w w w w w

x y z u x u y u z h

y z v x v y v z h

h is fixed for standard camera calibration

Focal Length 、 Rotation and Translation

• •

• form 3 equations

Where equation unknownK = 1 3 4K = 2 6 6

Minimum Solution : two detectors case

• The equations in (7) give

Six equation with six unknown

Overdetermined Solution

• More than two homologies :solvable as a Quadratic Eigenvalue Problem (QEP)

• Find scalars λ and nonzero vectors x, satisfying (λ2D3 + λD2 + D1)x = 0 • The authors create D1, D2, D3 using the known

values in (7), λ = f.

Overdetermined Solution

• Solve With

• D1, D2, D3 very sparse containing only:

Solving QEP

• One approach to solving the QEP : Convert it to a linear system (remove the f2):

• Solving ( A - f B ) v = 0

Foot-head Homology

• Result of QEP : K, R, t, f • From this construct the homology HFH with

uH H≃ FH*uF

– uH : image points of head

– uF :image points of feet

(x0k,y0

k,0)

(x0k,y0

k,l)

Hk

Hk

uF

uH

HFH

Camera Image 3-D points

Result

Conclusion

• Use 3D-2D point correspondences (model to contour)

• Encode camera parameters that define relation between 3D 2D as a matrix H

• Solve H and get the camera parameters

simultaneous surveillance camera calibration and foot-head homology estimation from human detection...

Documents