image acquisition camera geometry summary
TRANSCRIPT
Image acquisition
Camera geometry
Summary
Prof. Ioannis Pitas
Aristotle University of Thessaloniki
www.aiia.csd.auth.gr
Version 3.0
Image acquisition
โข A still image visualizes a still object or scene, using a still picture camera.
โข A video sequence (moving image) is the visualization of an object or scene
illuminated by a light source, using a video camera.
โข The captured object, the light source and the video camera can all be either
moving or still.
โข Thus, moving images are the projection of moving 3D objects on the camera
image plane, as a function of time.
โข Digital video corresponds to their spatiotemporal sampling.
2
Light reflection
โข Objects reflect or emit light.
โข Reflection can be decomposed in two components:
โข Diffuse reflection (distributes light energy equally along any spatial
direction, allows perceiving object color).
โข Specular reflection (strongest along the direction of the incident light,
incident light color is perceived).
โข Lambertian surfaces perform only diffuse reflection, thus being
dull and matte (e.g., cement surface).
3
Light reflection
5
Light reflection
โข Reflected irradiance when object surface produces diffuse
reflectance and incident light source comes from:
โข Ambient illumination:
๐๐ ๐, ๐, ๐, ๐ก, ๐ = ๐ ๐, ๐, ๐, ๐ก, ๐ โ ๐ธ๐ผ ๐ก, ๐
โข Point light source:
๐๐ ๐, ๐, ๐, ๐ก, ๐ = ๐ ๐, ๐, ๐, ๐ก, ๐ โ ๐ธ๐ ๐ก, ๐ โ cos ๐
โข Distant point source and ambient illumination:
๐ธ ๐ก, ๐ = ๐ธ๐ผ ๐ก, ๐ + ๐ธ๐ ๐ก, ๐ โ cos ๐
6
Camera structure
7
โข The lens is the most important part of the camera.
โข Incident light rays pass through a lens (or a group of lenses) and
get focused on the semiconductor chip.
โข The distance between the lens center (optical center, ๐) and the
point of convergence of the light rays inside the camera (focal
point, ๐ ) is called focal length.
โข Focal length characterizes the lens and determines the scene part
to be captured as well as scene object sizes (magnification).
Camera structure
8
Pinhole Camera and Perspective Projection
13
Pinhole camera geometry.
โข Let us consider 2 coordinate systems:
โข the camera (or standard) coordinate system (๐๐ , ๐๐ , ๐๐ , ๐๐) and
โข the image coordinate system (๐จ, ๐ฅ, ๐ฆ).
โข ๐๐, and ๐๐ define the plane โฑ that is parallel to the camera image
plane ๐, lying at a focal length ๐ behind the optical center ๐๐
along the optical axis ๐๐.
Pinhole Camera and Perspective Projection
14
Pinhole Camera and Perspective Projection
15
โข We want to derive the equations that connect a 3D point (3D
vector) ๐๐ = ๐๐ , ๐๐ , ๐๐๐ referenced in the camera coordinate
system with its projection point (2D vector) ๐ฉโฒ = ๐ฅโฒ, ๐ฆโฒ ๐ on the
virtual image plane.
โข By employing the similarity of triangles ๐๐๐จโฒ๐ฉโฒ and ๐๐๐๐๐๐:
๐ฅโฒ
๐๐=
๐ฆโฒ
๐๐=
๐
๐๐, ๐ฅโฒ = ๐
๐๐
๐๐, ๐ฆโฒ = ๐
๐๐
๐๐
โข Coordinates on the real image plane are given by the same
equations, differing only by a minus sign.
Pinhole Camera and Perspective Projection
18
โข Perspective projection equations are rational, rather than linear.
Thus:
โข straight lines are mapped to straight lines but
โข distances between points and the angles between straight lines are not
preserved after projection.
โข We can linearise them applying two transformations:
โข orthographic projection ๐ฅโฒ = ๐๐ and ๐ฆโฒ = ๐๐ and
โข isotropic scaling ฮค๐ าง๐
The Weak-Perspective Camera Model
19
The Weak-Perspective Camera Model
20
Orthographic projection
โข While a weak-perspective camera preserves parallelism in the
projected lines, as orthographic projection does (b), perspective
projection (a) does not.
The Weak-Perspective Camera Model
22
โข A way to linearize the perspective projection equations is by using
the so-called homogeneous coordinates.
โข A 2D image point is mapped to a 3D point:
๐ฉ = ๐ฅ, ๐ฆ ๐ โ โ2 โถ ๐ฉ๐ป = ๐ฅ, ๐ฆ, 1 ๐ โ โ2.
โข A 3D scene point is mapped to a 4D point:
๐ = ๐, ๐, ๐ ๐ โ โ3 โถ ๐๐ป = ๐, ๐, ๐, 1 ๐ โ โ3.
Central Projection and Homogeneous Coordinates
23
Camera Parameters and Projection Matrix
26
โข Extrinsic camera parameters:
โข Translation vector ๐ โ โ3 (3 degrees of freedom)
โข Orthonormal rotation matrix ๐ โ โ3ร3 (3 degrees of freedom: only 3 of
the 9 rotation matrix entries are independent from each other).
โข The relationship between a point ๐๐ค โ โ3 in world coordinates
and its camera coordinate counterpart ๐๐ โ โ3 is:
๐๐ = ๐ ๐๐ค โ ๐
Camera Parameters and Projection Matrix
27
where:
โข [๐๐ฅ , ๐๐ฆ]๐: the location of the principal camera point ๐จ in pixel
coordinates.
โข ๐ ๐ฅ , ๐ ๐ฆ: the effective pixel sizes in millimeters
โข Coordinate system origin: at the top left corner of the image, not
at the image center.
Camera Parameters and Projection Matrix
30
Camera Parameters and Projection Matrix
31
โข Definition of the 3 ร 4 matrix of extrinsic parameters ๐๐ธ:
๐๐ธ =
๐11 ๐12 ๐13 โ๐1๐๐
๐21 ๐22 ๐23 โ๐2๐๐
๐31 ๐32 ๐33 โ๐3๐๐
โข Definition of the 3 ร 3 matrix of intrinsic parameters ๐๐ผ:
๐๐ผ =
โ ๐๐ ๐ฅ
0 ๐๐ฅ
0 โ ๐๐ ๐ฆ
๐๐ฆ
0 0 1
Camera Parameters and Projection Matrix
33
โข The transformation of a point ๐ โ โ3 to ๐ฉ โ โ2 is given by:
๐๐ฅ๐๐๐ฆ๐๐
= ๐๐ผ๐๐ธ
๐๐ค๐๐ค๐๐ค1
๐ฉ = ๐๐ผ๐๐ธ๐ = ฦค๐
Camera Parameters and Projection Matrix
34
Properties of the Projective Transformation
Vanishing points
Properties of the Projective Transformation
โข The cross-ratio of four collinear
points remains invariant under a
projective transformation:
๐ถ๐ ๐ฉ1, ๐ฉ2, ๐ฉ3, ๐ฉ4 =
๐ถ๐ ๐1, ๐2, ๐3, ๐4 .
โข Chirp effect: the increase in local
image spatial frequency
proportionally to the distance of
the projected scene area from the
camera.
โข It is evident in 2D image regions
where distant and close-up scene
parts are projected.
Properties of the Projective Transformation
46
โข The weak-perspective projection is valid, if the depth variations
amongst the points of a viewed object are small, in comparison
with its average distance from the camera (object depth),
represented by ๐3๐ เดฅ๐ โ ๐ .
โข Another camera-at-infinity model is the affine camera model, with
projection matrix:
ฦค๐๐ =
๐11 ๐12 ๐13 ๐14๐21 ๐22 ๐23 ๐240 0 0 ๐34
Cameras at Infinity
49
โข Camera calibration deals with determining the extrinsic and
intrinsic camera parameters.
โข To this end, a calibration pattern, also called calibration grid is
employed, so that the projection matrix ฦค of the camera can be
computed from a single view by mapping known points ๐๐ค =
๐๐ค, ๐๐ค, ๐๐ค๐ in world coordinates to their projections ๐ฉ = ๐ฅ, ๐ฆ ๐
on the image plane.
Camera Calibration
52
Camera Calibration
54
Calibration patterns.
โข The calibration pattern is a 3D object of common, known
dimensions and positioning, with a checkerboard pattern clearly
visible on each side.
โข Pattern dimensions must be known, in an accuracy much greater
than the desired calibration accuracy.
Camera Calibration
55
โข The most popular calibration techniques utilizing solely a planar
calibration pattern are:
โข Direct camera parameter estimation and
โข Zhangโs calibration method.
โข Other calibration methods do not require a calibration object and
are jointly referenced by the term self-calibration or
autocalibration.
Camera Calibration
56
โข ๐๐ค = ๐๐ค, ๐๐ค , ๐๐ค๐: a known point in world coordinates.
โข ๐๐ = ๐๐ , ๐๐ , ๐๐๐ : the same point in camera coordinates.
โข ๐ฉ๐ = ๐ฅ๐ , ๐ฆ๐๐: its image point in pixel coordinates.
โข The transformation between the world and camera coordinate
systems involves an orthonormal 3 ร 3 rotation matrix ๐ and a
3 ร 1 translation vector ๐ (equivalent to determining the extrinsic
camera parameters).
Direct camera parameter estimation
57
โข Each of the linear homogeneous equations has 8 unknown parameters:
๐ฎ = ๐๐11, ๐๐12, ๐๐13, ๐21, ๐22, ๐23, ๐๐ฅ, ๐๐ฆ๐โ ๐ข1, ๐ข2, โฆ , ๐ข8
๐
โข Expressing the homogeneous system of equation as a product of the matrix ๐
๐ โ
๐ฅ๐1๐๐ค1 ๐ฅ๐1๐๐ค1 ๐ฅ๐1๐๐ค1 ๐ฅ๐1 โ๐ฆ๐1๐๐ค1 โ๐ฆ๐1๐๐ค1 โ๐ฆ๐1๐๐ค1 โ๐ฆ๐1๐ฅ๐2๐๐ค2 ๐ฅ๐2๐๐ค2 ๐ฅ๐2๐๐ค2 ๐ฅ๐2 โ๐ฆ๐2๐๐ค2 โ๐ฆ๐2๐๐ค2 โ๐ฆ๐2๐๐ค2 โ๐ฆ๐2โฎ
๐ฅ๐๐๐๐ค๐
โฎ๐ฅ๐๐๐๐ค๐
โฎ๐ฅ๐๐๐๐ค๐
โฎ๐ฅ๐๐
โฎโ๐ฆ๐๐๐๐ค๐
โฎโ๐ฆ๐๐๐๐ค๐
โฎโ๐ฆ๐๐๐๐ค๐
โฎโ๐ฆ๐๐
and the vector ๐ฎ:
๐๐ฎ = ๐
Direct camera parameter estimation
64
โข Thus, the desired solution is the null space of the matrix ๐.
โข Provided that ๐ โฅ 7 pairs are not coplanar:
โข Matrix ๐ will have rank 7.
โข The system of equations will have one non-trivial solution ๐ฎ โ ๐ ,
obtained via the singular value decomposition (SVD) of matrix ๐:
๐ = ๐๐บ๐๐
โข ๐บ is a diagonal matrix containing the singular values. The solution ๐ฎ is the
column of matrix ๐ corresponding to the zero singular value of ๐ฎ.
Direct camera parameter estimation
65
โข It uses a planar calibration pattern, posing in ๐ different
orientations ๐ โฅ 2 , by moving either the pattern of the camera.
โข Precise knowledge of this motion is not
required.
โข The calibration pattern can simply be
printed on a paper and attached to any
planar surface.
โข It has educed complexity.
Zhangโs calibration method
66
โข These approaches:
โข do not require a calibration object and recover the camera parameters
from image information alone;
โข are flexible but not very robust;
โข exploit properties of the absolute conic of the projective geometry;
โข typically require information equivalent to a partial 3D reconstruction of
the scene.
โข Self-calibration is strongly related to the geometry of multiple
cameras.
Self-calibration
77
Bibliography[PIT2019] I. Pitas, โComputer visionโ, Createspace/Amazon, in press. [SZE2011] R.Szelinski, โ Computer Vision โ , Springer 2011[HAR2003] Hartley R, Zisserman A. , โ Multiple view geometry in computer visionโ .
Cambridge university press; 2003.[DAV2017] Davies, E. Roy. โComputer vision: principles, algorithms, applications,
learning โ. Academic Press, 2017[TRU1998] Trucco E, Verri A. โIntroductory techniques for 3-D computer visionโ, Prentice
Hall, 1998.[PIT2017] I. Pitas, โDigital video processing and analysis โ , China Machine Press, 2017
(in Chinese).[PIT2013] I. Pitas, โDigital Video and Television โ , Createspace/Amazon, 2013.[PIT2000] I. Pitas, Digital Image Processing Algorithms and Applications, J. Wiley, 2000. [NIK2000] N. Nikolaidis and I. Pitas, 3D Image Processing Algorithms, J. Wiley, 2000.
62