image formation & geometrical transforms francisco gómez j mms u. central y ujtl
TRANSCRIPT
![Page 1: Image formation & Geometrical Transforms Francisco Gómez J MMS U. Central y UJTL](https://reader035.vdocuments.us/reader035/viewer/2022062308/56649dac5503460f94a9c376/html5/thumbnails/1.jpg)
Image formation& Geometrical Transforms
Francisco Gómez JMMS
U. Central y UJTL
![Page 2: Image formation & Geometrical Transforms Francisco Gómez J MMS U. Central y UJTL](https://reader035.vdocuments.us/reader035/viewer/2022062308/56649dac5503460f94a9c376/html5/thumbnails/2.jpg)
Digital images
• Arrays of numbers
• Numbers can represent:– Intensity (gray level)– Range– X-ray absortion
coefficient– Amount of light
![Page 3: Image formation & Geometrical Transforms Francisco Gómez J MMS U. Central y UJTL](https://reader035.vdocuments.us/reader035/viewer/2022062308/56649dac5503460f94a9c376/html5/thumbnails/3.jpg)
Human eye
www.youreyescenter.com
Color sensors
Intensitysensors
100,0000,000 100,0000,000
![Page 4: Image formation & Geometrical Transforms Francisco Gómez J MMS U. Central y UJTL](https://reader035.vdocuments.us/reader035/viewer/2022062308/56649dac5503460f94a9c376/html5/thumbnails/4.jpg)
Distribution of cones and rods
![Page 5: Image formation & Geometrical Transforms Francisco Gómez J MMS U. Central y UJTL](https://reader035.vdocuments.us/reader035/viewer/2022062308/56649dac5503460f94a9c376/html5/thumbnails/5.jpg)
Sensor simplified
Pupil Limit amount of light and incidence angle
Lens Focus the light to a single image point
Fotosensitive surface Film
![Page 6: Image formation & Geometrical Transforms Francisco Gómez J MMS U. Central y UJTL](https://reader035.vdocuments.us/reader035/viewer/2022062308/56649dac5503460f94a9c376/html5/thumbnails/6.jpg)
Digital cameras – sensor type
![Page 7: Image formation & Geometrical Transforms Francisco Gómez J MMS U. Central y UJTL](https://reader035.vdocuments.us/reader035/viewer/2022062308/56649dac5503460f94a9c376/html5/thumbnails/7.jpg)
Sensor type
CCD type (Charge coupled device)Charge is accumulated during exposure
CMOS type (complementary metal oxide on silicon)Light affects the conductivity (or gain) of each photodetector
www.parentesis.com
Pixel!
To take into account: Number and size of sensor elements – Chip size – ADC resolution
![Page 8: Image formation & Geometrical Transforms Francisco Gómez J MMS U. Central y UJTL](https://reader035.vdocuments.us/reader035/viewer/2022062308/56649dac5503460f94a9c376/html5/thumbnails/8.jpg)
Pinhole camera
The pinhole camera and camera obscura principle illustrated in 1925, in The Boy Scientist.
![Page 9: Image formation & Geometrical Transforms Francisco Gómez J MMS U. Central y UJTL](https://reader035.vdocuments.us/reader035/viewer/2022062308/56649dac5503460f94a9c376/html5/thumbnails/9.jpg)
Pinhole camera
The «Camera obscura» was used by renaissance paienters to help to understandPerspective projection
![Page 10: Image formation & Geometrical Transforms Francisco Gómez J MMS U. Central y UJTL](https://reader035.vdocuments.us/reader035/viewer/2022062308/56649dac5503460f94a9c376/html5/thumbnails/10.jpg)
Pinhole and lens model
https://foundationsofvision.stanford.edu/appendix/
Pinhole
Given a projection plane parallel to X-Y located at distance f where the point (p1,p2,p3) is going to be projected, i.e, which is the value for (u,v)?
u=(p1/p3)f and v=(p2/p3)f
![Page 11: Image formation & Geometrical Transforms Francisco Gómez J MMS U. Central y UJTL](https://reader035.vdocuments.us/reader035/viewer/2022062308/56649dac5503460f94a9c376/html5/thumbnails/11.jpg)
Pinhole and lens model
This is the CCD sensor units in mmImage plane coordinate
This is the real object in m
![Page 12: Image formation & Geometrical Transforms Francisco Gómez J MMS U. Central y UJTL](https://reader035.vdocuments.us/reader035/viewer/2022062308/56649dac5503460f94a9c376/html5/thumbnails/12.jpg)
Example
• Camera– Focal length: 5mm
• You have an scene point located at (1m,2m,5m)– Where at the image plane coordinate this point is going to be
located?– If the image plane is 10mm,10mm which is the FoV?– A building is 100m wide. How far away do we have to be in order
that it fills the field of view?
(0,0)
Optical center (cx,cy)Optical center
![Page 13: Image formation & Geometrical Transforms Francisco Gómez J MMS U. Central y UJTL](https://reader035.vdocuments.us/reader035/viewer/2022062308/56649dac5503460f94a9c376/html5/thumbnails/13.jpg)
Image buffer
Optical center
digital image
Pixel
Image Origin
![Page 14: Image formation & Geometrical Transforms Francisco Gómez J MMS U. Central y UJTL](https://reader035.vdocuments.us/reader035/viewer/2022062308/56649dac5503460f94a9c376/html5/thumbnails/14.jpg)
Image Buffer
![Page 15: Image formation & Geometrical Transforms Francisco Gómez J MMS U. Central y UJTL](https://reader035.vdocuments.us/reader035/viewer/2022062308/56649dac5503460f94a9c376/html5/thumbnails/15.jpg)
Example
• A camera observes a rectangle 2m away– The rectangle is known to be 50 cm x 30 cm– If the image in the rectangle measures 60 x 15
pixels• Where is located the focal length in pixels?
![Page 16: Image formation & Geometrical Transforms Francisco Gómez J MMS U. Central y UJTL](https://reader035.vdocuments.us/reader035/viewer/2022062308/56649dac5503460f94a9c376/html5/thumbnails/16.jpg)
Other kind of sensors (Color camera)
www.eoshd.com
![Page 17: Image formation & Geometrical Transforms Francisco Gómez J MMS U. Central y UJTL](https://reader035.vdocuments.us/reader035/viewer/2022062308/56649dac5503460f94a9c376/html5/thumbnails/17.jpg)
Other kind of sensors (Kinect)
http://123kinect.com/kinect-forums/thread-569.html
![Page 18: Image formation & Geometrical Transforms Francisco Gómez J MMS U. Central y UJTL](https://reader035.vdocuments.us/reader035/viewer/2022062308/56649dac5503460f94a9c376/html5/thumbnails/18.jpg)
Other kind of sensors
http://mesh.brown.edu/3dpgp-2009/homework/hw2/figures/teaser.jpg
![Page 19: Image formation & Geometrical Transforms Francisco Gómez J MMS U. Central y UJTL](https://reader035.vdocuments.us/reader035/viewer/2022062308/56649dac5503460f94a9c376/html5/thumbnails/19.jpg)
Other kind of sensors
http://images.anandtech.com/doci/4057/StructuredLightSystem.png
![Page 20: Image formation & Geometrical Transforms Francisco Gómez J MMS U. Central y UJTL](https://reader035.vdocuments.us/reader035/viewer/2022062308/56649dac5503460f94a9c376/html5/thumbnails/20.jpg)
Other kind of sensors (MRI)
http://www.howequipmentworks.com/physics/medical_imaging/mri/magnetic_resonance_imaging.html
![Page 21: Image formation & Geometrical Transforms Francisco Gómez J MMS U. Central y UJTL](https://reader035.vdocuments.us/reader035/viewer/2022062308/56649dac5503460f94a9c376/html5/thumbnails/21.jpg)
Other kind of sensors (MRI)
http://www.howequipmentworks.com/physics/medical_imaging/mri/magnetic_resonance_imaging.html
![Page 22: Image formation & Geometrical Transforms Francisco Gómez J MMS U. Central y UJTL](https://reader035.vdocuments.us/reader035/viewer/2022062308/56649dac5503460f94a9c376/html5/thumbnails/22.jpg)
Other kind of sensors (MRI)
https://electrosome.com/light-field-camera/
![Page 23: Image formation & Geometrical Transforms Francisco Gómez J MMS U. Central y UJTL](https://reader035.vdocuments.us/reader035/viewer/2022062308/56649dac5503460f94a9c376/html5/thumbnails/23.jpg)
Other kind of sensors
![Page 24: Image formation & Geometrical Transforms Francisco Gómez J MMS U. Central y UJTL](https://reader035.vdocuments.us/reader035/viewer/2022062308/56649dac5503460f94a9c376/html5/thumbnails/24.jpg)
All is about tranform between frames
![Page 25: Image formation & Geometrical Transforms Francisco Gómez J MMS U. Central y UJTL](https://reader035.vdocuments.us/reader035/viewer/2022062308/56649dac5503460f94a9c376/html5/thumbnails/25.jpg)
2D to 2d Transforms (Rigid)
(1,1) (?,?)
2
2
Dx
(x’,y’)(?,?)
Dy q
Preserves shape and sizeNumber of degrees of freedom (3): t=(Dx,Dy) and q
![Page 26: Image formation & Geometrical Transforms Francisco Gómez J MMS U. Central y UJTL](https://reader035.vdocuments.us/reader035/viewer/2022062308/56649dac5503460f94a9c376/html5/thumbnails/26.jpg)
2D to 2d Transforms (Euclidean - Rigid)
q
R is orthonormalTranspose is the inverseHow to invert the transform?
![Page 27: Image formation & Geometrical Transforms Francisco Gómez J MMS U. Central y UJTL](https://reader035.vdocuments.us/reader035/viewer/2022062308/56649dac5503460f94a9c376/html5/thumbnails/27.jpg)
Homogeneous coordinates
- Homogeneous coordinates simply add an extra element 1- If during operations the third element is different of 1 divide by this number- This representation is quite convenient to represent transformations
to
![Page 28: Image formation & Geometrical Transforms Francisco Gómez J MMS U. Central y UJTL](https://reader035.vdocuments.us/reader035/viewer/2022062308/56649dac5503460f94a9c376/html5/thumbnails/28.jpg)
Homogeneous coordinates
![Page 29: Image formation & Geometrical Transforms Francisco Gómez J MMS U. Central y UJTL](https://reader035.vdocuments.us/reader035/viewer/2022062308/56649dac5503460f94a9c376/html5/thumbnails/29.jpg)
Example
• Transform the image point (10,40) using a rotation of 90 degrees and a translation of (15,-60)
![Page 30: Image formation & Geometrical Transforms Francisco Gómez J MMS U. Central y UJTL](https://reader035.vdocuments.us/reader035/viewer/2022062308/56649dac5503460f94a9c376/html5/thumbnails/30.jpg)
2D to 2d Transforms (Similarity - Scaled)
Preserve angles but not distances
![Page 31: Image formation & Geometrical Transforms Francisco Gómez J MMS U. Central y UJTL](https://reader035.vdocuments.us/reader035/viewer/2022062308/56649dac5503460f94a9c376/html5/thumbnails/31.jpg)
2D to 2d Transforms (Afinne)
Models rotation, translation, scaling, shearing, and reflection
![Page 32: Image formation & Geometrical Transforms Francisco Gómez J MMS U. Central y UJTL](https://reader035.vdocuments.us/reader035/viewer/2022062308/56649dac5503460f94a9c376/html5/thumbnails/32.jpg)
Example
I = imread('cameraman.tif'); tform = maketform('affine',[1 0 0; .25 1.5 0; 0 0 1]); J = imtransform(I,tform); imshow(I), figure, imshow(J)
![Page 33: Image formation & Geometrical Transforms Francisco Gómez J MMS U. Central y UJTL](https://reader035.vdocuments.us/reader035/viewer/2022062308/56649dac5503460f94a9c376/html5/thumbnails/33.jpg)
2D to 2d Transforms (Afinne)
Models rotation, translation, scaling, shearing, and reflection
![Page 34: Image formation & Geometrical Transforms Francisco Gómez J MMS U. Central y UJTL](https://reader035.vdocuments.us/reader035/viewer/2022062308/56649dac5503460f94a9c376/html5/thumbnails/34.jpg)
2D to 2d Transforms (Projective - Homography)
![Page 35: Image formation & Geometrical Transforms Francisco Gómez J MMS U. Central y UJTL](https://reader035.vdocuments.us/reader035/viewer/2022062308/56649dac5503460f94a9c376/html5/thumbnails/35.jpg)
What preserves?
![Page 36: Image formation & Geometrical Transforms Francisco Gómez J MMS U. Central y UJTL](https://reader035.vdocuments.us/reader035/viewer/2022062308/56649dac5503460f94a9c376/html5/thumbnails/36.jpg)
3d to 3d Transforms
• Coordinate frames– {A}, {B}
• How to describe points in {B} respect to {A}– We need a t– Rotation matrix R
![Page 37: Image formation & Geometrical Transforms Francisco Gómez J MMS U. Central y UJTL](https://reader035.vdocuments.us/reader035/viewer/2022062308/56649dac5503460f94a9c376/html5/thumbnails/37.jpg)
3d to 3d Transforms (Rotation)
http://upload.wikimedia.org/wikipedia/commons/thumb/c/c1/Yaw_Axis_Corrected.svg/2000px-Yaw_Axis_Corrected.svg.png
3 degrees of freedom! But we need a convention
![Page 38: Image formation & Geometrical Transforms Francisco Gómez J MMS U. Central y UJTL](https://reader035.vdocuments.us/reader035/viewer/2022062308/56649dac5503460f94a9c376/html5/thumbnails/38.jpg)
Rotations around the axis
![Page 39: Image formation & Geometrical Transforms Francisco Gómez J MMS U. Central y UJTL](https://reader035.vdocuments.us/reader035/viewer/2022062308/56649dac5503460f94a9c376/html5/thumbnails/39.jpg)
Rotation around the axis
• A rotation matrix can be expressed using a 3 x3 matrix
Rotation in XRotation in YRotation in Z
![Page 40: Image formation & Geometrical Transforms Francisco Gómez J MMS U. Central y UJTL](https://reader035.vdocuments.us/reader035/viewer/2022062308/56649dac5503460f94a9c376/html5/thumbnails/40.jpg)
Rotational transformation
• R represents a rotational transformation of frame A to frame B
From
From
Point represented in frame B
Point represented in frame A
![Page 41: Image formation & Geometrical Transforms Francisco Gómez J MMS U. Central y UJTL](https://reader035.vdocuments.us/reader035/viewer/2022062308/56649dac5503460f94a9c376/html5/thumbnails/41.jpg)
Transforming a 3d point
![Page 42: Image formation & Geometrical Transforms Francisco Gómez J MMS U. Central y UJTL](https://reader035.vdocuments.us/reader035/viewer/2022062308/56649dac5503460f94a9c376/html5/thumbnails/42.jpg)
In homogeneous coordinates
Homogeneous
What is the inverse?
![Page 43: Image formation & Geometrical Transforms Francisco Gómez J MMS U. Central y UJTL](https://reader035.vdocuments.us/reader035/viewer/2022062308/56649dac5503460f94a9c376/html5/thumbnails/43.jpg)
Example
• Let p=(2,2,5) in frame A• Frame B is located at (10,-5,6) and rotated 65
degrees around z-axis with respect to frame A• Where is point p located respect to frame B?
![Page 44: Image formation & Geometrical Transforms Francisco Gómez J MMS U. Central y UJTL](https://reader035.vdocuments.us/reader035/viewer/2022062308/56649dac5503460f94a9c376/html5/thumbnails/44.jpg)
Transformations are maps
A B
Transformation are answers to the question:If there are points in A how they can be written in B?
![Page 45: Image formation & Geometrical Transforms Francisco Gómez J MMS U. Central y UJTL](https://reader035.vdocuments.us/reader035/viewer/2022062308/56649dac5503460f94a9c376/html5/thumbnails/45.jpg)
Small angle approximation
sin( )= , q q sin( )=1q
Useful if you want to rotate objects in a video
![Page 46: Image formation & Geometrical Transforms Francisco Gómez J MMS U. Central y UJTL](https://reader035.vdocuments.us/reader035/viewer/2022062308/56649dac5503460f94a9c376/html5/thumbnails/46.jpg)
Recovering angles
![Page 47: Image formation & Geometrical Transforms Francisco Gómez J MMS U. Central y UJTL](https://reader035.vdocuments.us/reader035/viewer/2022062308/56649dac5503460f94a9c376/html5/thumbnails/47.jpg)
3d to 2d Transforms
Projects 3d points into 2d
![Page 48: Image formation & Geometrical Transforms Francisco Gómez J MMS U. Central y UJTL](https://reader035.vdocuments.us/reader035/viewer/2022062308/56649dac5503460f94a9c376/html5/thumbnails/48.jpg)
Perspective projection – Intrinsic matrix
Optical center (cx,cy)
How can we write the perspective projection as a transform? orHow to project 3D points represented in the coordinate system attached to the camera, to the 2D image plane?
![Page 49: Image formation & Geometrical Transforms Francisco Gómez J MMS U. Central y UJTL](https://reader035.vdocuments.us/reader035/viewer/2022062308/56649dac5503460f94a9c376/html5/thumbnails/49.jpg)
Extrinsic cameraIf 3D points are in world coordinates, we first need to transform them to camera coordinates
We can write this as an extrinsic camera matrix, that does the rotation and translation, then a projection from 3D to 2D
![Page 50: Image formation & Geometrical Transforms Francisco Gómez J MMS U. Central y UJTL](https://reader035.vdocuments.us/reader035/viewer/2022062308/56649dac5503460f94a9c376/html5/thumbnails/50.jpg)
World to camera
![Page 51: Image formation & Geometrical Transforms Francisco Gómez J MMS U. Central y UJTL](https://reader035.vdocuments.us/reader035/viewer/2022062308/56649dac5503460f94a9c376/html5/thumbnails/51.jpg)
What about color?
![Page 52: Image formation & Geometrical Transforms Francisco Gómez J MMS U. Central y UJTL](https://reader035.vdocuments.us/reader035/viewer/2022062308/56649dac5503460f94a9c376/html5/thumbnails/52.jpg)
Images in matlab
![Page 53: Image formation & Geometrical Transforms Francisco Gómez J MMS U. Central y UJTL](https://reader035.vdocuments.us/reader035/viewer/2022062308/56649dac5503460f94a9c376/html5/thumbnails/53.jpg)
RGB color space
![Page 54: Image formation & Geometrical Transforms Francisco Gómez J MMS U. Central y UJTL](https://reader035.vdocuments.us/reader035/viewer/2022062308/56649dac5503460f94a9c376/html5/thumbnails/54.jpg)
RGB color space
Hue
Purity
Luminosity