kuliah umum pada masa matrikulasi s2 mmsi 24feb2012
TRANSCRIPT
1
SISTEM PENCITRAAN PENDETEKSI SISTEM PENCITRAAN PENDETEKSI POLARISASI CAHAYA MENGGUNAKAN POLARISASI CAHAYA MENGGUNAKAN
SISTEM VISI STEREOSISTEM VISI STEREO
Mohammad IqbalDisampaikan pada 24 Februari 2012
Di Kuliah Umum Pasca Sarjana Universitas Gunadarma
2
1 – 2 – 3 – 4 - 5
Itot
• Intensity
• Wave length – color
• Polarization
The fact of light in Nature
Introduction
3
1 – 2 – 3 – 4 - 5
Introduction
• Polarization is a property of waves that describes the orientation of their oscillations
• Light can be polarized by several processes :• Selective Absorption – Dichroism• Reflection• Scattering• Birefringent
What is Polarization of Light?
4
1 – 2 – 3 – 4 - 5
IntroductionPolarization of Light Model
Type of Polarized Light• Unpolarized light: random phase• Polarized light :
– Linear polarization– Circular polarization– Elliptical Polarization
5
1 – 2 – 3 – 4 - 5
IntroductionParameter of Polarization of Light
Itot
• Total Intensity (Itot)• Phase (δ) : phase shift between orthogonal transverse
• δ=0 linear polization• δ =±π/2 circular polarization.
• Angle of Polarization (ϕ): main direction of the electromagnetic vibration
• Degree of Polarization (ρ): proportion of the polarized light
• Perfectly polarized wave = DOP of 100%, • unpolarized wave = a DOP of 0%. • Partially polarized, = DOP somewhere in between 0 and 100%.
6
1 – 2 – 3 – 4 - 5
Introduction
• Useful for navigation tasks :– Self Localization – Orientation– Detection of water, food, prey or obstacle
• Bio-inspired examples :– terrestrial animal
• visual ability to analyze light pattern in the sky or in the reflected surfaces.
– marine animals • Camouflage or communication• Enhance the visibility of the scenes
• Most of animal have a stereo capabilities to aware scene around.
Why Polarization & Stereo ?
7
1 – 2 – 3 – 4 - 5
Introduction
The ChallengePolarization Imaging : The images are ‘darker’ than intensity images, need at least three different images.
Stereo vision : matching point problem, need clear images, different view, need more geometric approach.
HOW TO GET A WIN-WIN COMBINATION?
8
1 – 2 – 3 – 4 - 5
Introduction
• To build a prototype of stereo vision system with polarization sensitivity :–Can measure DoP and AoP for every
angle of incident light.–Can reconstruct 3D point of stereo
images
• To develop a simple and fast polarization imaging algorithm based-on stereo vision
–Simple and Efficient in setup and algorithm
–Easy to Use–Not expensive
The Objective
9
1 – 2 – 3 – 4 - 5
Combination Polarization & Stereo ImagingLearning from Previous Researches
Wolff• Increase the polarization
parameters estimation• No 3D information
Wolff et al (1990, 1994, 1995)
• Stereo video polarimetrysystem to visualize the polarization patterns in stereovision
• Displacement of the camera
Mizera et al 2001
1
2
Sarafraz 2009• Two images are taken
simultaneously with different polarization filter settings
• Only the degree of polarization is estimated from the ratio of the images difference.
• No 3D reconstruction
3
BeamSplitter
1 camera
Multi camera
10
1 – 2 – 3 – 4 - 5
Itot
Setup Design and CalibrationSetup Design and Calibration• Description of System• Polarization calibration• Geometric calibration
11
1 – 2 – 3 – 4 - 5
Description of SystemPrinciple of Design
Itot
• Stereovision => 2 cameras• Measurement of partially linearly polarized light =>
at least need 3 images• Automatic acquisition => Liquid crystal components
12
1 – 2 – 3 – 4 - 5
Description of SystemExample of Captured Polarized light Images
Itot
Right CameraLeft camera
45°0°90°0°
13
1 – 2 – 3 – 4 - 5
ItotMain Capabilities
Calibration
Stereo Evaluation
Description of SystemPolarization-Stereo Imaging System Schema
Image Acquisition
Calibration
Polarization Calibration
Stereo Geometric Calibration
Extract Extract Polarization Polarization InformationInformation
Remove Outlier
3D 3D ReconstructionReconstruction
Feature Detection
Rectifying Image
Stereo Matching
14
1 – 2 – 3 – 4 - 5
Imaging System Calibration1 - Polarization Calibration
Itot
•Why Calibration is Important ?–Misalignment of optical devices–Polarizers settings may be different–To provide accuracy result
• Principle–Provide incident light with known
polarization state–Estimate the offset of the LC
component
Rotating polarizer LC component
βα = 0°-180°
s s’ s"
Mpol MLC
s"= MLC(β) . Mpol(α) . s
15
1 – 2 – 3 – 4 - 5
Imaging System Calibration2 - Geometric Calibration
Itot
This geometric calibration step is providing :
‐ Intrinsic parameters‐Extrinsic parametersOf stereo cameras
Using Bouguet’s Toolbox
16
1 – 2 – 3 – 4 - 5
Itot
Stereo Vision Evaluation• Rectifying Image• Feature Detector• Stereo Matching
17
1 – 2 – 3 – 4 - 5
Stereo Vision Evaluation1 – Rectifying Image
Itot
Epipolar geometry
Image Rectification
18
1 – 2 – 3 – 4 - 5
Stereo Vision Evaluation2 – Feature Detector
Itot
• Criteria to choose the feature detector :– to deal with the polarization effect
on images– Less complexity and not expensive
computation
• The features to be extracted can be grouped into three main classes [1], namely:– Low-level (e.g. colour, gradient,
motion)– Mid-level (e.g. edges, interest point
; corners, regions)– High-level (objects)
[1] Cavallaro and Maggio, 2011
Design scheme for compare the capability of Harris
Corner detector and SIFT feature detector algorithm in
Polarization effect
19
1 – 2 – 3 – 4 - 5
Stereo Vision Evaluation2 – Feature Detector (Cont.)
Itot
1. Harris Result
2. SIFT Result
20
1 – 2 – 3 – 4 - 5
Stereo Vision Evaluation2 – Feature Detector (Cont.)
ItotPrimary results show that SIFT feature detector is more
appropriated to work on polarized images than the Harris Corner detector.
Conclusion Feature Detector Experiment
Left Image Right Image
21
1 – 2 – 3 – 4 - 5
Stereo Vision Evaluation3 – Stereo Matching Score Algorithm
Itot
• Three different metrics have to be taken Choosing Local Matching Score for Polarized Images– Test 6 Local matching score algorithm : SAD, SSD, NSSD, NCC, Census,
Rank for Polarized Images
Rank metrics
Correlation based
Intensity differences
22
1 – 2 – 3 – 4 - 5
Stereo Vision Evaluation3 – Stereo Matching Score Algorithm (Cont.)
Mean of SAD Error Result for Six Matching Algorithm For Each Scenes Incident Light (0°,10°,20°,30°,45°) –
Lowest is the best
Conclusion
Normalized SSD algorithm gave better matching results when applied to polarized images, compared to other local matching algorithms
23
1 – 2 – 3 – 4 - 5
Itot
Main Capabilities• 3D reconstruction• Polarization estimation• Evaluation of System
Capabilities
24
1 – 2 – 3 – 4 - 5
3D Recontruction1 – Remove Outliers
RANSAC considers the features that do not fit the current geometric model as outliers and eliminates them in an iterative manner and the geometric model is estimated again on the basis of newly identified inliers.
Putative Match Inlying Matches
Fischler and Bolles, 1981
25
1 – 2 – 3 – 4 - 5
3D Reconstruction2 – Recovery 3D Coordinates
Triangulation need :1. The relative position and
orientation of the two cameras (intrinsic and extrinsic camera parameters) from geometric calibration result
2. Correspondence point From matching
point
26
1 – 2 – 3 – 4 - 5
3D Reconstruction2 – Recovery 3D Coordinates (Cont.)
First experiment : • extract all points in
the stereo images rather than using a feature detector algorithm.
27
1 – 2 – 3 – 4 - 5
3D Reconstruction2 – Recovery 3D Coordinates (Cont.)
Visual Result 3D Recontruction
Images for 3D Reconstruction : (I0left and I0right )
Feature Detector (SIFT)
Matching score(NSSD)
Remove Outliers (RANSAC)
Triangulation The reconstructed 3D points still need to be increased in quantity.
Second experiment :• Using points from feature
detector algorithm.
28
1 – 2 – 3 – 4 - 5
Polarization Estimation1 – Our Method
The Computation is :• Based on the polarization calibration result. • Only in the point matches from stereo vision
part results. • Using Least mean Square of 4 Pol Intensity
to get S0, S1 and S2
right right right0 0 1 0 2 0
right right right45 0 1 45 2 45
1I ( S S cos 2 S sin 2 )21I ( S S cos 2 S sin 2 )2
α α
α α
⎧ = + +⎪⎪⎨⎪ = + +⎪⎩
left left left0 0 1 0 2 0
left left left90 0 1 90 2 90
1I ( S S cos 2 S sin 2 )21I ( S S cos 2 S sin 2 )2
α α
α α
⎧ = + +⎪⎪⎨⎪ = + +⎪⎩
We using four polarizer intensity images through a set ofpolarization filters in stereo system : I0left, I0right, I45right
and I90left
29
1 – 2 – 3 – 4 - 5
Polarization Estimation2 – How to Vizualize Polarization (AoP and DoP)
(x,y) AOP /AOP /Angle ofPolarization
DOP /DOP /Degree of
Polarization
(x2,y2)
(x1,y1)
10°
20°
30°
30
1 – 2 – 3 – 4 - 5
Evaluation of System Capabilities1 – Homogenous Scene
0° and 90°
0° and 45°
For Stereo Matching using: left I0+I90 and right I45
Result :
Combination Polarization Intensity for Input Imaging System
A-First Test
B-Enchanced Result Test
31
1 – 2 – 3 – 4 - 5
Evaluation of System Capabilities2 – Heterogenous Scene
• In nature, the light reflected from real objects would have many variations in orientations.
The experiments is to show how our setup has the ability to capture the variations of incident light and extract the polarization information with the
proper orientation
Result :
32
1 – 2 – 3 – 4 - 5
Resume Polarization-Stereo System22 33
44
5B5B
3D Recontruction by Triangulation
Extract features by SIFTStereo matching by NSSDRANSAC to remove outliers
Source image : • Stereo Evaluation :
• Class data 4 or• Class data 1 : I0left and
I0right
• Polarization Extraction :Point matches at :• I0left and I0right
• I90left and I45
right
11
22
33
44
5A5A
Extract polarization information5B5B
5A5A
33
1 – 2 – 3 – 4 - 5
Conclusion and Future Work• Conclusion• Future Work
34
1 – 2 – 3 – 4 - 5
Conclusion
In this work, we have done to develop :• Polarization Stereoscopic Imaging system Prototype:
– Polarization calibration technique– Feature detection and Stereo Matching evaluation algorithm– Extract Polarization Information from matching point of stereo
System– Tested the system on various real scenes.
• Prototype design consists of: – two optical devices, based on liquid
crystal : polarizer oriented at 0°-90°, and 0°- 45°.
– two intensity cameras.
– Grayscale images with resolution 640x480.
35
1 – 2 – 3 – 4 - 5
Future Work
Image Acquisition
Calibration
Polarization Calibration
Stereo Geometric Calibration
Extract Polarization Information
Remove Outlier
Feature Detection
Rectifying Image
Stereo Matching
RoboticsApplication
Object normal estimation
Improvement Photometric Invariant
Feature Detector
3D Reconstruction
36
1 – 2 – 3 – 4 - 5
Terima KasihThank Youmerci
Itot