http:// computer graphics laboratory, hiroshima city university all images are compressed. the...
TRANSCRIPT
http://www.cg.info.hiroshima-cu.ac.jp/
Computer Graphics Laboratory, Hiroshima City University
All images are compressed.
The images in this file are not the original ones.
SVD Photometric stereo (Miyazaki, Ikeuchi)
http://www.cg.info.hiroshima-cu.ac.jp/
Computer Graphics Laboratory, Hiroshima City University
Photometric stereo under unknown light sources using robust SVD with
missing data
Daisuke Miyazaki The University of Tokyo PreviouslyHiroshima City University Currently
Katsushi Ikeuchi The University of Tokyo
SVD Photometric stereo (Miyazaki, Ikeuchi)
http://www.cg.info.hiroshima-cu.ac.jp/
CG Lab, Hiroshima City University
Geometric modeling
SVD Photometric stereo (Miyazaki, Ikeuchi)
[Laser range sensor] [Uncalibrated photometric stereo]
Algorithm(10) Experiment(5) Conclusion(1)Introduction(1/2)
http://www.cg.info.hiroshima-cu.ac.jp/
CG Lab, Hiroshima City University
Proposed method
• Photometric stereo• SVD: Our hole-filling SVD
– Related work: [Mukaigawa 07] [Tomasi 92] [Shum 95] [Brand 02]
• Robust to outliers: shadow, specular reflection• Point light source in far distance with small size:
directional light• Typically around 30 to 100 number of images• The direction (and the power) of light sources are
unknown– Related work: [Hayakawa 94] [Woodham 91] [Yuille 99]
[Belhumeur 99]
SVD Photometric stereo (Miyazaki, Ikeuchi)
Algorithm(10) Experiment(5) Conclusion(1)Introduction(2/2)
http://www.cg.info.hiroshima-cu.ac.jp/
CG Lab, Hiroshima City University
Photometric stereo
SVD Photometric stereo (Miyazaki, Ikeuchi)
Lambertian model for diffuse reflection
ln ,0maxdkI
cos,0maxdkI
Algorithm(1/10) Experiment(5) Conclusion(1)Introduction(2)
http://www.cg.info.hiroshima-cu.ac.jp/
CG Lab, Hiroshima City University
Linearization
SVD Photometric stereo (Miyazaki, Ikeuchi)
Specular reflection
Shadow
pos
neg
Linearization ssd kkI BRDF,0max ln
ln dkI
Linear algebra
Cosine function
Algorithm(2/10) Experiment(5) Conclusion(1)Introduction(2)
http://www.cg.info.hiroshima-cu.ac.jp/
CG Lab, Hiroshima City University
Three orthonormal basis images using SVD
SVD Photometric stereo (Miyazaki, Ikeuchi)
PCA/SVD
3 bases
[Shashua 97]
-192 4 8
-192 -128 -4
-192 -4 128
Linearized image
Representableby 3 bases
pos: greenneg: red
http://www.cg.info.hiroshima-cu.ac.jp/
CG Lab, Hiroshima City University
Outlier
SVD Photometric stereo (Miyazaki, Ikeuchi)
Shadow
Specular reflection
Outlier
"hole"
Algorithm(4/10) Experiment(5) Conclusion(1)Introduction(2)
http://www.cg.info.hiroshima-cu.ac.jp/
CG Lab, Hiroshima City University
Hole filling
SVD Photometric stereo (Miyazaki, Ikeuchi)
Without hole
Bases
With hole
???
cba
cba
Algorithm(5/10) Experiment(5) Conclusion(1)Introduction(2)
http://www.cg.info.hiroshima-cu.ac.jp/
CG Lab, Hiroshima City University
Gradually filling holes
SVD Photometric stereo (Miyazaki, Ikeuchi)
SVD
Bases
Hole filling
SVD
Bases
Hole filling
Algorithm(6/10) Experiment(5) Conclusion(1)Introduction(2)
http://www.cg.info.hiroshima-cu.ac.jp/
CG Lab, Hiroshima City University
Matrix representation
SVD Photometric stereo (Miyazaki, Ikeuchi)
Pixelnumber
Imagenumber
Matrix-vector representation
Image matrix
SVD
...
Image 1 Image 2
Algorithm(7/10) Experiment(5) Conclusion(1)Introduction(2)
http://www.cg.info.hiroshima-cu.ac.jp/
CG Lab, Hiroshima City University
Row/column scraping
SVD Photometric stereo (Miyazaki, Ikeuchi)
Input imagewith outliers
Algorithm(8/10) Experiment(5) Conclusion(1)Introduction(2)
http://www.cg.info.hiroshima-cu.ac.jp/
CG Lab, Hiroshima City University
Row/column sticking
SVD Photometric stereo (Miyazaki, Ikeuchi)
Holefilling
Linearizedinput imageHole
filling
SVD
SVD
Algorithm(9/10) Experiment(5) Conclusion(1)Introduction(2)
http://www.cg.info.hiroshima-cu.ac.jp/
CG Lab, Hiroshima City University
Algorithm flow
SVD Photometric stereo (Miyazaki, Ikeuchi)
Thresholding & graph cut
Hole filling SVD
Region segmentation
Occluding boundary
Outlier detection
Linearization
Light source direction estimation
[Hayakawa 94] [Shi 10] [Belhumeur 99]
[Sato 07]
[Our method]
Light source direction estimation
1
2
3
4
Algorithm(10/10) Experiment(5) Conclusion(1)Introduction(2)
http://www.cg.info.hiroshima-cu.ac.jp/
CG Lab, Hiroshima City University
Linearizaiton result
SVD Photometric stereo (Miyazaki, Ikeuchi)
Input image
Linearized image
Algorithm(10) Experiment(1/5) Conclusion(1)Introduction(2)
http://www.cg.info.hiroshima-cu.ac.jp/
CG Lab, Hiroshima City University
Result
SVD Photometric stereo (Miyazaki, Ikeuchi)
Algorithm(10) Experiment(2/5) Conclusion(1)Introduction(2)
http://www.cg.info.hiroshima-cu.ac.jp/
CG Lab, Hiroshima City University
Result
SVD Photometric stereo (Miyazaki, Ikeuchi)
Algorithm(10) Experiment(3/5) Conclusion(1)Introduction(2)
http://www.cg.info.hiroshima-cu.ac.jp/
CG Lab, Hiroshima City University
Comparison
SVD Photometric stereo (Miyazaki, Ikeuchi)
With outlier detection[Our result]
Without outlier detection
Algorithm(10) Experiment(4/5) Conclusion(1)Introduction(2)
http://www.cg.info.hiroshima-cu.ac.jp/
CG Lab, Hiroshima City University
Comparison
SVD Photometric stereo (Miyazaki, Ikeuchi)
With outlier detection[Our result]
Without outlier detection
Algorithm(10) Experiment(5/5) Conclusion(1)Introduction(2)
http://www.cg.info.hiroshima-cu.ac.jp/
CG Lab, Hiroshima City University
Conclusion
SVD Photometric stereo (Miyazaki, Ikeuchi)
SVD Photometric Stereo
Con: Computation time (1h~1d)
Light source unknown Shadow Specular reflection
Hole filling SVD
Algorithm(10) Experiment(5) Conclusion(1/1)Introduction(2)
http://www.cg.info.hiroshima-cu.ac.jp/
Computer Graphics Laboratory, Hiroshima City University
(c) Crypton Future Media, Inc.
Crypton Future Media, Inc. owns the copyright of this character, Miku Hatsune. This picture is
drawn by Daisuke Miyazaki under Piapro Character License.
http://piapro.jp/license/pcl/summary
SVD Photometric stereo (Miyazaki, Ikeuchi)
http://www.cg.info.hiroshima-cu.ac.jp/
Computer Graphics Laboratory, Hiroshima City University
(c) Daisuke Miyazaki 2010All rights reserved.
http://www.cg.info.hiroshima-cu.ac.jp/~miyazaki/Daisuke Miyazaki, Katsushi Ikeuchi, "Photometric stereo
under unknown light sources using robust SVD with missing data," Proceedings of International Conference on Image
Processing, pp. 4057-4060, Hong Kong, China, Sep. 2010
SVD Photometric stereo (Miyazaki, Ikeuchi)