All images are compressed. The images in this file are not the original ones. SVD Photometric stereo (Miyazaki, Ikeuchi)

Photometric stereo under unknown light sources using robust SVD with missing data Daisuke Miyazaki The University of Tokyo Previously Hiroshima City University Currently Katsushi Ikeuchi The University of Tokyo SVD Photometric stereo (Miyazaki, Ikeuchi)

Geometric modeling [Laser range sensor] SVD Photometric stereo (Miyazaki, Ikeuchi) Introduction(1/2) Algorithm(10) Experiment(5) Conclusion(1) [Laser range sensor] [Uncalibrated photometric stereo] Geometric modeling

Proposed method Photometric stereo SVD: Our hole-filling SVD SVD Photometric stereo (Miyazaki, Ikeuchi) Introduction(2/2) Algorithm(10) Experiment(5) Conclusion(1) Photometric stereo SVD: Our hole-filling SVD Related work: [Mukaigawa 07] [Tomasi 92] [Shum 95] [Brand 02] Robust to outliers: shadow, specular reflection Point light source in far distance with small size: directional light Typically around 30 to 100 number of images The direction (and the power) of light sources are unknown Related work: [Hayakawa 94] [Woodham 91] [Yuille 99] [Belhumeur 99] Proposed method

Photometric stereo Lambertian model for diffuse reflection SVD Photometric stereo (Miyazaki, Ikeuchi) Introduction(2) Algorithm(1/10) Experiment(5) Conclusion(1) Lambertian model for diffuse reflection Photometric stereo

Linearization Specular reflection Shadow Cosine function Linearization SVD Photometric stereo (Miyazaki, Ikeuchi) Introduction(2) Algorithm(2/10) Experiment(5) Conclusion(1) Specular reflection Shadow Cosine function Linearization pos Linear algebra neg Linearization

Three orthonormal basis images using SVD SVD Photometric stereo (Miyazaki, Ikeuchi) PCA/SVD pos: green neg: red 3 bases [Shashua 97] -192 4 8 Linearized image -192 -128 -4 Representable by 3 bases -4 128 -192 Three orthonormal basis images using SVD

Outlier Outlier Shadow Specular reflection "hole" SVD Photometric stereo (Miyazaki, Ikeuchi) Introduction(2) Algorithm(4/10) Experiment(5) Conclusion(1) Outlier Shadow Specular reflection "hole" Outlier

Hole filling Without hole With hole Bases ? ? ? a b c a b c SVD Photometric stereo (Miyazaki, Ikeuchi) Introduction(2) Algorithm(5/10) Experiment(5) Conclusion(1) Without hole With hole Bases ? ? ? a b c a b c Hole filling

Gradually filling holes SVD Photometric stereo (Miyazaki, Ikeuchi) Introduction(2) Algorithm(6/10) Experiment(5) Conclusion(1) SVD Bases Hole filling SVD Bases Hole filling Gradually filling holes

Matrix representation SVD Photometric stereo (Miyazaki, Ikeuchi) Introduction(2) Algorithm(7/10) Experiment(5) Conclusion(1) Image number ... Image 1 Image 2 Pixel number Matrix-vector representation SVD Image matrix Matrix representation

Row/column scraping Input image with outliers SVD Photometric stereo (Miyazaki, Ikeuchi) Introduction(2) Algorithm(8/10) Experiment(5) Conclusion(1) Input image with outliers Row/column scraping

Row/column sticking SVD Hole filling Linearized input image SVD Hole SVD Photometric stereo (Miyazaki, Ikeuchi) Introduction(2) Algorithm(9/10) Experiment(5) Conclusion(1) SVD Hole filling Linearized input image SVD Hole filling Row/column sticking

Algorithm flow 1 2 3 4 Thresholding & graph cut Outlier detection SVD Photometric stereo (Miyazaki, Ikeuchi) Introduction(2) Algorithm(10/10) Experiment(5) Conclusion(1) 1 Thresholding & graph cut Outlier detection 2 Hole filling SVD Linearization [Our method] 3 Region segmentation Light source direction estimation [Hayakawa 94] [Shi 10] [Belhumeur 99] 4 Occluding boundary Light source direction estimation [Sato 07] Algorithm flow

Linearizaiton result Input image Linearized image SVD Photometric stereo (Miyazaki, Ikeuchi) Introduction(2) Algorithm(10) Experiment(1/5) Conclusion(1) Input image Linearized image Linearizaiton result

Result SVD Photometric stereo (Miyazaki, Ikeuchi) Introduction(2) Algorithm(10) Experiment(2/5) Conclusion(1) Result

Result SVD Photometric stereo (Miyazaki, Ikeuchi) Introduction(2) Algorithm(10) Experiment(3/5) Conclusion(1) Result

With outlier detection SVD Photometric stereo (Miyazaki, Ikeuchi) Introduction(2) Algorithm(10) Experiment(4/5) Conclusion(1) Without outlier detection With outlier detection [Our result] Comparison

With outlier detection SVD Photometric stereo (Miyazaki, Ikeuchi) Introduction(2) Algorithm(10) Experiment(5/5) Conclusion(1) Without outlier detection With outlier detection [Our result] Comparison

Conclusion SVD Photometric Stereo Hole filling SVD SVD Photometric stereo (Miyazaki, Ikeuchi) Introduction(2) Algorithm(10) Experiment(5) Conclusion(1/1) SVD Photometric Stereo Hole filling SVD Light source unknown Shadow Specular reflection Con: Computation time (1h~1d) Conclusion

(c) Crypton Future Media, Inc. Crypton Future Media, Inc. owns the copyright of this character, Miku Hatsune. This picture is drawn by Daisuke Miyazaki under Piapro Character License. http://piapro.jp/license/pcl/summary SVD Photometric stereo (Miyazaki, Ikeuchi)

Daisuke Miyazaki 2010 Creative Commons Attribution 4 Daisuke Miyazaki 2010 Creative Commons Attribution 4.0 International License. http://www.cg.info.hiroshima-cu.ac.jp/~miyazaki/ Daisuke Miyazaki, Katsushi Ikeuchi, "Photometric stereo under unknown light sources using robust SVD with missing data," Proceedings of International Conference on Image Processing, pp. 4057-4060, Hong Kong, China, Sep. 2010 以上で発表を終わります。ありがとうございました。 SVD Photometric stereo (Miyazaki, Ikeuchi)