1. All images are compressed.
The images in this file are not the original ones.
SVD Photometric stereo (Miyazaki, Ikeuchi)

2. Photometric stereo under unknown light sources using robust SVD with missing data
Daisuke Miyazaki The University of Tokyo Previously
Hiroshima City University Currently
Katsushi Ikeuchi The University of Tokyo
SVD Photometric stereo (Miyazaki, Ikeuchi)

3. Geometric modeling [Laser range sensor]
SVD Photometric stereo (Miyazaki, Ikeuchi)
Introduction(1/2)
Algorithm(10)
Experiment(5)
Conclusion(1)
[Laser range sensor]
[Uncalibrated photometric stereo]
Geometric modeling

4. Proposed method Photometric stereo SVD: Our hole-filling SVD
SVD Photometric stereo (Miyazaki, Ikeuchi)
Introduction(2/2)
Algorithm(10)
Experiment(5)
Conclusion(1)
Photometric stereo
SVD: Our hole-filling SVD
Related work: [Mukaigawa 07] [Tomasi 92] [Shum 95] [Brand 02]
Robust to outliers: shadow, specular reflection
Point light source in far distance with small size: directional light
Typically around 30 to 100 number of images
The direction (and the power) of light sources are unknown
Related work: [Hayakawa 94] [Woodham 91] [Yuille 99] [Belhumeur 99]
Proposed method

5. Photometric stereo Lambertian model for diffuse reflection
SVD Photometric stereo (Miyazaki, Ikeuchi)
Introduction(2)
Algorithm(1/10)
Experiment(5)
Conclusion(1)
Lambertian model for diffuse reflection
Photometric stereo

6. Linearization Specular reflection Shadow Cosine function Linearization
SVD Photometric stereo (Miyazaki, Ikeuchi)
Introduction(2)
Algorithm(2/10)
Experiment(5)
Conclusion(1)
Specular reflection
Shadow
Cosine function
Linearization
pos
Linear algebra
neg
Linearization

7. Three orthonormal basis images using SVD
SVD Photometric stereo (Miyazaki, Ikeuchi)
PCA/SVD
pos: green
neg: red
3 bases
[Shashua 97]
-192 4 8
Linearized image
-192
-128 -4
Representable
by 3 bases -4
128
-192
Three orthonormal basis images using SVD

8. Outlier Outlier Shadow Specular reflection "hole"
SVD Photometric stereo (Miyazaki, Ikeuchi)
Introduction(2)
Algorithm(4/10)
Experiment(5)
Conclusion(1)
Outlier
Shadow
Specular reflection
"hole"
Outlier

9. Hole filling Without hole With hole Bases ? ? ? a b c a b c
SVD Photometric stereo (Miyazaki, Ikeuchi)
Introduction(2)
Algorithm(5/10)
Experiment(5)
Conclusion(1)
Without hole
With hole
Bases ? ? ? a b c a b c
Hole filling

10. Gradually filling holes
SVD Photometric stereo (Miyazaki, Ikeuchi)
Introduction(2)
Algorithm(6/10)
Experiment(5)
Conclusion(1)
SVD
Bases
Hole filling
SVD
Bases
Hole filling
Gradually filling holes

11. Matrix representation
SVD Photometric stereo (Miyazaki, Ikeuchi)
Introduction(2)
Algorithm(7/10)
Experiment(5)
Conclusion(1)
Image
number
...
Image 1
Image 2
Pixel
number
Matrix-vector representation
SVD
Image matrix
Matrix representation

12. Row/column scraping Input image with outliers
SVD Photometric stereo (Miyazaki, Ikeuchi)
Introduction(2)
Algorithm(8/10)
Experiment(5)
Conclusion(1)
Input image
with outliers
Row/column scraping

13. Row/column sticking SVD Hole filling Linearized input image SVD Hole
SVD Photometric stereo (Miyazaki, Ikeuchi)
Introduction(2)
Algorithm(9/10)
Experiment(5)
Conclusion(1)
SVD
Hole
filling
Linearized
input image
SVD
Hole
filling
Row/column sticking

14. Algorithm flow 1 2 3 4 Thresholding & graph cut Outlier detection
SVD Photometric stereo (Miyazaki, Ikeuchi)
Introduction(2)
Algorithm(10/10)
Experiment(5)
Conclusion(1) 1
Thresholding & graph cut
Outlier detection 2
Hole filling SVD
Linearization
[Our method] 3
Region segmentation
Light source direction estimation
[Hayakawa 94] [Shi 10] [Belhumeur 99] 4
Occluding boundary
Light source direction estimation
[Sato 07]
Algorithm flow

15. Linearizaiton result Input image Linearized image
SVD Photometric stereo (Miyazaki, Ikeuchi)
Introduction(2)
Algorithm(10)
Experiment(1/5)
Conclusion(1)
Input image
Linearized image
Linearizaiton result

16. Result SVD Photometric stereo (Miyazaki, Ikeuchi) Introduction(2)
Algorithm(10)
Experiment(2/5)
Conclusion(1)
Result

17. Result SVD Photometric stereo (Miyazaki, Ikeuchi) Introduction(2)
Algorithm(10)
Experiment(3/5)
Conclusion(1)
Result

18. With outlier detection
SVD Photometric stereo (Miyazaki, Ikeuchi)
Introduction(2)
Algorithm(10)
Experiment(4/5)
Conclusion(1)
Without outlier detection
With outlier detection
[Our result]
Comparison

19. With outlier detection
SVD Photometric stereo (Miyazaki, Ikeuchi)
Introduction(2)
Algorithm(10)
Experiment(5/5)
Conclusion(1)
Without outlier detection
With outlier detection
[Our result]
Comparison

20. Conclusion SVD Photometric Stereo Hole filling SVD
SVD Photometric stereo (Miyazaki, Ikeuchi)
Introduction(2)
Algorithm(10)
Experiment(5)
Conclusion(1/1)
SVD Photometric Stereo
Hole filling SVD
Light source unknown
Shadow
Specular reflection
Con: Computation time (1h~1d)
Conclusion

21. (c) Crypton Future Media, Inc.
Crypton Future Media, Inc. owns the copyright of this character, Miku Hatsune. This picture is drawn by Daisuke Miyazaki under Piapro Character License.
SVD Photometric stereo (Miyazaki, Ikeuchi)

22. Daisuke Miyazaki 2010 Creative Commons Attribution 4
Daisuke Miyazaki 2010 Creative Commons Attribution 4.0 International License.
Daisuke Miyazaki, Katsushi Ikeuchi, "Photometric stereo under unknown light sources using robust SVD with missing data," Proceedings of International Conference on Image Processing, pp , Hong Kong, China, Sep. 2010
以上で発表を終わります。ありがとうございました。
SVD Photometric stereo (Miyazaki, Ikeuchi)