1. Illumination Estimation via Non- Negative Matrix Factorization By Lilong Shi, Brian Funt, Weihua Xiong, ( Simon Fraser University, Canada) Sung-Su Kim, Byoung-Ho Kang, Sung-Duk Lee, and Chang-Yeong Kim (Samsung Advanced Institute of Technology, Korea) Presented by: Lilong Shi

2. Automatic White Balance Problem AWB Colour constancy accounting for differences in illumination colour

3. Overview N sub-windows Take log and apply NMFsc Illumination component (low sparseness) M Reflectance basis (high sparseness) Illumination image by anti-log Reflectance images by anti-log With this we can do AWB

4. The Model of Illumination and Feature Reflectances RGB sensor response is defined by E(λ) : illumination spectral power distribution S(λ) : matte surface reflectance function R k (λ) : sensor sensitivity function of channel k Assuming narrowband sensors:

5. The Model of Illumination and Feature Reflectances In logarithm space Linear combination of illumination and reflectance For an entire colour image I, with E and S the illumination and reflectance

6. Linear Reflectance Features Illumination log E Changes slowly cross an image Reflectance log S Linear combination of M “features” F i weights h i

7. 7 Linear Reflectance Features “Feature” Reflectances “building blocks” e.g. basis images derived from the ORL face image database following Li et al. (2001) Independent  No non-zero pixels in common  Dot product of 2 blocks is zero The complete model

8. Non-Negative Matrix Factorization NMF Input data matrix Basis vectorsWeights Factored result A data instance v is a weighted combination of basis

9. Constraints on the Factorization Illumination & reflectance non-negative => NMF basis non-negative E smooth, R non-smooth Sparseness vs. Smoothness 1D example Increasing smoothness Increasing sparseness

10. Sparseness Constraint Sparseness implies most entries zero 2D example Increasing sparseness

11. Sparseness Measure Sparseness s(x) of x= Sparseness constraint is enforced during matrix factorization L-1 norm L-2 norm

12. NMFsc Using Non-Negative Matrix Factorization with sparseness constraint Calling it NMFsc

13. NMFsc for Auto White Balancing The Illumination-Reflectance model NMFsc form In combination

14. Incorporating Sparseness Finding M+1 basis vectors Set low sparseness for 1 st basis vector (illumination) Set high sparseness for 2 nd -(M+1) th basis (feature reflectance)

15. The Algorithm N sub-windows Take log and apply NMFsc Illumination basis (low sparseness) M Reflectance basis (high sparseness) Illumination image by anti-log Reflectance images by anti-log

16. Experiment on MNFsc (M=4) Input Ground Truth NMFsc result

17. Experiment on MNFsc (M=4) Illumination Image Reflectance Images

18. More Experiment on NMFsc (M=4) Input Ground Truth NMFsc result

19. Experiment on MNFsc (M=4) Illumination Image Reflectance Images

20. Experiment on MNFsc (M=1) Ground Truth Input Illumination Image NMFsc Result Reflectance Image

21. More Experiments (M=1) NMFsc Result Reflectance Image Ground Truth Input Illumination Image

22. Tests on Large Dataset (M=4) 16 sub-windows (16x16) Take log and apply NMFsc Illumination basis (sparseness=0.001) 4 Reflectance basis (sparseness = 0.45) Illumination image by anti-log Reflectance images by anti-log 7661 images (64x64) Average to estimate illumination

23. Tests on Large Dataset (M=1) Single sub-window (64x64) Take log and apply NMFsc Illumination basis (sparseness=0.001) One reflectance basis (sparseness = 0.45) Illumination image by anti-log Reflectance images by anti-log 7661 images (64x64) Average to estimate illumination

24. Results Processing Time: 0.83 sec/image for M = 4; 2.43 sec/image for M = 1; Method Angular DegreesL-2 Distance (x10 2 ) MeanMaxMeanMax GW7.6942.285.9738.33 SoG7.5034.525.5027.67 MAX RGB9.9927.427.2421.72 NMFsc (M = 4)7.66 8.9634.79 5.59 NMFsc (M = 1)6.82 8.1538.27 5.11

25. Algorithm Comparison via Wilcoxon MethodGWSoGMAX NMFsc (M=4) NMFsc (M=1) GW =+=- SoG =++- MAX ---- NMFsc (M=4) =-+- NMFsc (M=1) ++++ NMFsc better than Greyworld, Shades of Gray, Max RGB

26. Conclusions New AWB method using NMF NMF ‘factors’ illumination from reflectance Provides separate estimate for each pixel Globally minimizes objective function across all three colour channels Incorporates both colour and spatial (sparseness) information Assumptions spatially smooth illumination variation non-smooth reflectance variation

27. Conclusions Insensitive to sparseness setting NMFsc converges quickly 20-30 iterations Good AWB results Tested on large data set of natural images

28. Financial support provided by Samsung Advanced Institute of Technology

29. Thank you! Yoho National Park British Columbia, Canada