1. Scene illumination and surface albedo recovery via L1-norm total variation minimization Hong-Ming Chen hc2599@columbia.edu Advised by: John Wright

2. Decomposition of a scene 2 =.* sceneReflectance (albedo) illumination.* : Matlab element multiplication operation

3. Image Formation 3 =.* scenereflectanceillumination Sensor response (camera or eyes) Light source power spectrum Object reflectance intensityresponseSensor response integration Pixel i signals : shutter speed, aperture size, quantization factor etc

4. It is VERY HARD to directly model / simulate / solve this problem! 4

5. Narrowing down our target problem Simplification: – mean wavelength response (impulse response) Assumption (on surface reflectance) : – Lambertian Surface (Perfect diffuse reflection, no specular light) Simulation (of light source model) : – We need a formula to describe the behavior of the light source – Blackbody radiation: parameterize the light source with: Light color (color temperature) Light intensity 5

6. Problem formulation: 6

7. 7 log Assume: λ R λ G λ G are known If there are N pixels in an image: 3N observations 5N unknowns (I, T, ref ) + 3 quantize factors underdetermined system!

8. 8

9. Recovering unknown x Previous approach – Introducing regularization terms into objective function Current approach – Minimizing L1-norm total variation 9

10. Previous Approach 10 1-D grayscale visualization A segmentation-like result A result of: Intrinsic images by entropy minimization, Finlayson, ECCV2004 psps p ps 0 255

11. Drawbacks of this approach There are at least 2 parameters (λ, σ) to be fine tuned. The results of Finlayson’s approach heavily affects the accurateness of our prior. – 1. Its Achilles heel: projection problem – 2. it is still an open problem to find the best rotation angle. 11

12. 12 (λ =50, σ = 10)(λ =10, σ = 30) (λ =120, σ = 5)(λ =120, σ = 8)

13. A brief review of Finlayson’ solution Its Achilles heel: 13

14. L1 norm Total Variation Minimization 14 Image From Wikipedia

15. L1 norm Total Variation Minimization Widely used in image denoise / Compressive sensing – E(x, y) + λTV(y). 15 Image From Wikipedia

16. Current approach: L1 TV norm Applying L1-norm total variation on albedo term, The L1-norm encourages a spiky result on gradient – Which means: we want most of the albedo gradients are 0 unless necessary => when albedo changes 16

17. 17 Results Original image Light color (temperature) imageLight intensity image Albedo (reflectance) image

18. 18 Results Original image Light color (temperature) imageLight intensity image Albedo (reflectance) image

19. 19 Results Original image Light color (temperature) imageAlbedo (reflectance) image

20. 20 Results Original image Light color (temperature) imageAlbedo (reflectance) image

21. Editing 21 Original imageAverage T-1000Average T+1000 Average T+2000Average T+3000Average T+4000 Average T = 3940

22. THANK YOU 22