1. Relighting with 4D Incident Light Fields Vincent Masselus Pieter Peers Philip Dutré Yves D. Willems

2. Relighting: apply incident illumination to a real object

3. Reflectance field Transfer of light through the sceneTransfer of light through the scene 4D exitant 4D incident

4. Previous work Debevec et al. SIGGRAPH2000Debevec et al. SIGGRAPH2000 2D exitant 2D incident Images ACM SIGGRAPH (Environment map) (Image)

5. Previous work Matusik et al. SIGGRAPH2002Matusik et al. SIGGRAPH2002 4D exitant Images ACM SIGGRAPH 2D incident (Environment map) (Light field)

6. Incident Illumination Usually represented by an environment mapUsually represented by an environment map 2D incident

7. Spatial varying incident illumination Environment Map: incident illumination at one pointEnvironment Map: incident illumination at one point

8. New effects:New effects: –Spot light –Shadows –… New effects:New effects: –Spot light –Shadows –… Goal: relighting with angular and spatially varying illumination

9. One view pointOne view point Use artificial incident light fields or captured from real environmentUse artificial incident light fields or captured from real environment Goal: relighting with angular and spatially varying illumination 2D exitant (Image) 4D incident (Light Field)

10. Acquire the reflectance field Use camera to capture exitant light fieldUse camera to capture exitant light field Use projector to create incident light fieldUse projector to create incident light field Projector Camera 2D exitant (Image) 4D incident (Light Field)

11. Data acquisition

16. Data Acquisition: setup Projector Camera Object

17. Data Acquisition: setup

18. Relighting Incident Light Field Scene x W 1 W1W1 Apply

19. Relighting Incident Light Field Scene x W 1 x W 2 W2W2

20. Relighting Incident Light Field Scene x W 1 x W 2 x W 3 W3W3

21. Relighting Incident Light Field Scene x W 1 x W 2 x W 3 … x W n WnWn

22. Relighting x W 1 x W 2 x W 3 … x W n + + +

23. NxN light bundles

24. #images per projector position = N 2#images per projector position = N 2 Reduce #images per projector position: O(N)Reduce #images per projector position: O(N) Speedup … … … … … N N

25. O(N) Patterns

26. Reconstruct basis image

27. Take the minimum value per pixelTake the minimum value per pixel

28. #Patterns per projector position = O(N)#Patterns per projector position = O(N) Assumption: limited local influenceAssumption: limited local influence Total complexity:Total complexity: O(PxN 2 ) to O(PxN) with P = #projector positions O(PxN 2 ) to O(PxN) with P = #projector positions Speedup … … 4 N

29. Results 32x7 projector positions32x7 projector positions 16x16 light bundles16x16 light bundles 57344 basis images57344 basis images reconstructed from reconstructed from 14336 photographs

31. Relighting with a captured 4D Incident Light Field Unger et al. EGSR2003Unger et al. EGSR2003

32. 4D Incident light field vs. 2D Environment map 4D 2D

33. Comparison 4D vs. 2D Illumination 2D 4D

34. Results 32x7 projector positions32x7 projector positions 32x32 light bundles32x32 light bundles 229376 basis images229376 basis images reconstructed from 28672 photographs

35. Results

37. Conclusion Acquire reflectance field to relight with 4D incident light fieldsAcquire reflectance field to relight with 4D incident light fields Speedup: O(PxN 2 ) to O(PxN)Speedup: O(PxN 2 ) to O(PxN) Use other illumination basis functionsUse other illumination basis functions –Gaussians, Wavelets,… Smarter acquisitionSmarter acquisition –What images do we really need? Move the cameraMove the camera –Capture the complete 8D reflection field Future work

38. Acknowledgements Graphics group from K.U.LeuvenGraphics group from K.U.Leuven All reviewersAll reviewers Captured ILF by Jonas UngerCaptured ILF by Jonas Unger