1. Global Illumination with a Virtual Light Field Mel Slater Jesper Mortensen Pankaj Khanna Insu Yu Dept of Computer Science University College London Vlfproject@cs.ucl.ac.uk GR/R13685/01 Research funded by:

2. Introduction Problem Background – Light Field Fundamental data structures & algorithm Implementation Issues Results Conclusions

3. Problem: Global Illumination in Real-Time The goal of this research is to achieve real- time walkthough of globally illuminated scenes. – Interaction between any type of surface properties (I.e., BRDFs) Technically – to provide a fast approximate solution of the rendering equation. – L(p,  ) = L e (p,  ) +  f(p,  i,  ) L(p,  i ) cos  i d  i

4. Global Illumination Ray tracing – highly view dependent, models only specular reflection Path tracing – Monte Carlo solution to rendering equation (view dependent) Radiosity – view independent, models only diffuse reflection Photon mapping/Monte Carlo – rays through light emitters (view independent) view dependent ray tracing phase

5. Faster Global Illumination? Caching – reuse elements of a G.I. for one view in other views. Ward, G. and Simmons, M. (1999) The Holodeck Ray Cache: An Interactive Rendering System for Global Illumination in Nondiffuse Environments, ACM TOG, 18(4):361-98. Precompute – e.g., estimate point lights that give similar solution to global illumination. Walter, B., Alppay, G., Lafortune, E.P.F., Fernandez, S. and Greenberg, D. P. (1997) Fitting virtual lights for non-diffuse walkthroughs, SIGGRAPH 97, 45–48. H/W ray tracing – implement BSP ray tracing in hardware efficient way. Wald, I., Slusallek, P., Benthin, C., Wagner, M.: Interactive Rendering with Coherent Ray Tracing, Eurographics 2001 Proceedings.

6. Virtual Light Field (VLF) Gortler used term ‘virtual light field’ to refer to light field for a synthetic scene. We extend the term to mean a LF for a virtual scene – But without the use of a separate rendering system – With illumination computed within the LF itself.

7. Virtual Light Field Any ray is a point in 4D space R 4 Suppose ‘all rays’ can (somehow) be represented. Consider one ray:- t1 t2 0 1 t3 t4

8. Virtual Light Field: Basic Concepts Discrete representation of all rays in a scene All rays = union of parallel subfields (PSF) Parallel subfield is a rectangular grid of rays all in the same direction Each ray is segmented by the objects it intersects Light is propagated out from light sources following the paths defined by the segmented rays The rays are used as approximation to true rays Rendering achieved in manner similar to normal light field.

9. Use Ray Approximations Approximate each ‘true’ ray by its nearest ray in the collection of all rays

10. Use Ray Approximations Perfect specular reflection – use closest approximate ray to the true reflected ray Diffuse reflection – use all rays through the intersection point

11. Use Ray Approximations Perfect specular reflection

12. Use Ray Approximations Perfect specular reflection – use closest approximate ray to the true reflected ray

13. Use Ray Approximations Perfect specular reflection – use closest approximate ray to the true reflected ray Diffuse reflection – use all rays through the intersection point

14. Spherical Angle Representation Base is rotated through ‘all’ spherical angles base z y x   Spherical angles vary ‘uniformly’ cover the hemisphere

15. Scene Covered Each box is one parallel subfield – a parallel subfield has all rays parallel total light field = set of all parallel subfields (i,j, ,  ) Vertical rays

16. Rendering into VLF is 2D Rasterisation Vertical subfield rendering is equivalent to rasterisation Any other angle result can be obtained by rotation – Standard hardware can be used geometry (rotation) raster-scan + z-buffer z y x

17. VLF Overview Rasterisation – Precomputes all ray object intersections to segment the rays Propagation – Propagates light out from the light sources until convergence Rendering – Uses the radiance stored in the ray segments to reconstruct an image

18. Tiling Approach – Exploit Coherence Divide each PSF into tiles, in 3D these are tubes For each PSF{ for each object propagate(); }

19. Tiling The N  N resolution of the PSF is divided into square tiles of resolution M  M (M<N) PSF

20. Tiling The N  N resolution of the PSF is divided into square tiles of resolution M  M (M<N) Each tile maintains a list of object representations: PSF

21. Tiling The N  N resolution of the PSF is divided into square tiles of resolution M  M (M<N) Each tile maintains a list of object representations: { u_short object_id; EdgeTable visibility_mask; Image radiance; Image unshot_radiance; } PSF Objects are sampled at full PSF resolution Images may be sampled at less than full resolution

22. PSF (S) tile Propagation run through each tile in each PSF, propagating and storing radiance for objects in the tiles

23. PSF (S) tile Propagation run through each tile in each PSF, propagating and storing radiance for objects in the tiles

24. PSF (S) tile Propagation run through each tile in each PSF, propagating and storing radiance for objects in the tiles

25. PSF (S) tile Propagation run through each tile in each PSF, propagating and storing radiance for objects in the tiles

26. PSF (S) tile PSF (R) For specular – compute new PSF and store unshot radiance for same surface in respective tile.

27. PSF (S) tile PSF (R) For diffuse – accumulate radiance surface map for distribution at next iteration.

28. PSF (S) tile PSF (R) For diffuse – accumulate radiance surface map for distribution at next iteration.

29. Propagation … No need to keep surfaces sorted within a tile Construct an ‘exchange buffer’ on the fly – For each surface can use z-buffer to compute – which other surfaces it can exchange energy with along a tile.

30. ‘Uniform’ Subdivisions on a Hemisphere Major issue is to get ‘uniform’ partition of a hemisphere into ‘data points’ p=(x,y,z) – These correspond to the PSF directions Problem – given any query point on the hemisphere find the nearest data point in (fast!) constant time. Can be done in constant time using image based method: – Slater, M. (2002) Constant Time Queries on Uniformly Distributed Points on a Hemisphere, Journal of Graphics Tools, 7(1):33-44.

31. Implementation Issues Getting good specular reflections – Propagation approximates radiance on diffuse and specular surfaces – To get exact specular reflections backwards ray trace to a diffuse surface: Radiance on the rays will be Inaccurate…

32. Implementation Issues Getting good specular reflections – Propagation approximates radiance on diffuse and specular surfaces – To get exact specular reflections backwards ray trace to a diffuse surface: Follow exact ray paths…

33. Implementation Issues Getting good specular reflections – Propagation approximates radiance on diffuse and specular surfaces – To get exact specular reflections backwards ray trace to a diffuse surface: Follow exact ray paths…

34. Implementation Issues Getting good specular reflections – Propagation approximates radiance on diffuse and specular surfaces – To get exact specular reflections backwards ray trace to a diffuse surface: until diffuse surface is hit…

35. No shadow rays Fast ray traversal Implementation Issues Getting good specular reflections – Propagation approximates radiance on diffuse and specular surfaces – To get exact specular reflections backwards ray trace to a diffuse surface: until diffuse surface is hit…

36. Rendering 4 Methods 1. Render only from ray-information in tiles 2. Texture mapping for diffuse surfaces 3. Backwards ray tracing for specular surfaces 4. Progressive (1 or 2 + 3 when still)

37. Results A simple Cornell type scene with 6 specular reflectors. Propagation Time = 16 hours. FPS =8.5

38. Light field(2.1 fps) Diffuse textures(9.8 fps) Raytracing (0.2 fps) H/W raytracing(19.2fps)

39. Office test scene

40. Caustic Test Scene Smits, B. and Jensen, H.W. (2000) Global Illumination Test Scenes; Tech. Rep. UUCS-00-013, Computer Science Department, University of Utah, June 2000. Using Light Field Original Test Image Propagation time = 33.53 hours FPS =18.2

41. Some Properties Propagation time is long (but only has to be done once). Memory requirement is large. Propagation time is quadratic in number of surfaces Memory is linear in numbers of polygons Smaller tile size results in less polygons per tile, reduces propagation time, increases memory Rendering is fast for global illumination.

42. Videos

43. Conclusions An experimental method Shows some promise for fast walkthrough for globally illuminated scenes Problems remain in huge memory requirements and long propagation times Further work….in progress.

44. Additional materials Tech papers Presentations Videos, images etc. http://www.cs.ucl.ac.uk/research/vr/Projects/VLF/Media/