1. Sebastian Enrique Columbia University senrique@cs.columbia.edu Real-Time Rendering Using CUReT BRDF Materials with Zernike Polynomials CS6998 - Topics on Computational Vision and Graphics Apr 20 th, 2004

2. CS6998 Real-Time Rendering Using CUReT BRDF Materials with Zernike Polynomials Sebastian Enrique - Columbia University - senrique@cs.columbia.edu Apr 20 th, 2004 2 Problem As far as I know, real-time rendering of objects using its BRDF was not achieved yet. I decided to deal with this problem, with the goal to create a system with the following initial requirements: Allow rendering of any kind of mesh. Allow real-time viewpoint changes. Allow real-time lighting changes. Allow selection of material from a set of BRDF samples. Compute final colors per pixel.

3. CS6998 Real-Time Rendering Using CUReT BRDF Materials with Zernike Polynomials Sebastian Enrique - Columbia University - senrique@cs.columbia.edu Apr 20 th, 2004 3 Decisions and Simplifications Meshes should be made of triangles. Topology information of the mesh (vertices positions and indices of each triangle) should be completed with vertex normals. Viewpoint can be rotated and zoomed in and out, but not translated. Change of camera target –the origin of the coordinate system- is also not allowed. Perspective projection will be used. Illumination will be limited to a white distant point light source. This means that every point in the surface will be receiving light from the same direction. The user should be able to change this direction. CUReT public BRDF materials databse will be used. It contains 61 material samples. Final colors should be computed in the GPU using fragment shaders. We will not deal with shadows and interreflections.

4. CS6998 Real-Time Rendering Using CUReT BRDF Materials with Zernike Polynomials Sebastian Enrique - Columbia University - senrique@cs.columbia.edu Apr 20 th, 2004 4 CUReT 205 measurements of 61 materials under different light and view directions. A representation or interpolation should be used to get the BRDF of each material under novel light and view directions: Zernike polynomials (mapping of points on a hemisphere over the unit disk). It is a good representation for smooth BRDFs and scattered data. It is not so good for materials with high specular lobes. 55 coefficients for polynomials of order 8. 5 coefficients for polynomials of order 2, which is good for most of the materials in the database.

5. CS6998 Real-Time Rendering Using CUReT BRDF Materials with Zernike Polynomials Sebastian Enrique - Columbia University - senrique@cs.columbia.edu Apr 20 th, 2004 5 Per Vertex Implementation Colors are computed for each vertex in software and then passed to the graphics hardware using OpenGL. Pixel values in between vertices are interpolated using Gouraud shading. Zernike polynomials are evaluated with corresponding material coefficients to get the correct color depending on light and viewing directions every frame. Some costly operations on angles are precomputed and stored in a table to speed up real-time processing. Extra features of the application such as to show surface normals and N dot L type of rendering are done using pixel and vertex shaders.

6. CS6998 Real-Time Rendering Using CUReT BRDF Materials with Zernike Polynomials Sebastian Enrique - Columbia University - senrique@cs.columbia.edu Apr 20 th, 2004 6 Per Pixel Implementation Same operations done per vertex could be done per pixel using a fragment shader. Precomputed values can be passed as textures. Current implementation involves two 2D-textures (Zernike coefficients and precomputed operations on angles) and one 3D-texture (OpenGL 1.2 extensions required). Color computation for Zernike polynomials of order 2 (5 coefficients) requires only 1 rendering pass (1 pixel shader). To compute order 5 (55 coefficients) 8 passes are required with current Cg implementation (quite slow) in current graphics hardware. I haven’t finished software part + debugging of per pixel implementation.

7. CS6998 Real-Time Rendering Using CUReT BRDF Materials with Zernike Polynomials Sebastian Enrique - Columbia University - senrique@cs.columbia.edu Apr 20 th, 2004 7 Results Happy Buddha model: 32,328 vertices; 69,451 triangles – Per Vertex – 5 coefficients. Materials: Plant, Orange Peel, and Insulation, using same light direction / pose.

8. CS6998 Real-Time Rendering Using CUReT BRDF Materials with Zernike Polynomials Sebastian Enrique - Columbia University - senrique@cs.columbia.edu Apr 20 th, 2004 8 Results (cont.) Dragon model: 22,998 vertices; 47,794 triangles – Per Vertex – 5 coefficients. Material Rabbit Fur changing light direction (top) and view direction (bottom).

9. CS6998 Real-Time Rendering Using CUReT BRDF Materials with Zernike Polynomials Sebastian Enrique - Columbia University - senrique@cs.columbia.edu Apr 20 th, 2004 9 Results (cont.) Stanford Bunny model: 35,947 vertices; 69,451 triangles – Per Vertex – 5 coefficients. Material Sponge changing light direction and view direction in every image.

10. CS6998 Real-Time Rendering Using CUReT BRDF Materials with Zernike Polynomials Sebastian Enrique - Columbia University - senrique@cs.columbia.edu Apr 20 th, 2004 10 Results (cont.) Stanford Bunny model: 35,947 vertices; 69,451 triangles – Per Vertex – 5 coefficients. Left: rendered with Rug B material. Middle: showing surface normals. Right: wireframe using N dot L.

11. CS6998 Real-Time Rendering Using CUReT BRDF Materials with Zernike Polynomials Sebastian Enrique - Columbia University - senrique@cs.columbia.edu Apr 20 th, 2004 11 Conclusions Real-time rendering of BRDF materials using any mesh was implemented and possible. Images shown before were rendered per vertex with Zernike polynomials of order 2; for order 8, vertices and triangles should be reduced 4 times for real-time rendering in test machine (Pentium 4 3Gz 1Gb RAM nVidia GeForce FX5900). Other BRDF representations should be analyzed to render in real-time more specular BRDFs materials. Next step is to finish per pixel implementation. Natural extension for this is to use environment lighting. Shadows could be added using traditional techniques like shadow mapping. First, I must correct some problems like artifacts in grazing angles and not matching final colors with original CUReT rendered spheres (coefficients are not in good shape?).

12. CS6998 Real-Time Rendering Using CUReT BRDF Materials with Zernike Polynomials Sebastian Enrique - Columbia University - senrique@cs.columbia.edu Apr 20 th, 2004 12 The End Aknowledgments Ravi Ramamoorthi for CUReT materials Zernike coefficients and some Zernike polynomials code. The Stanford 3D Scanning Repository for models used. Georgia Institute of Technology for tools to manipulate PLY files. Questions? Thanks for listening...