1. Real-Time High Quality Rendering COMS 6160 [Fall 2004], Lecture 1 Ravi Ramamoorthi http://www.cs.columbia.edu/~ravir/6160

2. Motivation  Today, we create photorealistic computer graphics  Complex geometry, realistic lighting, materials, shadows  Computer-generated movies/special effects (difficult to tell real from rendered…)  But algorithms are very slow (hours to days)

3. Real-Time Rendering  Goal is interactive rendering. Critical in many apps  Games, visualization, computer-aided design, …  So far, focus on complex geometry  Chasm between interactivity, realism

4. Evolution of 3D graphics rendering Interactive 3D graphics pipeline as in OpenGL  Earliest SGI machines (Clark 82) to today  Most of focus on more geometry, texture mapping  Some tweaks for realism (shadow mapping, accum. buffer) SGI Reality Engine 93 (Kurt Akeley)

5. Offline 3D Graphics Rendering Ray tracing, radiosity, photon mapping  High realism (global illum, shadows, refraction, lighting,..)  But historically very slow techniques “So, while you and your children’s children are waiting for ray tracing to take over the world, what do you do in the meantime?” Real-Time Rendering Pictures courtesy Henrik Wann Jensen

6. New Trend: Acquired Data  Image-Based Rendering: Real/precomputed images as input  Also, acquire geometry, lighting, materials from real world  Easy to obtain or precompute lots of high quality data. But how do we represent and reuse this for (real-time) rendering?

7. 5 years ago  High quality rendering: ray tracing, global illum., image-based rendering (This is what a rendering course would cover)  Real-Time rendering: Interactive 3D geometry with simple texture mapping, maybe fake shadows (OpenGL, DirectX)  Complex environment lighting, real materials (velvet, satin, paints), soft shadows, caustics often omitted in both  No COMS 6160: Realism, interactivity at cross purposes

8. Outline of Lecture  Background  Motivation, effects for high quality real-time rendering  Recent technological, algorithmic developments  Preview of results currently possible  Logistics of course

9. High quality real-time rendering  Photorealism, not just more polygons  Natural lighting, materials, shadows Interiors by architect Frank Gehry. Note rich lighting, ranging from localized sources to reflections off vast sheets of glass.

10. High quality real-time rendering  Photorealism, not just more polygons  Natural lighting, materials, shadows Real materials diverse and not easy to represent by simple parameteric models. Want to support measured reflectance. Glass Vase Glass Star (courtesy Intel) Peacock feather

11. High quality real-time rendering  Photorealism, not just more polygons  Natural lighting, materials, shadows Natural lighting creates a mix of soft diffuse and hard shadows. Agrawala et al. 00 small area light, sharp shadows soft and hard shadows Ng et al. 03

12. Demo

13. Applications  Entertainment: Lighting design  Architectural visualization  Material design: Automobile industry  Realistic Video games  Electronic commerce

14. Outline of Lecture  Background  Motivation, effects for high quality real-time rendering  Recent technological, algorithmic developments  Preview of results currently possible  Logistics of course

15. 5 years ago  High quality rendering: ray tracing, global illum., image-based rendering (This is what a rendering course would cover)  Real-Time rendering: Interactive 3D geometry with simple texture mapping, maybe fake shadows (OpenGL, DirectX)  Complex environment lighting, real materials (velvet, satin, paints), soft shadows, caustics often omitted in both  No COMS 6160: Realism, interactivity at cross purposes

16. Today  Vast increase in CPU power, modern instrs (SSE, SIMD, …)  Real-time raytracing techniques are possible  4 th generation of graphics hardware is programmable  (First 3 gens were wireframe, shaded, textured)  Modern nVidia, ATI cards allow vertex, fragment shaders  Great deal of current work on acquiring and rendering with realistic lighting, materials… [Especially at Columbia]  Goals of COMS 6160: Understand these new ideas for high- quality rendering, and recent literature (last 5 years)  (This is not a course about real-time rendering with complex geometry. It is primarily about high quality shading effects)

17. Outline of Lecture  Background  Motivation, effects for high quality real-time rendering  Recent technological, algorithmic developments  Preview of results currently possible  Programmable graphics hardware  Precomputation-based methods  Interactive RayTracing  Logistics of course

18. Programmable Graphics Hardware Nvidia dusk demo http://www.nvidia.com/object/demo_dusk.html http://www.nvidia.com/object/demo_dusk.html

19. Precomputation-Based Methods  Static geometry  Precomputation  Real-Time Rendering (relight all-frequency effects)  Involves sophisticated representations, algorithms

20. Relit Images Ng, Ramamoorthi, Hanrahan 04

21. Video: Real-Time Relighting Followed by demo with real-time relighting with caustics

22. Interactive RayTracing Advantages  Very complex scenes relatively easy (hierarchical bbox)  Complex materials and shading for free  Easy to add global illumination, specularities etc. Disadvantages  Hard to access data in memory-coherent way  Many samples for complex lighting and materials  Global illumination possible but expensive Modern developments: Leverage power of modern CPUs, develop cache-aware, parallel implementations

23. Interactive Ray-Tracing Video

24. Outline of Lecture  Background  Motivation, effects for high quality real-time rendering  Recent technological, algorithmic developments  Preview of results currently possible  Logistics of course

25. Course Goals, Format  Goal: Background and current research on high- quality real-time rendering in computer graphics  Columbia is one of the best places for this!!  Format: Alternate lectures, student presentations of papers  http://www.cs.columbia.edu/~ravir/6160/

26. Course Logistics  No textbooks. Required readings are papers available online (except 2 handouts today)  The book “Real-Time Rendering (2 nd ed)” by Moller and Haines may be helpful (though we will not follow it closely)  Office hours: before class. My contact info is on my webpage: http://www.cs.columbia.edu/~ravir  Counts for PhD elective breadth in graphics/HCI, MS elective for vision/graphics, BS elective etc.

27. Requirements  Pass-Fail  Show up to class regularly  Present 2 (maybe 1 if enough people) paper(s)  Prefer you do this rather than just sit in  Grades  Attend class, participate in discussions (10%)  Present 3 (maybe 2 if enough people) papers (30%)  Project (60%)

28. Project  Wide flexibility if related to course. Can be done in groups  Default: Implement (part of) one of papers and produce an impressive real-time high quality rendering demo  See/e-mail me re ideas  Best projects will go beyond simple implementation (try something new, some extensions)  Alternative (less desirable): Summary of 3+ papers in an area  Best projects will explore links/framework not discussed by authors, and suggest future research directions

29. Prerequisites  Strong interest in graphics, rendering  Computer graphics experience (4160)  What if lacking prerequisites? Next slide  Course will move quickly  Covering recent and current active research  Some material quite technical  Assume some basic knowledge  Many topics. Needn’t fully follow each one, but doing so will be most rewarding.

30. If in doubt/Lack prerequisites  Material is deep, not broad  May be able to pick up background quickly  Course requirements need you to really fully understand only one/two areas (topics)  But if completely lost, won’t be much fun  If in doubt, see if you can more or less follow some of papers after background reading  Ultimately, your call

31. Assignment this week  E-mail me (ravir@cs)  Name, e-mail, status (Senior, PhD etc.)  Will you be taking course grades or P/F  Background in graphics/any special comments  Optional: Papers you’d like to present FCFS  Paper presenters for next week [You (may) get a one-paper reduction in load]  Hardware: Geometry Engine, Reality-Engine

32. Questions?