1. CS 563 Advanced Topics in Computer Graphics Introduction To IBR By Cliff Lindsay Slide Show ’99 Siggraph[6]

2. What Is IBR?  IBR:  Multidisciplinary field that includes computer vision and graphics  Techniques that replace and/or augment polygon models  Primary data pre-rendered images and photographs as input The Rendering Spectrum [3]

3. Where Did IBR Come From?  Photo-realism  Modeling ability has been stifled by rendering advancements  Availability of inexpensive digital image acquisition hardware  Recent graphics accelerators trends  Necessity to render object that can’t be rendered using polygons

4. Tenants and Common Techniques of IBR  Rendering time is decoupled from scene complexity  Images are used as input  Exploits coherence  Pre-calculation of scene data/images

5. Computer Vision  Computer Graphics = IBR. Combining CG and Computer Vision  Vision Technology Lacks robust Algorithms  The Graphics Industry Needs better modelling Siggraph ’99 Course on Image-based Rendering[6]

6. How Does IBR Compare to Traditional Rendering?  Image Warping Vs. Matrix Transformations  Perspective Division Vs. Projective Normalization  What the !@#$ Is A Splat Kernel? [1] Image from Leonard McMillan

7. Approaches  2D Approaches:  Texture Mapping  Sprite, Billboards, and Impostors  Image Layering  3D approaches:  LDI (2.5D)  View Interpolation & Morphing  Mosaics  4D approaches:  The Lumigraph  Light Fields

8. Texturing Mapping  Texture Space (u,v)  3D Object Space (x 0, y 0, z 0 )  Screen Space (x, y) [5]  Texture mapping has close ties to Image Warping  Wide Industry Support (hardware and Software)  Filtering

9. Sprites, Billboards, and Impostors (Oh my!)  Sprites:  Pure 2D image  No warping, or projection (like mouse cursor)  Billboards:  Sprite applied to a polygon  Alpha channel usually employed  Uses texture mapping for acceleration  Impostors:  Billboards created on the fly.  Can represent complex models  Error metric associated w/ changed views

10. Billboards  Billboards:  Oriented toward viewer  Matrix transformations (classical pipeline)  Special effects (lens flares, laser/light bursts, etc)  Hard to render objects (clouds, fire, smoke)

11. Impostors  Impostor Techniques:  Error Angle  Off Screen Rendering  Polygon Texturing  Texture resolution need not exceed screen resolution  texres = screenres * objsize/(2 * distance * tan(fov/2))

12. Billboard Example

14. Lumigraph/Light Fields 0 Plenoptic function  An image is a collection of radiance values a long a ray.  Radiance value for all possible rays = Plenoptic function  4D (for our purpose)  [7]

15. Lumigraph/Light Fields 1  Represent an object by it’s extents  Each point on a cube has multiple rays eminating.  Each wall has 2 planes (12 planes make a cube)

16. Lumigraph/Light Fields 2  You parameterize a ray using the 2 planes  L(s, t, u, v) = radiance for a ray  Ray – plane intersection make it easy and fast

17. Lumigraph/Light Fields 3  Sample of the objects on the plane are not continuos  Gaps are Created [10]

18. Lumigraph/Light Fields 4  Continuos luminance is a linear sum  B – basis function for which we can calculate at grid points  If we use a constant value, the coefficient take on the values of the grid points [10]

19. Lumigraph/Light Fields 5 [10]

20. Lumigraph/Light Fields 6 Example Rendering [10]

21. View Interpolation Reference Image 1Reference Image 2 Corresponding Pixels Morph maps Based on diagrams from Watt[8] View Morphing - more to come next presentation!

22. View Morphing View Morphing - more to come next presentation! View Morphing[9]

23. View Morphing 11 2 2 3 View Morphing - more to come next presentation! View Morphing[9]

24. Recent Developments & The Future of IBR  Surface Light fields  High Dynamic Range Radiance Maps  View-dependent texture-mapping (VDTM)  IBO (Image Based Objects)

25. Conclusion  Rendering time is decoupled from scene complexity  Images are used as input  Pre-calculation of scene data/images

26. Additional Resources  http://citeseer.nj.nec.com/cs - NEC Digital Library http://citeseer.nj.nec.com/cs  http://www.siggraph.org http://www.siggraph.org  http://www.debevec.org/ (View Morphing, High Dynamic Range Radiance Maps, Projective Texture- Mapping) http://www.debevec.org/  http://www-2.cs.cmu.edu/%7Eph/869/www/misc.html (a cool site with a bunch of IBR links) http://www-2.cs.cmu.edu/%7Eph/869/www/misc.html  http://www.peter-oel.de/ibmr-focus/ (Another cool site) http://www.peter-oel.de/ibmr-focus/

27. References [1] McMillan, Leonard, “An Image-Based Approach to Three-Dimensional Computer Graphics ”,, http://graphics.laces.mitt.edu/~mcmillan/IBRwork/defense23.html, date unknown, Cited slide #6. [2] McMillan, Leonard, Gortler, Steven, “Applications of Computer Vision to Computer Graphics”, ACM Siggraph, Vol. 33 no. 4, Nov. 99 [3] Akenine-Moller, Tomas, Haines, Eric, “Real-Time Rendering, 2 nd Edition”, A K Peters, 2002 [4] Watt, Alan, “3D Computer Games”, Addison-Wesley Pub Co, Volume 1, 2nd edition, 1999 [5] Heckbert, Paul S., “Survey of Texture Mapping,” IEEE Computer Graphics & Applications, Cited slide #10, November 1986, [6] Cohen, Michael, “Course on Image-based, Modeling, Rendering, and Lighting”, Siggraph ‘99 [7] Mcmillian, Leonard, Bishop, Gary, “Plenoptic Modeling: An Image-Based Rendering System”, Proceedings of SIGGRAPH 95, (Los Angeles, CA August 6-11, 1995), pp. 39-46 [8] Watt, Alan, “3D Computer Graphics”, Addison-Wesley Pub Co, 3nd edition (), 2000 [9] Chen, S.E., Williams L., “View Interpolation for Image Synthesis”, ACM Siggraph ’95 [10]Gortler, S, Cohen, M, Girzesczuk, R, Szeliski, R, “The Lumigraph”, ACM Siggraph, 1996