Representations of Visual Appearance COMS 6160 [Spring 2007], Lecture 4 Image-Based Modeling and Rendering

Motivation for Lecture  IBR was initial catalyst for much of work in course  Especially light field rendering papers from 1996 (presented today), that first showed 4D practical  Today, even purely synthetic real-time rendering methods based on precomputation inspired by IBR  In this course, we are mostly interested not in “pure” IBR, but rather in use of measured/sampled datasets

Next few slides courtesy Paul Debevec; SIGGRAPH 99 course notes

Pure IBR: Pros and Cons  Advantages  Easy to capture images: photorealistic by definition  Simple, universal representation  Often bypass geometry estimation?  Independent of scene complexity?  Disadvantages  WYSIWYG but also WYSIAYG  Explosion of data as flexibility increased  Often discards intrinsic structure of model?  In this course  Use measured datasets of appearance functions  Not interested in geometry or proxies for it that much  IBR is use of sampled representations  Key question is compact high-dimensional representations

IBR: A brief history  Texture maps, bump maps, env. maps [70s]  First 2D representations from angle of course  Poggio et al. MIT: Faces, image-based analysis/synthesis  Modern Era  Chen Williams 93, View Interpolation [Images with depth]  Chen 95 Quicktime VR [Images from many viewpoints]  McMillan Bishop 95 Plenoptic Modeling [Images w disparity]  Gortler et al, Levoy and Hanrahan 96 Light Fields [4D]  Started revolution in acquired appearance  Shade et al. 98 Layered Depth Images [2.5D]  Debevec et al. 00 Reflectance Field [4D]

Outline  Overview of IBR  Basic approaches  Image Warping  Light Fields  Survey of some recent work

Warping slides courtesy Leonard McMillan

Outline  Overview of IBR  Basic approaches  Image Warping  [2D + depth. Requires correspondence/disparity]  Light Fields [4D]  Survey of some recent work

Outline  Overview of IBR  Basic approaches  Image Warping  [2D + depth. Requires correspondence/disparity]  Light Fields [4D]  Survey of some recent work

Historical Timeline: Details 4D  BRDFs have a long history (70 s – 80 s ), but work on measuring them is accelerating in last 10 years  Light Fields and more general 4D functions  Reflectance fields (relighting faces) Debevec et al. 00  Surface Light Fields (Wood et al. 00 ; Nishino et al. 99)  Incident Light Fields (Unger et al. 03 ; Goesele et al. 03)  Heterogeneous Subsurface Scattering (Peers et al. 06)

Surface Light Fields  Miller 98, Nishino 99, Wood 00  Reflected light field (lumisphere) on surface  Explicit geometry as against light fields. Easier compress

Acquiring Reflectance Field of Human Face [Debevec et al. SIGGRAPH 00] Illuminate subject from many incident directions

Example Images Images from Debevec et al. 00