Computer Graphics Inf4/MSc Computer Graphics Lecture Notes #16 Image-Based Modelling, Rendering and Lighting.

Slides:

Advertisements

Similar presentations

SJS SDI_161 Design of Statistical Investigations Stephen Senn Random Sampling I.

Advertisements

Lecture 161 Macroeconomic Analysis 2003 Monetary Policy: Transmission Mechanism

1GR2-00 GR2 Advanced Computer Graphics AGR Lecture 18 Image-based Rendering Final Review of Rendering What We Did Not Cover Learning More...

Organizational Behavior, 8e Schermerhorn, Hunt, and Osborn

Chapter 16 Aromatic Compounds Jo Blackburn Richland College, Dallas, TX Dallas County Community College District  2003,  Prentice Hall Organic Chemistry,

Light Fields PROPERTIES AND APPLICATIONS. Outline  What are light fields  Acquisition of light fields  from a 3D scene  from a real world scene 

16421: Vision Sensors Lecture 6: Radiometry and Radiometric Calibration Instructor: S. Narasimhan Wean 5312, T-R 1:30pm – 2:50pm.

Dana Cobzas-PhD thesis Image-Based Models with Applications in Robot Navigation Dana Cobzas Supervisor: Hong Zhang.

Image-Based Modeling, Rendering, and Lighting

Advanced Computer Graphics CSE 190 [Spring 2015], Lecture 14 Ravi Ramamoorthi

18.1 Si31_2001 SI31 Advanced Computer Graphics AGR Lecture 18 Image-based Rendering Light Maps What We Did Not Cover Learning More...

Advanced Computer Graphics (Fall 2010) CS 283, Lecture 16: Image-Based Rendering and Light Fields Ravi Ramamoorthi

Advanced Computer Graphics (Spring 2005) COMS 4162, Lecture 21: Image-Based Rendering Ravi Ramamoorthi

Representations of Visual Appearance COMS 6160 [Spring 2007], Lecture 4 Image-Based Modeling and Rendering

Image-Based Modeling and Rendering CS 6998 Lecture 6.

Image-Based Rendering Computer Vision CSE576, Spring 2005 Richard Szeliski.

Image-Based Lighting : Computational Photography Alexei Efros, CMU, Fall 2005 © Eirik Holmøyvik …with a lot of slides donated by Paul Debevec.

View interpolation from a single view 1. Render object 2. Convert Z-buffer to range image 3. Re-render from new viewpoint 4. Use depths to resolve overlaps.

Image-Based Rendering Produce a new image from real images. Combining images Interpolation More exotic methods.

Image-based Rendering of Real Objects with Complex BRDFs.

Copyright  Philipp Slusallek IBR: View Interpolation Philipp Slusallek.

Image or Object? Michael F. Cohen Microsoft Research.

CSCE 641 Computer Graphics: Image-based Rendering (cont.) Jinxiang Chai.

Siggraph’2000, July 27, 2000 Jin-Xiang Chai Xin Tong Shing-Chow Chan Heung-Yeung Shum Microsoft Research, China Plenoptic Sampling SIGGRAPH’2000.

Computational Photography Light Field Rendering Jinxiang Chai.

CSCE 641: Computer Graphics Image-based Rendering Jinxiang Chai.

 Marc Levoy IBM / IBR “The study of image-based modeling and rendering is the study of sampled representations of geometry.”

 Marc Levoy IBM / IBR “The study of image-based modeling and rendering is the study of sampled representations of geometry.”

View interpolation from a single view 1. Render object 2. Convert Z-buffer to range image 3. Re-render from new viewpoint 4. Use depths to resolve overlaps.

Image Based Rendering And Modeling Techniques And Their Applications Jiao-ying Shi State Key laboratory of Computer Aided Design and Graphics Zhejiang.

The Story So Far The algorithms presented so far exploit: –Sparse sets of images (some data may not be available) –User help with correspondences (time.

Computer Graphics Inf4/MSc Computer Graphics Lecture Notes #16 Image-Based Lighting.

Real-Time High Quality Rendering CSE 291 [Winter 2015], Lecture 6 Image-Based Rendering and Light Fields

Technology and Historical Overview. Introduction to 3d Computer Graphics  3D computer graphics is the science, study, and method of projecting a mathematical.

Computer Visualization BIM Curriculum 03. Topics  History  Computer Visualization Methods  Visualization Workflow  Technology Background.

Advanced Computer Graphics (Spring 2013) CS 283, Lecture 15: Image-Based Rendering and Light Fields Ravi Ramamoorthi

Image-Based Rendering. 3D Scene = Shape + Shading Source: Leonard mcMillan, UNC-CH.

Dynamically Reparameterized Light Fields Aaron Isaksen, Leonard McMillan (MIT), Steven Gortler (Harvard) Siggraph 2000 Presented by Orion Sky Lawlor cs497yzy.

Image-based rendering Michael F. Cohen Microsoft Research.

Image-based Rendering. © 2002 James K. Hahn2 Image-based Rendering Usually based on 2-D imagesUsually based on 2-D images Pre-calculationPre-calculation.

03/24/03© 2003 University of Wisconsin Last Time Image Based Rendering from Sparse Data.

Image Based Rendering. Light Field Gershun in 1936 –An illuminated objects fills the surrounding space with light reflected of its surface, establishing.

Image-Based Lighting © Eirik Holmøyvik …with a lot of slides donated by Paul Debevec CS194: Image Manipulation & Computational Photography Alexei Efros,

What is light? Electromagnetic radiation (EMR) moving along rays in space R( ) is EMR, measured in units of power (watts) – is wavelength Light: Travels.

Rendering Synthetic Objects into Real Scenes: Bridging Traditional and Image-based Graphics with Global Illumination and High Dynamic Range Photography.

03/09/05© 2005 University of Wisconsin Last Time HDR Image Capture Image Based Rendering –Improved textures –Quicktime VR –View Morphing NPR Papers: Just.

Image-Based Rendering A Brief Overview David Luebke University of Virginia.

The Plenoptic Function Lázaro Hermoso Beltrán. 2 Previous Concepts “The body of the air is full of an infinite number of radiant pyramids caused by the.

Image-Based Rendering of Diffuse, Specular and Glossy Surfaces from a Single Image Samuel Boivin and André Gagalowicz MIRAGES Project.

CSL 859: Advanced Computer Graphics Dept of Computer Sc. & Engg. IIT Delhi.

Panorama artifacts online –send your votes to Li Announcements.

Interreflections : The Inverse Problem Lecture #12 Thanks to Shree Nayar, Seitz et al, Levoy et al, David Kriegman.

Announcements Office hours today 2:30-3:30 Graded midterms will be returned at the end of the class.

- Laboratoire d'InfoRmatique en Image et Systèmes d'information

112/5/ :54 Graphics II Image Based Rendering Session 11.

Local Illumination and Shading

CSCE 641 Computer Graphics: Image-based Rendering (cont.) Jinxiang Chai.

Image-Based Rendering Geometry and light interaction may be difficult and expensive to model –Think of how hard radiosity is –Imagine the complexity of.

Radiometry of Image Formation Jitendra Malik. A camera creates an image … The image I(x,y) measures how much light is captured at pixel (x,y) We want.

CS559: Computer Graphics Lecture 36: Raytracing Li Zhang Spring 2008 Many Slides are from Hua Zhong at CUM, Paul Debevec at USC.

Sub-Surface Scattering Real-time Rendering Sub-Surface Scattering CSE 781 Prof. Roger Crawfis.

Radiometry of Image Formation Jitendra Malik. What is in an image? The image is an array of brightness values (three arrays for RGB images)

Advanced Computer Graphics

Advanced Computer Graphics

donated by Paul Debevec

Image-Based Rendering

3D Graphics Rendering PPT By Ricardo Veguilla.

© 2005 University of Wisconsin

Common Classification Tasks

Image Based Modeling and Rendering (PI: Malik)

Presentation transcript:

Computer Graphics Inf4/MSc Computer Graphics Lecture Notes #16 Image-Based Modelling, Rendering and Lighting

Computer Graphics Inf4/MSc 13/11/2007Lecture Notes #162 Global Illumination and Image-Based Lighting Traditional Computer Graphics involves: –Modelling with matter: geometry with reflectance properties. Image-Based Modelling & Rendering is: –Modelling and rendering with Light, often deriving geometry and materials in the process. Image-Based Lighting allows: –Combination of real and synthetic graphics with consistent illumination, using images as light sources.

Computer Graphics Inf4/MSc 13/11/2007Lecture Notes #163 Pursuit of Photorealism For complex models, tools can require lot of effort and rendering is very slow. Choice: do usual modelling or just take a photo (that would provide photorealism!) IBMR is about using these photos and transcending their limitations (eg. Light). Don’t need anything like a complete model, but may need a number of photographs…

Computer Graphics Inf4/MSc 13/11/2007Lecture Notes #164 Pursuit of Photorealism Need enough photographs to give coverage. Major result is - rendering is faster. So Image-Based Modelling and Rendering (IBMR) starts with the answer! Research inspired by film industry. Related to techniques in Computer Vision. New so no taxonomy, just techniques.

Computer Graphics Inf4/MSc 13/11/2007Lecture Notes #165 IBMR Techniques Panoramas – movement within and between panoramic cylinders, eg. Quicktime VR. –Panoramas can be real or synthetic. View interpolation –from a few photos, interpolate view from any position (morphing). * Lighting – adding real or synthetic objects to a photographed scene, illuminated with scene lights. Modelling – extracting models from photos.

Computer Graphics Inf4/MSc 13/11/2007Lecture Notes #166 “Traditional graphics”

Computer Graphics Inf4/MSc 13/11/2007Lecture Notes #167 Computer Vision

Computer Graphics Inf4/MSc 13/11/2007Lecture Notes #168 Vision meets graphics

Computer Graphics Inf4/MSc 13/11/2007Lecture Notes #169 However... Vision falls short

Computer Graphics Inf4/MSc 13/11/2007Lecture Notes #1610 And... … so does graphics

Computer Graphics Inf4/MSc 13/11/2007Lecture Notes #1611 Image-Based Rendering (IBR)

Computer Graphics Inf4/MSc 13/11/2007Lecture Notes #1612 What is an image? Collection of radiance values radiance along a ray 3D position 2D direction

Computer Graphics Inf4/MSc 13/11/2007Lecture Notes #1613 Plenoptic function Radiance value for all possible rays = plenoptic function. All possible images are a subset of this plenoptic function. Too much stuff! Goal of IBR is generate a continuous representation of the plenoptic function.

Computer Graphics Inf4/MSc 13/11/2007Lecture Notes #1614 Plenoptic function Radiance is constant along a ray (line) 2D position 2D direction Once we know one ‘origin’, we know them all

Computer Graphics Inf4/MSc 13/11/2007Lecture Notes #1615 What is an image? Image = rays going through one point usually restricted to viewing frustum, but can also be panoramic

Computer Graphics Inf4/MSc 13/11/2007Lecture Notes #1616 What is an object? Image = rays going through one point + image plane 2D function (position on image plane)

Computer Graphics Inf4/MSc 13/11/2007Lecture Notes #1617 What is an object? Outgoing radiance field of an object 2D function (position on surface)

Computer Graphics Inf4/MSc 13/11/2007Lecture Notes #1618 What is an object? All light leaving the object

Computer Graphics Inf4/MSc 13/11/2007Lecture Notes #1619 What is an object? All light leaving the object 4D function (2D position + 2D direction)

Computer Graphics Inf4/MSc 13/11/2007Lecture Notes #1620 What is an object? All possible images of an object

Computer Graphics Inf4/MSc 13/11/2007Lecture Notes #1621 What is an object? All possible images of an object

Computer Graphics Inf4/MSc 13/11/2007Lecture Notes #1622 What is an object? We don’t really need the object

Computer Graphics Inf4/MSc 13/11/2007Lecture Notes #1623 What is an object? We don’t really need the object

Computer Graphics Inf4/MSc 13/11/2007Lecture Notes #1624 Lumigraph / Light Field Object is only defined by its radiance field stuff 4D function (Levoy - Cohen et al 96)

Computer Graphics Inf4/MSc 13/11/2007Lecture Notes #1625 Lumigraph - capture (Stanford - Levoy et al.)

Computer Graphics Inf4/MSc 13/11/2007Lecture Notes #1626 Layered Depth Images (McMillan) Problem with one photo is “holes” when view point moved: - this is why a number of photos are generally required - fill the holes using intensities of neighbouring pixels - interpolation …

Computer Graphics Inf4/MSc 13/11/2007Lecture Notes #1627 Image Based Lighting Add models or objects to scenes and allow them to be manipulated in the scenes. Modelling with light allows added objects to be illuminated consistent with image existent lighting - photorealistic inclusions. We start with the answer by finding the scene illumination.

Computer Graphics Inf4/MSc 13/11/2007Lecture Notes #1628 Image Based Lighting Real Scene Goal: place synthetic objects on table (all pictures P. Debevec 98-99)

Computer Graphics Inf4/MSc 13/11/2007Lecture Notes #1629 Extracting scene lighting Capture illumination using illumination sphere

Computer Graphics Inf4/MSc 13/11/2007Lecture Notes #1630 Image Based Lighting Real scene

Computer Graphics Inf4/MSc 13/11/2007Lecture Notes #1631 Image Based Lighting captured illumination field

Computer Graphics Inf4/MSc 13/11/2007Lecture Notes #1632 Image Based Lighting Real scene light based model local scene synthetic objects

Computer Graphics Inf4/MSc 13/11/2007Lecture Notes #1633 Image Based Lighting Use renderer - compute effects of synthetic objects on local scene local scene (brdf estimated)synthetic objects (brdf known) light based model

Computer Graphics Inf4/MSc 13/11/2007Lecture Notes #1634 Image Based Lighting Render into the scene background

Computer Graphics Inf4/MSc 13/11/2007Lecture Notes #1635 Image Based Lighting Render synthetic objects

Computer Graphics Inf4/MSc 13/11/2007Lecture Notes #1636 Image Based Lighting Effect of local scene on real scene

Computer Graphics Inf4/MSc 13/11/2007Lecture Notes #1637 Image Based Lighting Add differences to image

Computer Graphics Inf4/MSc 13/11/2007Lecture Notes #1638 Reconstruction from images? Computer Vision related. Constructs new views and extracts models. Epipolar geometry –expresses relationships between points in different images. Difficult to predict full impact in CG. State of the Art –3D reconstruction from uncalibrated images.

Computer Graphics Inf4/MSc 13/11/2007Lecture Notes #1639 References You won’t find this material in the major graphics texts since it is new and not yet mainstream but I recommend the following: –The Computer Image, Watt & Policarpo, Addison- Wesley –SIGGRAPH courses for the past few years, we have some on CD rom and on-line within EdVEC and not immediately available to Informatics machines – ask me if you have a strong interest and need.