1. Presented by : …….. Ramesh Raskar, Greg Welch and Henry Fuchs University of North Carolina at Chapel Hill Presented by : …….. Ramesh Raskar, Greg Welch and Henry Fuchs University of North Carolina at Chapel Hill Seamless Projection Overlaps using Image Warping and Intensity Blending

2. Acknowledgements Office of the Future group at UNC Chapel HillOffice of the Future group at UNC Chapel Hill Andrei State (sketches)Andrei State (sketches) SupportSupport – NSF Science and Technology Center for Graphics and Visualization, USA – Advanced Networks and Services,National Tele-Immersion Initiative – Defense Advanced Research Projects Agency, USA – Intel Corporation Office of the Future group at UNC Chapel HillOffice of the Future group at UNC Chapel Hill Andrei State (sketches)Andrei State (sketches) SupportSupport – NSF Science and Technology Center for Graphics and Visualization, USA – Advanced Networks and Services,National Tele-Immersion Initiative – Defense Advanced Research Projects Agency, USA – Intel Corporation

3. Seamless Projection Overlaps P1 P2 P3 P1 P2 P3 Display Surface Projectors Well Defined Display Configuration Irregular display and projector configuration Traditional Flexible Setup

4. Goal High Resolution, Wide Field of View DisplayHigh Resolution, Wide Field of View Display Overlapping front projection systemOverlapping front projection system Seamless displaySeamless display – Geometric Registration of overlaps – Intensity Blending near edges Irregular display surface configurationIrregular display surface configuration High Resolution, Wide Field of View DisplayHigh Resolution, Wide Field of View Display Overlapping front projection systemOverlapping front projection system Seamless displaySeamless display – Geometric Registration of overlaps – Intensity Blending near edges Irregular display surface configurationIrregular display surface configuration P1 P2 P3

5. Motivation : Office of the Future (UNC ‘98) Irregular surfacesIrregular surfaces No specific projector configurationNo specific projector configuration Irregular surfacesIrregular surfaces No specific projector configurationNo specific projector configuration

6. Other Spatially Immersive Displays 1. CAVE (UIC EVL) Non-overlapping projectionsNon-overlapping projections Flat displays wallsFlat displays walls Well-defined projection configurationWell-defined projection configuration 1. CAVE (UIC EVL) Non-overlapping projectionsNon-overlapping projections Flat displays wallsFlat displays walls Well-defined projection configurationWell-defined projection configuration

7. Other Spatially Immersive Displays (Cont.) 2. VisionDome (Alternate Realities) Single Projector with expensive opticsSingle Projector with expensive optics Well defined display surfacesWell defined display surfaces 2. VisionDome (Alternate Realities) Single Projector with expensive opticsSingle Projector with expensive optics Well defined display surfacesWell defined display surfaces

8. Other Spatially Immersive Displays (Cont.) 3. Cylindrical Screens (Trimension, Panoram) 3 Projectors with side-by-side rectangular overlap3 Projectors with side-by-side rectangular overlap Well defined display surfaceWell defined display surface Precise electro-mechanical setupPrecise electro-mechanical setup 3. Cylindrical Screens (Trimension, Panoram) 3 Projectors with side-by-side rectangular overlap3 Projectors with side-by-side rectangular overlap Well defined display surfaceWell defined display surface Precise electro-mechanical setupPrecise electro-mechanical setup

9. Office of the Future (UNC ‘98) Irregular surfacesIrregular surfaces No specific projector configurationNo specific projector configuration Irregular surfacesIrregular surfaces No specific projector configurationNo specific projector configuration

10. Office of the Future Configuration Projectors Single User Irregular Surfaces User Top View

11. Our Approach Calibration of display configuration using a video cameraCalibration of display configuration using a video camera Rendering using texture mapping hardwareRendering using texture mapping hardware Pair-wise registration and blending of projection overlapsPair-wise registration and blending of projection overlaps Calibration of display configuration using a video cameraCalibration of display configuration using a video camera Rendering using texture mapping hardwareRendering using texture mapping hardware Pair-wise registration and blending of projection overlapsPair-wise registration and blending of projection overlaps

12. Key Features +Display surfaces may not be flat walls +Projection axis not orthogonal to displays +Flexible projection configuration +Projector’s intrinsic or extrinsic parameters not needed –Generated image correct for a single ideal viewer’s location +Display surfaces may not be flat walls +Projection axis not orthogonal to displays +Flexible projection configuration +Projector’s intrinsic or extrinsic parameters not needed –Generated image correct for a single ideal viewer’s location

13. Calibration Step 1 : Geometric Registration Step 1 : Geometric Registration – Pre-distort images so that when projected they appear ◊Perspectively correct ◊Aligned with neighboring projection Step 2 : Intensity Normalization Step 2 : Intensity Normalization – Blend images from overlapping projections Step 1 : Geometric Registration Step 1 : Geometric Registration – Pre-distort images so that when projected they appear ◊Perspectively correct ◊Aligned with neighboring projection Step 2 : Intensity Normalization Step 2 : Intensity Normalization – Blend images from overlapping projections

14. Calibration Step 1 : Geometric Registration Warping Images For irregular display surfacesFor irregular display surfaces – Desired image is pre-warped – Warping function is found using a wide field of view camera (WFOV) For flat display surfacesFor flat display surfaces – Projected images related by a 3x2 transformation Warping Images For irregular display surfacesFor irregular display surfaces – Desired image is pre-warped – Warping function is found using a wide field of view camera (WFOV) For flat display surfacesFor flat display surfaces – Projected images related by a 3x2 transformation

15. Calibration Step 1 : Geometric Registration Warping Images for Irregular Display Surfaces Keep the WFOV camera at ideal viewer’s locationKeep the WFOV camera at ideal viewer’s location Project dots with the projector one by oneProject dots with the projector one by one Find mapping from projector image to camera imageFind mapping from projector image to camera image Invert mapping to find the warping functionInvert mapping to find the warping function Warping Images for Irregular Display Surfaces Keep the WFOV camera at ideal viewer’s locationKeep the WFOV camera at ideal viewer’s location Project dots with the projector one by oneProject dots with the projector one by one Find mapping from projector image to camera imageFind mapping from projector image to camera image Invert mapping to find the warping functionInvert mapping to find the warping function

16. Camera to Projector warping function Display Surface WFOV Camera I camera (u,v) Observing a point on display surface I camera (u,v) = Which projector pixel ?

17. Camera to Projector warping function P1 Display Surface Projector P2 I proj2 (x,y) I proj1 (x,y) Projector Which projector pixel illuminated the same point on display surface ? I camera (u,v) = I proj1 (x,y), I proj2 (x,y)

18. Calibration Step 2 : Intensity Normalization Blending Projected Images For irregular display surfacesFor irregular display surfaces Projection overlap observed in camera image spaceProjection overlap observed in camera image space Assign intensity weights for projector pixelsAssign intensity weights for projector pixels Blending Projected Images For irregular display surfacesFor irregular display surfaces Projection overlap observed in camera image spaceProjection overlap observed in camera image space Assign intensity weights for projector pixelsAssign intensity weights for projector pixels

19. Intensity in Projector Overlap P1 Display Surface Projector P2 I proj2 (x,y) I proj1 (x,y) Projector How to make intensity in overlap region same as everywhere else ? WFOV Camera I camera (u,v)

20. Intensity Normalization Camera Scanline Projected Intensity P1P2 P1P2 Resultant Intensity Before Normalization After Normalization Projected Intensity

21. Rendering Two pass rendering methodTwo pass rendering method First pass : Compute the desired imageFirst pass : Compute the desired image Second pass :Second pass : – Warping with standard OpenGL texture mapping – Intensity blending using Alpha channel of graphics hardware Two pass rendering methodTwo pass rendering method First pass : Compute the desired imageFirst pass : Compute the desired image Second pass :Second pass : – Warping with standard OpenGL texture mapping – Intensity blending using Alpha channel of graphics hardware

22. Results: correct view for one user Live Panoramic Video Image captured using a WFOV camera cluster

23. Summary Robust general purpose method to create seamless imagesRobust general purpose method to create seamless images General configuration of projectors and display surfacesGeneral configuration of projectors and display surfaces High-resolution wide-field of view display using cluster of projectorsHigh-resolution wide-field of view display using cluster of projectors No expensive optics or electro-mechanical setupNo expensive optics or electro-mechanical setup One-time calibration procedureOne-time calibration procedure Robust general purpose method to create seamless imagesRobust general purpose method to create seamless images General configuration of projectors and display surfacesGeneral configuration of projectors and display surfaces High-resolution wide-field of view display using cluster of projectorsHigh-resolution wide-field of view display using cluster of projectors No expensive optics or electro-mechanical setupNo expensive optics or electro-mechanical setup One-time calibration procedureOne-time calibration procedure

24. Applications Office Scenarios, flexible setupOffice Scenarios, flexible setup Dome shaped displays using cluster of projectors instead of expensive opticsDome shaped displays using cluster of projectors instead of expensive optics Ease of setup for cylindrical projection screens using multiple projectorsEase of setup for cylindrical projection screens using multiple projectors Synthetic images or Video images on large display environmentsSynthetic images or Video images on large display environments Office Scenarios, flexible setupOffice Scenarios, flexible setup Dome shaped displays using cluster of projectors instead of expensive opticsDome shaped displays using cluster of projectors instead of expensive optics Ease of setup for cylindrical projection screens using multiple projectorsEase of setup for cylindrical projection screens using multiple projectors Synthetic images or Video images on large display environmentsSynthetic images or Video images on large display environments

25. Ten Years from now.. Light bulbs replaced by cheap projectorsLight bulbs replaced by cheap projectors Sufficient number of inexpensive camerasSufficient number of inexpensive cameras Widely available texture mapping graphics hardwareWidely available texture mapping graphics hardware Projectors, LCD panels to create ‘desktop’ instead of looking at a 21’ monitorProjectors, LCD panels to create ‘desktop’ instead of looking at a 21’ monitor Light bulbs replaced by cheap projectorsLight bulbs replaced by cheap projectors Sufficient number of inexpensive camerasSufficient number of inexpensive cameras Widely available texture mapping graphics hardwareWidely available texture mapping graphics hardware Projectors, LCD panels to create ‘desktop’ instead of looking at a 21’ monitorProjectors, LCD panels to create ‘desktop’ instead of looking at a 21’ monitor