Panorama Photography and Multiperspective Imaging Szymon Rusinkiewicz, Tim Weyrich: Technology in Art and Cultural Heritage. Princeton Freshman Seminar 2006 Acknowledgment: some figures by B. Brown, P. Shilane, P. Debevec, A. Román and H. Lentsch

Overview Limitations of perspective projectionLimitations of perspective projection Cylindrical panoramasCylindrical panoramas Spherical panoramasSpherical panoramas Multiperspective imagingMultiperspective imaging

Limitations of Perspective Projection Field-of-view — lack of immersionField-of-view — lack of immersion

Limitations of Perspective Projection Field-of-view — lack of immersionField-of-view — lack of immersion Wide-angle imagingWide-angle imaging – Potential distortion – Extreme perspective does not look “natural”

Extreme Wide-Angle Up to 185° field-of-viewUp to 185° field-of-view Technically demandingTechnically demanding Prone to distortion and vignettingProne to distortion and vignetting

Limitations of Perspective Projection Field-of-view — lack of immersionField-of-view — lack of immersion Wide-angle imagingWide-angle imaging – Potential distortion – Extreme perspective does not look “natural” Viewer position fixedViewer position fixed

Changing Position of Viewer In real scenes we can move our viewpointIn real scenes we can move our viewpoint Changing viewpoint provides us withChanging viewpoint provides us with – Additional detail – Depth cues by parallaxis

Cylindrical Panoramas Address the problem ofAddress the problem of – Limited field-of-view – Distortion of wide-angle perspective Replace projection plane by cylindrical canvas:Replace projection plane by cylindrical canvas:

Cylindrical Panoramas

Panorama Paintings Large canvasesLarge canvases Sometimes within cylindrical buildingSometimes within cylindrical building Augmentation by scenery in front of the paintingAugmentation by scenery in front of the painting Bourbaki Panorama, Luzern, 1871

Panorama Acquisition Multiple imagesMultiple images – Rotating camera – Ideally: very thin stripe images Panorama by stitching images togetherPanorama by stitching images together Requires common center of projectionRequires common center of projection

Stitching Problems remainProblems remain – Lens distortion – “Wrong” projection within each image – Differences in color and contrast Stitching softwareStitching software – Compensates for lens distortion – Aligns images (registration) – Warps images into cylindrical domain – Smoothes out image boundaries – Adapts color and contrast

Catadioptric Cameras Mirror opticsMirror optics Single-shot acquisition of hemispherical panoramaSingle-shot acquisition of hemispherical panorama Raw image shows distortion…Raw image shows distortion…

Catadioptric Cameras Desired projection is obtained by un-distorting the acquired imageDesired projection is obtained by un-distorting the acquired image This includes cylindrical projection:This includes cylindrical projection:

Viewing Software Presents panorama as a perspective viewPresents panorama as a perspective view Interactive change of viewing parametersInteractive change of viewing parameters – Pan – Zoom Still restricted to single view-pointStill restricted to single view-point Commercial and free viewers existsCommercial and free viewers exists – Quicktime VR, Live Picture Viewer, …

Spherical Panoramas Restrictions of cylindrical projection remainRestrictions of cylindrical projection remain – Depiction of the “poles” (floor and ceiling) – Fixed view-point How would we capture floor and ceiling?How would we capture floor and ceiling? – Acquisition? – Representation?

Spherical Acquisition StitchingStitching Or: take an image of a mirrored sphere!Or: take an image of a mirrored sphere!

Representations Spherical reflection Projection onto cube faces

Viewing Spherical Panoramas Most viewers support spherical panoramasMost viewers support spherical panoramas Un-warp depending on representationUn-warp depending on representation Live Demo Live Demo

Multiperspective Imaging Artist create multiperspective paintingsArtist create multiperspective paintings Can we acquire multiperspective images?Can we acquire multiperspective images?

Multiperspective Imaging Experiment:Experiment: – Place a person on a revolving chair – Take a video of the head while person is revolving – From each video image, take central pixel column – Glue all columns together What do you get?What do you get?

Multiperspective Imaging

Can you modify a pinhole camera to create such an image?Can you modify a pinhole camera to create such an image? How do images look for general movements of the video camera?How do images look for general movements of the video camera?

Multiperspective Imaging

Pushbroom Fix the camera on a moving vehicleFix the camera on a moving vehicle Has been used to create movie back-dropsHas been used to create movie back-drops

Relation To Perspective Images Single-point Perspective Image Picture Surface Rays across top of image Ray directions on image corners Center of projection

Relation To Perspective Images Pushbroom multiperspective image Camera path

Cross-slits Images [Zomet 03 ]

Cross-slits Images Camera path

Cross-slits Images Camera path

Cross-slits Camera Replace pinhole by two slitsReplace pinhole by two slits Or simulate it using a video camera…Or simulate it using a video camera…

Cross-slits Images Cross-slits images [Zomet 03]Cross-slits images [Zomet 03] Zomet 2003

Cross-slits Images Appearance similar to central perspectiveAppearance similar to central perspective Allow for “impossible viewpoints”Allow for “impossible viewpoints” Zomet 2003

Generating Cross-slits Panoramas Work by A. Roman et al & 2006Work by A. Roman et al & 2006 Video Video