Image and View Morphing [Beier and Neely ’92, Chen and Williams ’93, Seitz and Dyer ’96]

Overview  Image morphing can look interesting, but morphs don’t usually account for differences in viewpoint  “shape-preserving” means that each in-between image looks like the same object, but in a different orientation, position, etc.  Most morphs are not shape-preserving, especially if the viewpoint or object orientation changes  View morphing is an extension to image morphing that handles 3D projection, scene transformations, and changes in viewpoint

Existing Morphing Techniques Distort  Image-morphing is a class of techniques for producing transitions between images  Recall Beier-Neely  Does morphing preserve shape?  Do all the in-between images look real?  If “Yes”, then the morph is shape-preserving  In general, morphs are not shape-preserving  View morphing tries to produce shape-preserving morphs

Existing Morphing Techniques Distort Linear interpolation between two perspective views of a clock face.[1]

View Morphing  The view morphing technique has three steps:  Pre-warp of source and destination images  Morph using some existing technique, such as Beier-Neely  Post-warp to get interpolated image (in-between image)  View morphing requires:  Two images  Information about the projections

View Morphing Demo [2]

Symbol Glossary

View Morphing with Parallel Views  Suppose we have two images of the same scene where the viewpoint is translated parallel to the view plane  This is already shape-preserving  Seitz and Dyer offer a proof  But what about non-parallel views?

Image Reprojection  We can use image reprojection to change the gaze direction of an existing image  Assumes that the camera’s optical center doesn’t move  Reprojection can be done efficiently with an algorithm due to Wolberg [3].

Image Reprojection Formulae

View Morphing with Non-Parallel Views  Image reprojection can be used to make two new images where the views are parallel  Parallel view morphing can be used to generate in-between images in this parallel space  A post-warp stage is added to get the real in-between image

View Morphing with Non-Parallel Views [1]

View Morphing with Non-Parallel Views  I 0 and I 1 are endpoint images  Recall that H s is the matrix that describes the placement of the image plane in space.  Morph is done in three steps  Apply H 0 -1 to I 0 and H 1 -1 to I 1  Morph using parallel view technique (e.g. Beier-Neely or linear interpolation)  Apply H s to in-between image from previous step

View Morphing with Non-Parallel Views  How do we pick H s ?  Interpolate an angle of rotation, which can be determined from the normals of the image view planes  or  H s can be determined by interactively selecting four non-collinear corresponding points before and after the post-warp step at s = 0.5.

Does it work?  Yes... ... but there are limitations  Blurriness  Relies on other morphing technique as an intermediate step, so we’re also stuck with the limitations of the selected morphing technique (e.g. holes)

More View Morphing Demos [2]

Summary  View morphing:  Does a morph between two images where the viewpoint has changed  Can produce a realistic looking transition  Uses some other morphing technique as an intermediate step  Uses two pre-warps and a post-warp

References, Acknowledgements  [1] Seitz, Steven and Charles Dyer. View Morphing. Proceedings of SIGGRAPH 96  [2] Movies are all from Steven Seitz’s view morphing web site:  [3] Wolberg, George. Digital Image Warping. IEEE Computer Society Press, Los Alamitos, CA, Unfortunately out of print.