Static Image Mosaicing Amin Charaniya (amin@cse.ucsc.edu) EE 264: Image Processing and Reconstruction

Presentation Overview Problem definition Background Literature Survey Image transformations Image Registration Coarse Image registration Transformation Optimization Image Blending Implementation and Results Conclusions (limitations and enhancements)

The Problem Q: “Static” ? Image 1 Image 2 + Mosaiced image Q: “Static” ? Ans.: No moving objects in the scene.

The Solution Original images Image Registration / Alignment / Warping Image Blending

Constraints Scene Camera Motion Other Constraints Static / Dynamic Planar / Non planar (perspective distortion) Camera Motion Translation (sideways motion) Panning and Tilting (rotation about the Y and X axes) Scaling (zooming, forward / backward motion) General motion Other Constraints Automated / User input

Background and Literature survey Barnea & Silverman, 1972 (L1 Norm) Kuglin & Hines, 1975 (Phase Correlation) Mann & Picard, 1994 (Cylindrical projection) Irani & Anandan, 1995 (Static and Dynamic mosaics) Szeliski, 1996 (Transformation optimization) Badra, 1998 (Rotation and Zooming) Peleg and Rousso, 2000 (Adaptive Manifolds, Mosaicing using strips)

Image transformations Input image Output Rigid transformation Original shape Affine transformation Projective transformation Homogeneous coordinates Polynomial transformations

Presentation Overview Problem definition Background Literature Survey Image transformations Image Registration Coarse Image registration Transformation Optimization Image Blending Implementation and Results Conclusions (limitations and enhancements)

{ Image Registration Coarse Image Transformation Registration Initial transformation Transformation Optimization Error Improved ? { Phase Correlation L1 Norm User input

Phase Correlation d(x,y) (x0, y0) Kuglin & Hines, 1975 Translation property of Fourier Transform Inverse transform d(x,y) maximum (x0, y0)

Spatial Correlation, L1 Norm Barnea and Silverman f2 f2 E(x0,y0) = |f1(x,y) – f2(x- x0, y- y0)| f1 Spatial correlation techniques User input

Transformation Optimization Richard Szeliski, “Video Mosaics for Virtual Environments”, 1996. Optimization of initial transformation matrix M, to minimize error. Levenberg-Marquardt non-linear minimization algorithm. minimize Compute partial derivatives

Transformation Optimization Advantages Faster convergence Statistically optimal solution Limitations Local minimization (need a good initial guess)

Presentation Overview Problem definition Background Literature Survey Image transformations Image Registration Coarse Image registration Transformation Optimization Image Blending Implementation and Results Conclusions (limitations and enhancements)

Image Blending Smooth transition (edges, illumination artifacts) Simple averaging Weighted averaging Sample weight function – “hat filter” xmax More weight at the center of the image, less at the edges

Image blending Simple averaging Weighted averaging

Presentation Overview Problem definition Background Literature Survey Image transformations Image Registration Coarse Image registration Transformation Optimization Image Blending Implementation and Results Conclusions (limitations and enhancements)

Implementation Implemented using Matlab Source Images BE 230 lab images (fixed tripod) College 8 images (free hand motion, perpective distortion) East Field House images (free hand motion) Equipment: Sony DCR-TRV 900 3CCD digital camcorder

Sample results

Sample results

Conclusions/Enhancements Better automatic coarse registration techniques needed. Need to handle more general camera motion.

Thanks for listening !! Questions ?