IMAGE MOSAICING Summer School on Document Image Processing Thotreingam Kasar MEDICAL INTELLIGENCE AND LANGUAGE ENGINEERING LAB, DEPARTMENT OF ELECTRICAL ENGINEERING, INDIAN INSTITUTE OF SCIENCE, BANGALORE, INDIA – 560 012

Image Mosaicing Given multiple images of a scene, with some degree of overlap, how do we seamlessly blend them into a single composite image? The basic ingredients Registration Warping Blending

Methods for Image Mosaicing Direct Method Preferred for images without many prominent details Windows of pre-defined size or even the entire images are used for the correspondence estimation Can handle only translation and small rotation Sensitive to intensity changes, due to noise, varying illumination, and/or by using different sensors Feature based Method Generally faster than direct methods More robust against scene movement Robust estimation algorithms available Suitable for fully automatic mosaicing The respective features might be hard to detect and/or unstable in time

Projective Geometry Set of rays in 3-D space with each point representing a projective point Planes through the origin are interpreted as lines

} are the same lines for any k ≠ 0 Homogeneous Co-ordinates A line in a plane (ax + by + c = 0) can be represented as (a, b, c)T ax + by + c = 0 (ka)x + (kb)y + kc = 0 Thus, (a, b, c)T and (ka, kb, kc)T are homogeneous vectors. A point X=(x,y)T lies on the line (a, b, c)T if or, (x, y,1)T · (a, b, c)T = 0 Thus, the point (x,y)T in R2 is represented as a 3-vector by adding a final co-ordinate of 1. Any arbitrary homogeneous point P=(x1, x2, x3)T represents the point (x1/x3, x2/x3) in R2 } are the same lines for any k ≠ 0

2-D Transformations Translation Rotation Scaling We can combine all the multiplicative and translation terms for 2-D geometric transformations into a single 3x3 matrix using Homogeneous Coordinates

Geometric Transformations RIGID AFFINE PERSPECTIVE

IMAGING GEOMETRY The general projective transformation of one projective plane to another is represented as y O X’ X x y’ x’ Image 1 Image2 For a pair of matching points (x, y) and (x’, y’) in the world and image plane respectively, the projective transformation in inhomogeneous form is 4 point correspondences lead to 8 such linear equations which can be solved up to an insignificant multiplicative factor.

Feature Detection Harris corner is widely used to localize interest points Small intensity change Large intensity change in one direction Large intensity change in both directions Local maxima of the Response function R gives the corners

Corner Detection

Feature Matching

RANdom SAmple Concensus (M. A. Fischler and R. C. Bolles,1981)

RANSAC Randomly select 4 point matches Estimate homography Hi Count the consensus set Si If |Si|> T, return Hi Repeat N times and return the model with max|Si| p = P(at least 1 of the random samples is free from outliers) w = P(any selected data point is an inlier) e = (1-w) is the probability that it is an outlier At least N selections are required where (1-w4)N = 1-p

RANSAC %age of outliers #Trials required 5 3 10 20 9 30 17 40 34 50 72 For a sample size of 4, the values of N required to ensure with a probability p =0.99 that at least 1 of the sample is free from outliers %age of outliers #Trials required 5 3 10 20 9 30 17 40 34 50 72

Feature Correspondence Target Image 1 Reference Image Target Image 2 Registered Image 1 Registered Image 2

BLENDING Simple Averaging Feathering - A weighting function is associated with each image decaying from a maximum at the centre to zero at the image boundary Where M and N represents the dimensions of the image and (x0, y0) is the image center

Mosaic Construction Target image 1 Reference Image Target image 2 Mosaic output

Why Mosaicing? To enhance the limited field of view of camera Some Applications Satellite image Analysis Medical image analysis 3-D Scene reconstruction and Robotic Navigation Creating Super-resolution images Video representation and indexing

Discussions Stable Feature Localization Invariant Feature Extraction Sub-pixel registration Blending

THANK YOU