1. IMAGE MOSAICING Summer School on Document Image Processing
Thotreingam Kasar
MEDICAL INTELLIGENCE AND LANGUAGE ENGINEERING LAB,
DEPARTMENT OF ELECTRICAL ENGINEERING,
INDIAN INSTITUTE OF SCIENCE, BANGALORE, INDIA –

2. Image Mosaicing
Given multiple images of a scene, with some degree of overlap, how do we seamlessly blend them into a single composite image?
The basic ingredients
Registration
Warping
Blending

3. Methods for Image Mosaicing
Direct Method
Preferred for images without many prominent details
Windows of pre-defined size or even the entire images are used for the correspondence estimation
Can handle only translation and small rotation
Sensitive to intensity changes, due to noise, varying illumination, and/or by using different sensors
Feature based Method
Generally faster than direct methods
More robust against scene movement
Robust estimation algorithms available
Suitable for fully automatic mosaicing
The respective features might be hard to detect and/or unstable in time

4. Projective Geometry
Set of rays in 3-D space with each point representing a projective point
Planes through the origin are interpreted as lines

5. } are the same lines for any k ≠ 0
Homogeneous Co-ordinates
A line in a plane (ax + by + c = 0) can be represented as (a, b, c)T
ax + by + c = 0
(ka)x + (kb)y + kc = 0
Thus, (a, b, c)T and (ka, kb, kc)T are homogeneous vectors.
A point X=(x,y)T lies on the line (a, b, c)T if
or, (x, y,1)T · (a, b, c)T = 0
Thus, the point (x,y)T in R2 is represented as a 3-vector by adding a final
co-ordinate of 1.
Any arbitrary homogeneous point P=(x1, x2, x3)T represents the point (x1/x3, x2/x3) in R2
} are the same lines for any k ≠ 0

6. 2-D Transformations Translation Rotation Scaling
We can combine all the multiplicative and translation terms for 2-D geometric transformations into a single 3x3 matrix using Homogeneous Coordinates

7. Geometric Transformations
RIGID
AFFINE
PERSPECTIVE

8. IMAGING GEOMETRY
The general projective transformation of one projective plane to another is represented as y O X’ X x y’ x’
Image 1
Image2
For a pair of matching points (x, y) and (x’, y’) in the world and image plane respectively, the projective transformation in inhomogeneous form is
4 point correspondences lead to 8 such linear equations which can be solved up to an insignificant multiplicative factor.

9. Feature Detection
Harris corner is widely used to localize interest points
Small intensity change
Large intensity change in one direction
Large intensity change in both directions
Local maxima of the Response function R gives the corners

10. Corner Detection

11. Feature Matching

12. RANdom SAmple Concensus (M. A. Fischler and R. C. Bolles,1981)

13. RANSAC Randomly select 4 point matches Estimate homography Hi
Count the consensus set Si
If |Si|> T, return Hi
Repeat N times and return the model with max|Si|
p = P(at least 1 of the random samples is free from outliers)
w = P(any selected data point is an inlier)
e = (1-w) is the probability that it is an outlier
At least N selections are required where (1-w4)N = 1-p

14. RANSAC %age of outliers #Trials required 5 3 10 20 9 30 17 40 34 50 72
For a sample size of 4, the values of N required to ensure with a probability p =0.99 that at least 1 of the sample is free from outliers
%age of outliers
#Trials required 5 3 10 20 9 30 17 40 34 50 72

15. Feature Correspondence
Target Image 1
Reference Image
Target Image 2
Registered Image 1
Registered Image 2

16. BLENDING Simple Averaging
Feathering - A weighting function is associated with each image decaying from a maximum at the centre to zero at the image boundary
Where M and N represents the dimensions of the image and
(x0, y0) is the image center

17. Mosaic Construction Target image 1 Reference Image Target image 2
Mosaic output

19. Why Mosaicing? To enhance the limited field of view of camera
Some Applications
Satellite image Analysis
Medical image analysis
3-D Scene reconstruction and Robotic Navigation
Creating Super-resolution images
Video representation and indexing

20. Discussions Stable Feature Localization Invariant Feature Extraction
Sub-pixel registration
Blending

21. THANK YOU