1. Image Registration  Mapping of Evolution

2. Registration Goals
I2(x,y)=g(I1(f(x,y)) f() – 2D spatial transformation g() – 1D intensity transformation
Assume the correspondences are known
Find such f() and g() such that the images are best matched

3. Spatial Transformations
Rigid
Affine
Projective
Perspective
Global Polynomial
Spline

4. Rigid Transformation
Rotation(R)
Translation(t)
Similarity(scale)

5. Affine Transformation
Rotation
Translation
Scale
Shear
No more preservation of lengths and angles
Parallel lines are preserved

6. Perspective Transformation (Planar Homography)

7. Perspective Transformation(2)
(xo,yo,zo)  world coordinates
(xi,yi)  image coordinates
Flat plane tilted with respect to the camera requires Projective Transformation

8. Projective Transformation
(xp,yp)  Plane Coordinates
(xi,yi)  Image Coordinates
amn  coefficients from the equations of the scene and the image planes

9. Complex Transformations
Global Polynomial Transformation(splines)

10. Methods of Registration
Correlation
Fourier
Point Mapping

11. Correlation Based Techniques
Given a two images T & I, 2D normalized correlation function measures the similarity for each translation in an image patch
Correlation must be normalized to avoid contributions from local image intensities.

12. Correlation Theorem
Fourier transform of the correlation of two images is the product of the Fourier transform of one image and the complex conjugate of the Fourier transform of the other.

13. Fourier Transform Based Methods
Phase-Correlation
Cross power spectrum
Power cepstrum
All Fourier based methods are very efficient, only only work in cases of rigid transformation

14. Point Mapping Registration
Control Points
Point Mapping with Feedback
Global Polynomial

15. Manually or Automatically selected
Control Points
Extrinsic
Manually or Automatically selected
Intrinsic
Markers within
the Image
After the control points have been
determined, cross correlation, convex hull
edges and other common methods are used
to register the sets of control points

16. Point mapping with Feedback
Clustering example: determine the optimal spatial transformation between images by an evaluation of all possible pairs of feature matches.
Initialize a point in cluster space for each transformation
Use the transformation that is closest to the best cluster
Too many points, thus use a subset

17. Global Polynomial Transformation(1)
Use a set of patched points to generate a single optimal transformation
Bi-Variate transformation:
(x,y) – reference image
(u,v) – working image

18. Global Polynomial Transformation(2)
When is polynomial transformation bad?
Splines approximate polynomial transformations(B-spline, TP-spline)

19. Characteristics of Registration Methods
Feature Space
Similarity Metrics
Search Strategy

20. Feature Spaces

21. Similarity Metrics

22. Search Strategies

23. Robust Multi-Sensor Image Alignment
Irani & Anandan
Direct Method(vs. Feature Based)

24. Multi Sensor Images EO IR Find features Loss of important information
Original Image (Intensity Map)
Assume global statistical correlation
Often violated

25. Multi Sensor Image Representation(1)
Same Modality Camera Sensors  enough correlated structure at all resolution levels
Different Modality Camera Sensors  primary correlation only in high resolution levels

26. Multi Sensor Image Representation(2)
Goal: Suppress non-common information & capture the common scene details
Solution: High pass energy images

27. Laplacian Energy Images
Apply the Laplacian high pass filter to the original images
Square the results  NO contrast reversal
BUT
The Laplacian is directionally invariant

28. Directional Derivative Energy Images
Filter with Gaussian
Apply directional derivative filter to the original image in 4 directions
Square the resultant images

29. Alignment Algorithm
Do not assume global correlation, use only local correlation information
Use Normalized Correlation as a similarity measure
Thus, no assumptions about the original data

30. Behavior of Normalized Correlation with Energy Images
NC=1
Two images are linearly related
NC<1(high)
Two images are not linearly related, yet local fluctuations are low
NC<1(low)≈0
Incorrect displacements

31. Global Alignment with Local Correlation(1)
a, b – original images
{ai,bi} – directional derivatives (i=1..4)
p=(p1…p6)T affine
(u,v) – shift from one image to another
Si(x,y)(u,v) – correlation surface at a pixel(x,y)

32. Global Alignment with Local Correlation(2)
Goal: Find the parametric transformation p, which maximizes the sum of all normalized correlation values. global similarity M(p)

33. Solving for M(p) Newton’s method is used to solve for M(p)
The quadratic approximation of M around p is obtained by combining the quadratic approximations of each of the local correlation surfaces S around local displacement.

34. Steps of the Algorithm Construct a Laplacian resolution pyramid
Compute a local normalized-correlation surface around a given displacement
Compute the parametric refinement
Update p
Start over(process terminates at the highest resolution level of the last image)

35. Outlier Rejection Due to the different modalities of the
sensors, the number of outliers may be very
large
Accept pixels based on concavity of the correlation surface
Weigh the contribution of a pixel by det|H(u)|

36. Results EO image IR image Composite before Alignment
Composite after Alignment
IR image

37. Acknowledgements
I would like to thank Compaq for making awful computers
Professor Belongie for reviewing the slides