1. An image registration technique for recovering rotation, scale and translation parameters
March 25, 1998
Morgan McGuire

2. Acknowledgements Dr. Harold Stone, NEC Research Institute
Bo Tao, Princeton University
NEC Research Institute
3/25/98
Morgan McGuire

3. Problem Domain
Satellite, Aerial, and Medical sensors produce series images which need to be aligned for analysis. These images may differ by any transformation (possible noninvertible).
Images courtesy of Positive Systems
3/25/98
Morgan McGuire

4. New Technique Solves subproblem (practical case)
O(ns(NlogN)/4k+Nk) compared to O(NlogN), O(N3)
Correlations typically > .75 compared to .03
3/25/98
Morgan McGuire

5. Structure of the Talk Differences Between Images Fourier RST Theorem
Degradation in the Finite Case
New Registration Algorithm
Edge Blurring Filter
Rotation & Scale Signatures
Experimental Results
Conclusions
3/25/98
Morgan McGuire

6. Differences Between Images
Alignment
Occlusion
Noise
Change
3/25/98
Morgan McGuire

7. Sub-problem Domain Alignment = RSTL Occlusion < 50%
Noise + Change = Small
Square, finite, discrete images
Image cropped from arbitrary infinite texture n
N pixels
3/25/98
Morgan McGuire

8. RST Transformation
3/25/98
Morgan McGuire

9. Fourier Rotation, Scale, and Translation Theorem†
Pixel Domain Fourier Domain
p = rotate(r, f) P = rotate(R, f)
p = dilate(r, s) Fp = s2 . dilate(Fr, 1/s)
p = translate(r, Dx, Dy) ÐFp = translate(ÐFr, Dx, Dy)
3/25/98
Morgan McGuire

10. †For Infinite Images
3/25/98
Morgan McGuire

11. In practice, we use the DFT
Let X0 = DFT(x0)
X0 and x0 are discrete, with N non-zero coefficients.
Let X = DTFT(x)
X0 and x0 are sub-sampled tiles (one period spans) of X and x. The Fourier RST theorem holds for X and x... does it also hold for X0 and x0?
3/25/98
Morgan McGuire

12. Fourier Transform and Rotations
3/25/98
Morgan McGuire

13. Theorem Infinite case: Fourier transform commutes with rotation
Folklore: It is true for the finite case
3/25/98
Morgan McGuire

14. Using Fourier-Mellin Theory
Magnitude of Fourier Transform exhibits rotation, but not translation
Registration algorithm:
Correlate Fourier Transform magnitudes for rotation
Remove rotation, find translation
Generalizes to find scale factors, rotations, and translation as distinct operations
3/25/98
Morgan McGuire

15. Folklore is wrong Image Tile Rotate Tile Image Rotate 3/25/98
Morgan McGuire

16. The Mathematical Proof
The Finite Fourier transform
continuous
Windowing, sampling, infinite tiling
Transform, then rotate
3/25/98
Morgan McGuire

17. The Mathematical Proof
Rotate, then transform
3/25/98
Morgan McGuire

18. Finite-Transform Pairs
3/25/98
Morgan McGuire

19. The Artifacts
3/25/98
Morgan McGuire

20. Fourier Transforms 3/25/98 Morgan McGuire
Oppenheim & Willsky Signals & Systems; Oppenheim and Schafer, Discrete-Time Signal Processing
3/25/98
Morgan McGuire

21. Tiling does not Commute with Rotation
Tiled Image
Rotated Tiled Image
Tiled Rotated Image
…so the Fourier RST Theorem does not hold for DFT transforms.
3/25/98
Morgan McGuire

22. Correlation Computation
3/25/98
Morgan McGuire

23. Prior Art Alliney & Morandi (1986) Reddy & Chatterji (1996)
use projections to register translation-only in O(n), show aliasing in Fourier T theorem
Reddy & Chatterji (1996)
use Fourier RST theorem to register in O(NlogN)
Stone, Tao & McGuire (1997)
show aliasing in Fourier RST theorem
3/25/98
Morgan McGuire

24. An Empirical Observation
Even though the Fourier RST Theorem does not hold for finite images, we observe the DFT does have a “signature” that transforms in a method predicted by the Theorem.
Image
DFT Magnitude
3/25/98
Morgan McGuire

25. Sources of Degradation
Frequency
Aliasing (from Tiling)
“+” Artifact
Sampling Error
Pixel
Image Window Occlusion
Image Noise
3/25/98
Morgan McGuire

26. 5. Recover Translation Parameters
Algorithm Overview r p G
FMT
fq,rd
fq,logrdq J q
Maximum Value Detector
Peak Detector
Norm. Corr.
List of scale factors (s)
exp W H
Coarse (Dx, Dy)
FFT
Dilate
Rotate
(Pixel) Correlation W
r m p h
1. Pre-Process
5. Recover Translation Parameters
2. FMLP Transform
4. Recover
Rotation
Parameter
3. Recover
Scale
Parameter
Norm. Circ. Corr.
3/25/98
Morgan McGuire

27. Problem: “+” Artifact Transformation
None Rotation Dilation Translation
Transformation
DFT
Image
3/25/98
Morgan McGuire

28. Solution: “Edge-Blurring” Filter, G
Image
None
DFT
Disk
Blur
Filter
3/25/98
Morgan McGuire

29. Problem:Need Orthogonal Invariants
Fourier-Mellin transform:
In the “log-polar” (logr,q) domain:
3/25/98
Morgan McGuire

30. Mapping (wx,wy) to (logr,q)
q=p/4
logr=2,
q=3p/4
wx=4
wy=4 wy wx
wx=8
wy=8
3/25/98
Morgan McGuire

31. Sample Image Pair f = 17.0o s = 0.80 Dx = 10.0 Dy = -15.0 G(r) G(p)
N = 65536
k = 2
G(r)
G(p)
3/25/98
Morgan McGuire

32. Nonzero Fourier CoefficientsP
3/25/98
Morgan McGuire

33. Solution I: Rotation Signature
1. Selectively weight “edge coefficients” (J filter)
2. Integrate along r axis
F is Scale and Translation Invariant. Pixel rotation appears as a cyclic shift => use simple 1d O(nlogn) correlation to recover rotation parameter.
3/25/98
Morgan McGuire

34. F Signatures of r and p
3/25/98
Morgan McGuire

35. F Correlations
3/25/98
Morgan McGuire

36. Solution II: Scale Signature
1. Integrate along q axis (rings)
2. Normalize by r (area)
3. Enhance S/N ratio (H filter)
S is Rotation and Translation Invariant. Pixel dilation appears as a translation => use simple 1d O(nlogn) correlation to recover scale parameter.
3/25/98
Morgan McGuire

37. Raw S Signature
3/25/98
Morgan McGuire

38. Filtered S Signature
3/25/98
Morgan McGuire

39. S Correlation
3/25/98
Morgan McGuire

40. New Registration Algorithmp G
FMT
fq,rd
fq,logrdq J q
Maximum Value Detector
Peak Detector
Norm. Corr.
List of scale factors (s)
exp W H
Coarse (Dx, Dy)
FFT
Dilate
Rotate
(Pixel) Correlation W
r m p h
Norm. Circ. Corr.
Compute full-resolution Correlation for small neighborhood of Coarse (Dx, Dy) to refine.
3/25/98
Morgan McGuire

41. Recovered Parameters
3/25/98
Morgan McGuire

42. Disparity Map
3/25/98
Morgan McGuire

43. Multiresolution for Speed
Algorithm is O(NlogN) because of FFT’s
With kth order wavelet, O((NlogN)/4k)
To refine, search 22k = 4k positions
Using binary search, k extra O(N) each
Total algorithm is O((NlogN)/4k + Nk)
3/25/98
Morgan McGuire

44. Results & Confidence
3/25/98
Morgan McGuire

45. Analysis of Results
3/25/98
Morgan McGuire

46. Future Directions Better scale signature
Use occlusion masks for FM techniques?
Combining FM technique with feature based techniques
3/25/98
Morgan McGuire