1. Image Registration Advanced DIP Project
Group #3:
Dave Grimm
Joe Handfield
Mahnaz Mohammadi
Yushan Zhu

2. Outline Image Registration Point Mapping Problem Statement
GCP Selection Methods
Manual
Contour Based GCP
Corner and Edge Based
Evaluation Methods
Group Plan
Time Table
Responsibilities

3. Image Registration
Spatial matching the pixels of two (or more) images of the same area or scene
Relates the geometric coordinate system in one image to another
Transforms one of the images so that the two images share a common coordinate system

4. Image Registration (Cont.)

5. Applications of Image Registration
Remote Sensing
Extract information from images of the same region taken at different times or in different spectral bands
Color Science
Creating a mutispectral images
Medical
Pathology analysis
Image Fusion
Image Mosaicking

6. Point Mapping Widely used
Standard technique for registering images misaligned by an unknown transformation
Requires ground control points (GCPs or primitives) to be found in the images
Intrinsic or extrinsic
Can be done either manually or automatically

7. Point Mapping (cont)
Mathematically relates the coordinate systems of the images
a0 and b0 are needed for a simple shift of origin, and the first two terms are needed for a combined scale adjustment and shifting of the origin
Higher order equations for more complicated transforms are possible, such as rotation, skew, and perspective differences

8. Point Mapping Steps Select GCPs from each image
Match GCPs to from point pairs (points that are spatially the same in the two images)
Register images via point mapping
Our focus is on Steps 1 & 2

9. Problem Statement Comparison of GCP selection algorithms Manual
Automated
Area-based
Feature-based
Contour Mapping
Corner and Edge Detection

10. Manual Registration
Ground control points (GCP’s) are located by human choosing.
These points should be easy to discriminate: such as the corner of man-made lake, the road crossing, and small pool. It better to locate the points as much vertical and horizontal variation as possible.
Advantage: easy and straightforward.
Disadvantage: tedious, labor-intensive, and repetitive work.
Left: the reference image Right: the image to be registered

11. Automated Registration
Area-based algorithms
A small window of points in the sensed image, correlation kernel, is compared statistically with windows of the same size in the reference image. The measure of similarity is usually the normalized cross correlation. The location of maximum in the normalized correlation image is a pair of GCP.
Disadvantage : the correlation value is sensitive to scale and rotation

12. Automated Registration (Cont.)
Feature-based algorithm
Spatial features usually include edges, boundaries, intersections, etc
The general feature representations are:
Chain code
Moment invariants
Fourier descriptor
Shape signatures
Advantage: invariant to scaling, rotation, and translation

13. Feature- based algorithm Contour- based
Image segmentation
Image matching

14. Image Segmentation
Producing closed-edged contours by convolving the original image with Laplacian of Gaussian (LoG) operator
Find zero crossing points, in which the convolved image is scanned to detect pixels that have zero value or pixels at which a change of sign has occurred. Staring with a pixel, its neighboring pixels were expanded until a sign change occurred.
Drawbacks:
Discontinuity at the weak edge pixels
Thick edges

15. Image segmentation (Cont.) Thin and Robust Zero- Crossing
Mark as an edge point every pixel that satisfies the following conditions:
The pixel is a zero-crossing point
The pixel lies in the direction of the steepest gradient change (edge strength)
The pixel is the closest pixel to the virtual zero plane of the LoG image among its eight neighbors

16. Image Segmentation Thin and Robust Zero- Crossing (Cont.)
Discarded the noisy edge points using
Edge sorting
Edge refinement

17. Image Matching Invariant moment
Produce a set of scaled moment-based descriptor of planar shapes, that are scale, rotation, and translation invariant
Improved chain-coded representation of regions

18. Image Matching (Cont.) Chain coding
A way to represent a boundary by a connected sequence of straight-line segments of specific length and direction based on 4- or 8- connectivity

19. Image Matching (Cont.) Draw backs of Standard Chain Coding
The resulting chain codes tend to be quite long
Any small disturbance along boundary due to noise or imperfect segmentation causes changes in the code that may not be related to the shape of the boundary

20. Improved Chain-Code Shift Operation: Smoothing: Gaussian
Normalization: Demean
Resampling Operation
Chain codes are used to represent a boundary by a connected sequence of straight-line segments of specified length and direction. A digital curve can be represented by an integer sequence ai (0,1.2,,,7), depending on the relative position of the current edge pixel with respect to the previous edge pixel.
The standard Freeman chain code has certain drawbacks for correlation between two potentially matched contours, such as being noisy and variant to rotational ,scaling, and translational differences.
3. Normalization: rotation of contours cause the means of chain-code different, this rotational difference can be removed by subtracting the means.
4. Resampling: to be scale invariant, the chainn code is resampled to the same length as that of its corresponding region.
A,b is a pair of example-matched contours and displayed for comparison.
C,d the standard chain codes.
E,f the shifted chain codes.
G,h the shifted and smoothed chain codes.
I,j the modified chain codes by all four operations: shift, smoothing, normalization, resampling.

21. Image Matching (Cont.) Invariant-Moment Distance Matrix
The pairs are accepted as candidate matches if their invariant-moment values are below the defined thresholds

22. Image Matching (Cont.) Chain-code Matching Matrix
Contour A and B selected as matched pair if:
1) DAB≥DAB’ where B’ includes all the contours with similar shapes to A
2) DAB≥T3 where T3 is a preset threshold which can eliminate matches with poor correlation

23. Image Matching (Cont.)
The smaller the Invariant-Moment distance, the more similar the shapes of two region
The greater the chain-code matching coefficient, the more contours resemble each other in the shape
when Dkl=1, there is a perfect match
The centroid of the matched contours are used as GCPs

24. Summary of Contour Based Algorithm

25. Corner and Edge Detection
The corners and edges present in each image are located
Harris and Stephens, 1988
A local window is placed in the image and changes due to shifting the window are considered

26. Corner and Edge Detection (cont)
An edge produces large changes when the window is shifted perpendicular to the edge direction and small changes when shifted parallel
A corner produces large changes when the window is shifted either perpendicular or parallel
Insignificant points (noise) are removed via thresholding

27. Corner and Edge Detection (cont)
Each found point in the image to be registered (image B) is then compared to each found point in the reference image (image A) to determine which pairs match
Any point without a match is considered an outlier (slack)

28. Test Images Images that we can introduce known transformations
Various images with unknown transformations
Limit testing to grayscale images for now

29. Statistical Assessment of Image Registration Results
Assessment would be done on unknown GCP pairs using calculated transformation matrix obtained by different point mapping algorithms

30. Statistical Assessment
Calculation of statistical distance for each pair of points, P(x1,y1), Q(x2,y2)

31. Statistical Assessment
Take maximum, standard deviation, and average error between the group of GCPs using different transformation matrix
Evaluate variability of predicted points along x and y axis using scatter plot

32. Statistical Assessment
Calculate the confidence intervals

33. Registration Time Table
Week 4 – Plan presentation, programming
Week 5 – Programming
Week 6 – At least manual code done, present preliminary results
Week 7 – Evaluation methods done
Week 8 – Both automated methods finished
Week 9 – Gathering image results, complete write-up
Week 10 – Final presentation & report

34. Point-Mapping Registration Algorithm
Plan Flowchart
Method flow:
Evaluation:
Methods:
Manual
Contour
C&E
GCPs
Correlations
Confidence Interval
Point-Mapping Registration Algorithm

35. Task Assignment List Joe - Manual pixel detection method
Yushan - Image contour detection method
Dave - Corner and edge detection
Mahnaz - Statistical evaluation (with Joe)

36. Questions?
Thank you