(1) Feature-point matching by D.J.Duff for CompVis Online: Feature Point Matching Detection, Extraction & Matching Part 2 – Matching via template matching (Previous slides: Feature point detection Moravec detection Harris detection Scale space detection Feature point Extraction ).
(2) Feature-point matching by D.J.Duff for CompVis Online: Template matching – X-correlation ● Template matching by Cross-correlation You have already learnt template matching with sum of square differences – SSD: There you were trying to minimize a dissimilarity score. Here you will learn about template matching by normalized cross-correlation: minimizing a similarity score.
(3) Feature-point matching by D.J.Duff for CompVis Online: Template matching – X-correlation ● Cross-correlation: In statistical parlance, the cross-correlation between two random variables is a measure of how closely the two variables vary together. {A,B} = {1,2},{8,16},{3,6},{-5,-10} ● The magnitude of these variables vary together closely – they have a high positive cross-correlation. {C,D} = {-1,2},{-8,16},{-3,6},{5,-10} ● The magnitude of these vary oppositely closely – they have a high negative cross-correlation. {E,F} = {-1,2},{8,0},{-3,6},{-5,-10} ● These don’t vary closely – they have a low cross-correlation.
(4) Feature-point matching by D.J.Duff for CompVis Online: Template matching – X-correlation ● Cross-correlation: ● Normalized cross-correlation (-1<0<-1): {A,B} = {1,2},{8,16},{3,6},{-5,-10} ● X(A,B) = 43 C(A,B) = 1.0 {C,D} = {-1,2},{-8,16},{-3,6},{5,-10} ● X(C,D) = -43 C(A,B) = -1.0 {E,F} = {-1,2},{8,0},{-3,6},{-5,-10} ● X(E,F) = 7.4 C(A,B) = 0.25
(5) Feature-point matching by D.J.Duff for CompVis Online: Template matching – X-correlation ● Lingo alert: Cross-correlation is usually called Covariance Normalized cross-correlation is usually called Correlation:
(6) Feature-point matching by D.J.Duff for CompVis Online: Template matching – X-correlation ● Take an image (I) ● Take a target area to be matched (T) ● For each point in the image (I) Find the normalized cross-correlation: Where: and ● Template matching with Cross-correlation
(7) Feature-point matching by D.J.Duff for CompVis Online: Template matching – X-correlation ● Pick a pixel to be matched (u,v) ● For each (x,y) in the template ● Multiply its value (minus the template average) with the equivalent value in the image (minus the image patch average). Add to running total. ● Then divide the total by the square root of the product of the variances of the image and template. ● How to calculate the cross-correlation function C = 0.86 here
(8) Feature-point matching by D.J.Duff for CompVis Online: Template matching – X-correlation ● Template matching with Cross-correlation Q: It looks similar to convolution. How does this relate to convolution? A: It is indeed very similar. What you learnt as convolution is actually almost the cross-correlation that you learnt here (you just don’t divide by N). But this that is yet another definition of cross-correlation, from signal detection theory… …so how would you do cross-correlation in matlab? Matlab: help conv2 help corr2 The cross-correlation function at the point is a measure of the degree of match between template and that point in the image.
(9) Feature-point matching by D.J.Duff for CompVis Online: Template matching – X-correlation ● Template matching with Cross-correlation But... ● How well would this template match the image? ● Answer: Undefined! Divide by zero! ● It’s the normalization factor’s fault! (Doesn’t affect non-normalized cross-correlation).
(10) Feature-point matching by D.J.Duff for CompVis Online: Template matching ● Problems with naïve template matching And... ● How well would this template match the image? ● Answer: ● Not rotationally invariant.
(11) Feature-point matching by D.J.Duff for CompVis Online: Template matching ● Problems with naïve template matching And... ● Problem: Rotational invariance. ● Possible solution: Match every possible rotation.
(12) Feature-point matching by D.J.Duff for CompVis Online: Template matching ● Problems with naïve template matching And... ● Problem: Rotational invariance. ● Possible solution: Align orientations. Direction of fastest change Direction of slowest change E.g. by gradient direction
(13) Feature-point matching by D.J.Duff for CompVis Online: Template matching ● Problems with naïve template matching Also… ● How well would this template match the image? ● Answer: 0.6 ● Needs to handle scale changes better too. ● Could check multiple scales…
(14) Feature-point matching by D.J.Duff for CompVis Online: Template matching ● Naïve cross-correlation is not the only way to do template matching. ● More sophisticated methods have to be able to handle Scale changes Rotation Affine transformation Lighting changes Etc ● More efficient approaches are needed. E.g. Using pyramidal (down-sampled) images Frequency domain filtering Iterative refinement
(15) Feature-point matching by D.J.Duff for CompVis Online: Alternatives to template matching ● Template matching can be refined to be made very useful. ● But other approaches exist too. ● These approaches generalize template matching by extracting a description (or descriptor) of the target feature point so that the descriptor can be matched. For example: Training a classifier from local image information Extracting local image statistics ● E.g. histogram of spatial distribution of local intensity gradient (as per SIFT) More? This improvement can be made at the extraction phase.
(16) Feature-point matching by D.J.Duff for CompVis Online: Summary ● Criteria for a good feature point detector are: Repeatably detectable under transforms: ● Scale ● Rotation ● Affine transformation ● Illumination ● Etc Computationally efficient Accurate at localisation High detection rate Robust to noise
(17) Feature-point matching by D.J.Duff for CompVis Online: Summary ● Criteria for a good feature point matcher are: Consistent Invariant to: ● Scale ● Rotation ● Illumination ● Affine transformation ● Etc Robust to noise Accurate