1. CSE 291 Seminar Presentation Andrew Cosand ECE CVRR
An Algorithm for Associating the Features of Two Images / G. L. Scott, H. C. Longuet-Higgins A direct method for stereo correspondence based on singular value decomposition / M. Pilu
CSE 291 Seminar Presentation
Andrew Cosand
ECE CVRR

2. Outline Correspondence Problem S&L-H Solution Pilu’s Contribution
Examples
Discrepancy
S&L-H Solution
Distance Measure
Singular Value Decomposition
Relation to Kernel Trick
Pilu’s Contribution

3. Correspondence Problem
Which features in image A correspond to features in image B?

4. Correspondence Problem
This task is trivial for humans, but difficult for computers.

5. Correspondence Problem
Used for stereo image pairs & motion images.
Feature correspondence should exhibit Similarity, Proximity and Exclusivity.
Complexity is combinatorial with number of features to compare.

6. Stereo Imaging
Trinocular camera captures 3 images, horizontally and vertically offset.

7. Stereo Imaging
Feature correspondence is used to extract depth information from stereo images
Distances between cameras are known.
Distances between the same feature in different images is determined.
Distance from cameras to actual object can be calculated.

8. Motion Tracking
Corresponding features are tracked through sequential images to determine object or camera motion.
Object Motion Only
Compound Motion

9. Local vs. Global

10. Discrepancy
Small scale discrepancy constrains corresponding features to be close together.
Slow object movement, slight camera motion, narrow baseline stereo
Large scale discrepancy allows widely separated features.
Fast object movement, large camera motion, wide baseline stereo

11. Ternus

12. Ternus

13. Ternus

14. Achieving Good Global Correspondence
Requires relationships between points
The inner product of x,y coordinates yields a deficient feature space. (Also location biased)
Gaussian weighted distance better captures the spatial relationships between points (location and proximity).
S&LH provides superior sphered (decorrelated) relationship.
Pilu adds similarity relationship.

16. Scott & Longuet-Higgins
Define a distance metric between features
Gij=e(-rij2/22)
Create matrix of relationships for all possible feature pairs
G11
Gij

17. S&LH Distance Measure Gaussian Weighted
 scales distance weighting (discrepancy)
Analytic with respect to distance, coordinates
Decreases monotonically with distance
Positive Definite for identical images

18. Positive Definite Matrices
Comparing identical feature sets yields a symmetric positive definite matrix.
Symmetric gets us real eigenvalues.
Positive definite has positive eigenvalues (which means real square roots).
G = UUT = QQT => Q = U1/2
Matrix Factors
Inner
Product
Real

19. Singular Value Decomposition
SVD factors a matrix into the product of two orthogonal matrices and a diagonal matrix of singular values (eigenvalues).
G = TDU, G is m-by-n,
T is orthogonal, m-by-m
D is diag(1, 2, … p), m-by-n, p=min{m,n}
U is orthogonal, n-by-n

20. Scott & Longuet-Higgins
Use Singular Value Decomposition on matrix. G = TDU

21. Scott & Longuet-Higgins
Set diagonal elements of D to 1, ‘recover’ relationship matrix.
P = TIU = TU
Eliminating singular matrix rescales data in feature space, essentially sphereing it.

22. Scott & Longuet-Higgins
Largest feature in row and column indicates mutual best match (correspondence)

23. Relation to Kernel Trick
Gaussian Distance is essentially a kernel
Relates to a dot product in infinite dimensionial space.
This gives a richer feature space with useful relationships between features.
This is why the SVD works here.

24. Pilu’s Improvement
Rogue features don’t correspond to anything, complicating the process.
S&LH only deals with proximity and exclusivity.
Similarity constraint can eliminate rogue features, which shouldn’t be similar to anything.

25. Pilu’s Improvement
Modify relationship metric to include gray-level correlation.
Gij = (e-(Cij – 1)2/22) e(-rij2/22)
Gij = ((Cij+1) /2) e(-rij2/22)
Adds similarity to feature space (kernel operation).
Rogue features can be eliminated because they are not similar to anything.

26. Results
Achieves globally better feature matches rather than locally good matches.
Resistant to rogue points.

27. Summary
S&LH essentially maps input to a rich, high dimensional feature space using kernel trick, then uses SVD to determine matches.
Pilu improves kernel to achieve better feature space.
Combination works well.

28. References This presentation drew material from the following sources
S. Belonge, Notes on Spectral Correspondence
M. Pilu, A direct method for stereo correspondence based on singular value decomposition
variants
G. L. Scott, H. C. Longuet-Higgins, An Algorithm for Associating the Features of Two Images