1. 1 Image Features Hao Jiang Sept. 2014

2. Image Matching 2

3. Image Mosaicing 3

4. How can This be Done Automatically? 4

5. 5 Easy to match Hard to match

6. Image Local Structures 6 Step Ridge Peak Valley CornerJunction

7. Image Local Structures 7 Step Ridge Peak Valley CornerJunction Line Structures: “Edge” Point Structures: “Corners”

8. Regions 8

9. An Example 9 edge Region corners

10. Edge Detection in Matlab 10 >> im = imread('flower.jpg'); >> im = im2double(im); >> im = rgb2gray(im); >> ed = edge(im, 'canny', 0.15);

11. How to Find an Edge? 11 A 1D edge

12. 12 f(x) f’(x) f’’(x)

13. Extend to 2D 13 a b There is a direction in which image f(x,y) increases the fastest. The direction is called the gradient direction. Gradient [df/dx df/dy] Magnitude: sqrt(fx^2 + fy^2) Direction: atan2(fy, fx)

14. Finite Difference  Approximating derivatives using finite difference.  Finite difference and convolution 14

15. Noise Reduction 15 0.01 noise 0.03 noise

16. Noise Reduction 16

17. Gaussian Filtering in Edge Detection 17

18. Gaussian Filtering in Edge Detection 18 h * (g * f) = (h * g) * f Difference KernelGaussian Kernel image Difference Gaussian Kernel 18 [1,-1]

19. Edge Detection in Images  Gaussian smoothed filtering in x and y directions: Ix, Iy  Non-maximum suppression for |Ix|+|Iy|  Edge Tracing – double thresholding. 19

20. Edge Detection Using Matlab  Canny edge detector: edge(image, ‘canny’, threshold)  Sobel edge detector: edge(image, ‘sobel’, threshold)  Prewitt edge detector: edge(image, ‘prewitt’, threshold) 20

21. 21 D. Martin, C. Fowlkes, D. Tal, J. Malik. "A Database of Human Segmented Natural Images and its Application to Evaluating Segmentation Algorithms and Measuring Ecological Statistics”, ICCV, 2001 Berkeley Segmentation DataSet [BSDS]

22. Corner Detection  Corner is a point feature that has large changing rate in all directions. 22 StepLine Corner Flat region

23. Harris corner detector  C.Harris, M.Stephens. “A Combined Corner and Edge Detector”. 1988

24. The Basic Idea  We should easily recognize the point by looking through a small window  Shifting a window in any direction should give a large change in intensity

25. Harris Detector: Basic Idea “flat” region: no change in all directions “edge”: no change along the edge direction “corner”: significant change in all directions

26. Find a Corner 26 Compute matrix H = in each window. If the ratio (Ix2 * Iy2 – Ixy ^2 ) R =------------------------ > T (Ix2 + Iy2 + eps) We have a corner Ix2 Ixy Ixy Iy2 =

27. Harris Detector: Workflow

28. Compute corner response R

29. Harris Detector: Workflow Find points with large corner response: R>threshold

30. Harris Detector: Workflow Take only the points of local maxima of R

31. Harris Detector: Workflow

32. Corner Detection Programming 32

33. Harris Detector: Some Properties  Rotation invariance Ellipse rotates but its shape (i.e. eigenvalues) remains the same Corner response R is invariant to image rotation

34. Harris Detector: Some Properties  Partial invariance to affine intensity change Only derivatives are used => invariance to intensity shift I  I + b Intensity scale: I  a I R x (image coordinate) threshold R x (image coordinate)

35. Harris Detector: Some Properties  But: non-invariant to image scale! All points will be classified as edges Corner !

36. Scale Invariant Detection  Consider regions (e.g. circles) of different sizes around a point  Regions of corresponding sizes will look the same in both images

37. Scale Invariant Detection  The problem: how do we choose corresponding circles independently in each image?  Choose the scale of the “best” corner

38. Descriptors Invariant to Rotation  Find local orientation Dominant direction of gradient Extract image patches relative to this orientation

39. Descriptor Vector  Orientation = blurred gradient  Rotation Invariant Frame  Scale-space position (x, y, s) + orientation (  )

40. Scale and Rotation Invariant Feature  SIFT (D. Lowe, UBC) 40

41. Stable Feature 41

42. Stable Feature 42 Local max/min point’s values are stable when the scale changes

43. SIFT 43 Filtering the image using filters at different scales. (for example using Gaussian filter)

44. Difference of Gaussian 44

45. SIFT Feature Points 45 (b) Shows the points at the local max/min of DOG scale space for the image in (a).

46. Feature matching ?

47. Exhaustive search  for each feature in one image, look at all the other features in the other image(s) Hashing  compute a short descriptor from each feature vector, or hash longer descriptors (randomly) Nearest neighbor techniques  kd-trees and their variants

48. What about outliers? ?

49. Feature-space outlier rejection  Let’s not match all features, but only these that have “similar enough” matches?  How can we do it?  SSD(patch1,patch2) < threshold  How to set threshold?

50. Feature-space outlier rejection  A better way [Lowe, 1999]:  1-NN: SSD of the closest match  2-NN: SSD of the second-closest match  Look at how much better 1-NN is than 2-NN, e.g. 1-NN/2-NN  That is, is our best match so much better than the rest?

51. Feature-space outliner rejection  Can we now compute H from the blue points?  No! Still too many outliers…  What can we do?