1. 3D Computer Vision Generalized Hough Boundaries Representations Snakes (Trucco: 109-113) Computational Vision / Ioannis Stamos

2. Generalized Hough Transform Detecting shapes not described by an analytic equation CSc 83020 3-D Computer Vision – Ioannis Stamos

3. From Shree Nayar’s Notes

4. CSc 83020 3-D Computer Vision – Ioannis Stamos From Shree Nayar’s Notes

5. Computational Vision / Ioannis Stamos From Shree Nayar’s Notes

6. Computational Vision / Ioannis Stamos From Shree Nayar’s Notes

7. Computational Vision / Ioannis Stamos From Shree Nayar’s Notes

8. Computational Vision / Ioannis Stamos From Shree Nayar’s Notes

9. Computational Vision / Ioannis Stamos From Shree Nayar’s Notes

10. Computational Vision / Ioannis Stamos From Shree Nayar’s Notes

11. From Shree Nayar’s Notes Find template t in image f t and f are both gray-scale images

12. From Shree Nayar’s Notes

13. From Shree Nayar’s Notes

14. From Shree Nayar’s Notes

15. From Shree Nayar’s Notes

16. From Shree Nayar’s Notes

17. From Shree Nayar’s Notes

18. From Shree Nayar’s Notes

19. Discussion Can you search for matches in Different scale (size) than template? Different orientation than template? Computational Vision / Ioannis Stamos

20. Comparison Template matching vs. Hough Computational Vision / Ioannis Stamos

21. Representation of 2-D Geometric Structures To MATCH image boundary/region with MODEL boundary/region, they must represented in the same manner. Boundary Representation Snakes – Extraction of arbitrary contours from image. Region Representation

22. Representation Issues Compact: Easy to Store & Match. Easy to manipulate & compute properties. Captures Object/Model shape. Computationally efficient.

23. Boundary Representation Polylines: concatenation of line segments. Breakpoint Matching on the basis of: # of line segments lengths of line segments angle between consecutive segments

24. Running Least Squares Method eiei Move along the boundary At each point find line that fits previous points (Least Squares) Compute the fit error E=Sum(e i ) using previous points If E exceeds threshold, declare breakpoint and start a new running line fit.

25. Approximating Curves with Polylines Draw Straight line between end-points of curve For every curve point find distance to line. If distance is less than tolerance level for all points, Exit Else, pick point that is farthest away and use as breakpoint. Introduce new segments. Recursively apply algorithm to new segments.

26. Ψ-s Curve s ψ x y s Ψ 12 3 4 5 6 x y 1 2 3 4 5 6 π 2π

27. Ψ-s Curve s ψ x y s Ψ 12 3 4 5 6 x y 1 2 3 4 5 6 Ψ-s is periodic (2π) Horizontal section in Ψ-s curve => straight line in the Image Non horizontal section in Ψ-s => arc in Image π 2π

28. Slope-Density Function s ψ x yH(Ψ) Ψ π2π HISTOGRAM Lines Arcs

29. Slope-Density Function s ψ x yH(Ψ) Ψ π2π HISTOGRAM H(Ψ) shifts as objects rotates. H(Ψ) wraps around Lines Arcs

30. Fourier Descriptors s ψ x y Find: Ψ(s) Define: Φ(s)= Ψ(s)-(2πs)/P P: Perimeter. 2π: Period of Φ(s) Φ(s) is a Continuous, Periodic function. Fourier Series for Periodic Functions: Fourier Coefficients: Φ k ’s capture shape information Match shapes by matching Φ k ’s Use finite number of Φ k ’s

31. Fourier Descriptors Input Shape Power Spectrum Reconstruction # of coefficients ΦkΦk Note: Reconstructed shapes are often not closed since only a finite # of Φ k ’s are used.

32. B-Splines Piecewise continuous polynomials used to INTERPOLATE between Data Points. Smooth, Flexible, Accurate. s=0 s=1 s=2 x0x0 x1x1 x2x2 xixi x s Spline X(s) Data Point We want to find X(s) from points xi Cubic Polynomials are popular: BASIS FUNCTIONS COEFFICIENTS N

33. B-Splines Bi(s) has limited support (4 spans) i-2i-1ii+1 i+2 Each span (i->i+1) has only 4 non-zero Basis Functions: B i-1 (s), B i (s), B i+1 (s), B i+2 (s) ii+1 B i (s) B i-1 (s) B i+2 (s) B i+1 (s) t=0 t=1 t s:

34. B-Splines 3-D Computer Vision CSc 83020 – Ioannis Stamos ii+1 B i (s) B i-1 (s)B i+2 (s) B i+1 (s) t=0 t=1 t Cubic Polynomials 1/6 4/6

35. B-Splines 3-D Computer Vision CSc 83020 – Ioannis Stamos ii+1 B i (s) B i-1 (s)B i+2 (s) B i+1 (s) t=0 t=1 t Cubic Polynomials 1/6 4/6

36. B-Splines 3-D Computer Vision CSc 83020 – Ioannis Stamos If we compute v 0, … v N => continuous representation for the curve.

37. B-Splines We have our N+1 data points: And in matrix form: We can solve for the v i ’s from the x i ’s Boundary condition for closed curves: v 0 =v N, v 1 =v N+1.

38. B-Splines So, for any i we can find: xixi x i+1 x i(t) t t=0 t=1 Note: Local support. Spline passes through all data points xi. B-Spline demo: http://www.doc.ic.ac.uk/~dfg/AndysSplineTutorial/BSplines.html

39. Snakes Fitting a curve to an arbitrary shape. Active/Deformable contour or Snake.

40. Snakes Elastic band of arbitrary shape. Located near the image contour.

41. Snakes Attracted towards the target contour.

42. Snakes Stops when Energy Functional is minimized.

43. Corpus Callosum [Davatzikos and Prince]

44. Applications Example: Corpus Callosum [Davatzikos and Prince]

45. Demo using gimp “Intelligent scissors”

46. User-Visible Options Initialization: user-specified, automatic Curve properties: continuity, smoothness Image features: intensity, edges, corners, … Other forces: hard constraints, springs, attractors, repulsors, … Scale: local, multiresolution, global

47. Behind-the-Scenes Options Framework: energy minimization, forces acting on curve Curve representation: ideal curve, sampled, spline, implicit function Evolution method: calculus of variations, numerical differential equations, local search

48. Snakes: Active Contour Models Introduced by Kass, Witkin, and Terzopoulos Framework: energy minimization Bending and stretching curve = more energy Good features = less energy Curve evolves to minimize energy Also “Deformable Contours”

49. Snakes Energy Equation Parametric representation of curve Energy functional consists of three terms

50. Internal Energy First term is “membrane” term – minimum energy when curve minimizes length (“soap bubble”) Second term is “thin plate” term – minimum energy when curve is smooth

51. Internal Energy Control  and  to vary between extremes Set  to 0 at a point to allow corner Set  to 0 everywhere to let curve follow sharp creases – “strings”

52. Edge Attraction Gradient-based: Laplacian-based: In both cases, can smooth with Gaussian Snakes paper, figures 3 and 4

53. Corner Attraction Can use corner detector we saw last time Alternatively, let  = tan -1 I y / I x and let n  be a unit vector perpendicular to the gradient. Then Snakes paper, figures 5 and 6

54. Variants on Snakes Balloons [Cohen 91] Add inflation force Helps avoid getting stuck on small features

55. Balloons [Cohen 91]

56. Parameterized by arc-length [Kass, Witkan and Terzopoulos ’87] Minimize Snakes

57. Continuity – compactness Parameterized by arc-length [Kass, Witkan and Terzopoulos ’87] Minimize Snakes

58. Continuity – compactness Parameterized by arc-length [Kass, Witkan and Terzopoulos ’87] Minimize Snakes

59. Continuity – compactness Parameterized by arc-length [Kass, Witkan and Terzopoulos ’87] Minimize Smoothness Snakes

60. Continuity – compactness Parameterized by arc-length [Kass, Witkan and Terzopoulos ’87] Minimize Smoothness Edge Attraction Term Snakes

61. Corner Attraction Can use corner detector we saw last time Alternatively, let  = tan -1 I y / I x and let n  be a unit vector perpendicular to the gradient. Then Snakes paper, figures 5 and 6

62. Problem Statement: Start with chain of image locations representing initial position of contour. Find the deformable contour by minimizing the energy functional: Snakes

63. Snakes – Implementation How can we minimize  Discrete version: sum over the points. Greedy minimization: For each point compute the best move over a small area.

64. Snakes – Implementation How can we minimize  Discrete version: sum over the points. Greedy minimization: For each point compute the best move over a small area. Set βj=0 for points of high curvature (corners)

65. Snakes – Implementation How can we minimize  Discrete version: sum over the points. Greedy minimization: For each point compute the best move over a small area. Set βj=0 for points of high curvature (corners) Stop when a user-specified fraction of points does not move.

66. Synthetic Results Real Experiment Snakes demo: http://www.markschulze.net/snak es/

67. Scale In the simplest snakes algorithm, image features only attract locally Greater region of attraction: smooth image Curve might not follow high-frequency detail Multiresolution processing Start with smoothed image to attract curve Finish with unsmoothed image to get details Looking for global minimum vs. local minima

68. Diffusion-Based Methods Another way to attract curve to localized features: vector flow or diffusion methods Example: Find edges using Canny For each point, compute distance to nearest edge Push curve along gradient of distance field

69. Gradient Vector Fields Xu and Prince

70. Gradient Vector Fields Xu and Prince Simple Snake With Gradient Vector Field

71. Gradient Vector Fields Xu and Prince

72. Region Representation 3-D Computer Vision CSc 83020 – Ioannis Stamos 1 1 11111 11111 1111 1 Spatial Occupancy Array Easy to implement. Large Storage area. Can apply set operations (unite/intersect). Expensive for matching!

73. 3-D Computer Vision CSc 83020 – Ioannis Stamos Quad Trees Efficient encoding of Spatial Occupancy Array using Resolution Pyramids. Black: Fully Occupied. White: Empty. Gray: Partially Occupied.

74. 3-D Computer Vision CSc 83020 – Ioannis Stamos Quad Tree Level 0 Level 1 Level 2 Level 3 Quad Tree Generation: Start with level 0. If Black or White, Terminate. Else declare Gray node & expand with four sons. For each son repeat above step. NWNE SWSE