1. Snakes - Active Contour Lecturer: Hagit Hel-Or
By: Saar Arbel
Lecturer: Hagit Hel-Or

2. What is our objective? But Wait...
To find a contour that best approximates the perimeter of an object
Subdividing or partitioning an image into its constituent regions or objects
But Wait...
Its just like segmentation and many other methods...

4. Problem with other methods
Not effective in presence of noise and sampling artifacts (e.g. medical images).
This is done with segmentation on a noisy image

5. This is done with snakes on a noisy image

6. What is a snake?
A framework for drawing an object outline from a possibly noisy 2D image.
An energy-minimizing curve guided by external constraint forces and influenced by image forces that pull it towards features (lines, edges).
Represents an object boundary or some other salient image feature as a parametric curve

8. So... Why snakes?
Snakes are autonomous and self-adapting in their search for a minimal energy state
They can be easily manipulated using external image forces
They can be used to track dynamic objects in temporal as well as the spatial dimensions

10. Applications Object tracking Shape recognition Segmentation
Segmentation example
Shape recognition
Segmentation
Edge detection

11. Aspects we need to consider
Representation of the contours
Defining the energy functions
- Internal
- External
Minimizing the energy function

12. How is a snake represented?
A Snake is represented as a set of points (x,y) that defines a function V(s).

13. Every snake include: External Energy Function Internal Energy Function
A set of k points (in the discreet world)
or a continuous  function that will represent the points

14. The Energy functions
The Snake's energy functions measure the appropriateness of the contour
Good solutions correspond to the minimum of the energy functions
Getting the snake closer to the target
In charge of the snake curvature and smoothness
Our goal is to minimize this function with respect to the contour parameters

15. A brief example:

16. The Energy functions
Our goal is to find a function V that will minimize E and give us a perfect match

17. Intuitive tip
You can think about the energy function like a heuristic function, they are similar because they both lead to a solution in a way that lets the programmer decide what is important and what’s not.

18. Internal Energy Econt Ecurv Forces the contour to be continuous
Forces the contour to be smooth
Decide if the snake wants to shrink/expand
Econt
Ecurv

19. Econt Forces the contour to be continuous
Minimizing the first derivative has two effects:
The snake wants to shrink
The squaring of  the differences causes the differences to be as similar as possible (the points are at equal distances from each other)

20. On the continuous world:
On the discrete world:
The contour is approximated by N points P1, P2, . .., Pn and the first derivative is approximated by a finite difference:

21. Example The points are equi-distances The points are
not equi-distances

22. Ecurv Forces the contour to be smooth.
By taking the second derivative which represent the difference of the differences  between 2 consecutive differences.
This 2 consecutive differences represent the curvature and we want to penalize if the curvature is too high.

23. On the continuous world:
On the discrete world:
The curvature can be approximated by the following finite difference:

24. Example: compare curvature
That’s better!

25. α,β meaning
Reminder:
We can play with α,β and to control the relation between the first derivative and the second derivative and get different results

26. Small α
Big α
Small alpha make the energy function insensitive to the amount of stretch
Increase the internal energy of the snake as it stretches more and more

27. Small β
Big β
Small beta causes snake to allow large curvature so that snake will curve into bends in the contour
When beta is large then high price for curvature so snake prefers to be smooth and not curving

28. Internal energy - Summary On the continuous world:
On the discrete world:

29. Discrete representation
Represent the snake with a set of n points
(X0 , Y0)
(X3 , Y3)
(X14 , Y14)

30. Open and close snake
closed snake
open snake
Dealing with open snake is easier and our algorithm will run faster (soon…)

31. External Energy – The Energy map
An energy map is a function f (x, y) that we extract from the image – I(x, y)
The energy map’s task is to emphasize the areas of the contour and to weaken the other areas of the image by creating a new image.
By doing this we can attract our snake near the “emphasized” pixels which represent the contour of the image.

32. How do we build an Energy map?
By given an image – I(x, y) , we can build an energy map – f(x, y), that will attract our snake to edges on our image.
For each pixel we will do:
(color image)
(black-white image)
By doing this we can easily locate the edges on the images because the gradient there is big.

34. Energy map
Image

35. External Energy
Attracts the contour towards the closest image edge with dependence on the energy map.
determines whether the snake feels attracted to object boundaries

36. Intuitive example
Initial state
Final state

37. Eexternal
In order to make snakes useful for early vision we need energy functions that attract them to salient features in images.
I will present you 2 different energy functions which attract a snake to lines & edges.

38. Eline
Depending on the sign of Wline the snake will be attracted either to bright lines or dark lines.

39. Eedge
Reminder: Gradient is a generalization of the usual concept of derivative to the functions of several variables

40. Clarification
Note: As long as the points on the snake are closer to the edge (where the gradient is bigger) and then the Eedge is smaller, so that’s a good thing because that's how the external energy works! this gets the snake closer and closer to the target contour.

41. Examle
Image
External force
Zoomed in

42. External energy - Summary On the continuous world:
On the discrete world:

43. Example – External energy

44. How to find the optimal function V?
Several algorithms have been proposed to fit
deformable contours.
We’ll look at two:
-  Greedy search
-  Dynamic programming (for 2d snakes)

45. Greedy algorithm
A greedy algorithm makes locally optimal choices, hoping that the final solution
will be globally optimal.
For each point of the snake, we search a window around it (window size: m) and move to where the energy function is minimal.
We will stop after enough points have not changed in the last iteration 

46. Points are calculates by an iterative process:
Energy function for each point in the local neighborhood is calculated
The point is moved to the next point with lowest energy function
This process is repeated for every point
Iteration is done until termination condition met
Defined number of iterations
Complexity:

47. But... Convergence not guaranteed!! Need decent initialization
Works very well as far as the initial snake is not too far from the desired solution

49. Viterbi algorithm Dynamic programming algorithm
We need to make the snake discrete in order to use the algorithm
The discretization of the external energy depends on the particular energy term but the internal energy is always the same and can be discretized as follows:

50. Our problem is to finding the set of vertex positions V that minimizes:

51. (The internal energy is a local property of neighboring vertices)
We will now make the assumption
that the snake is open and that β=0
In most cases, snake energy can be rewritten as a sum of pair-wise interaction potential:
(The internal energy is a local property of neighboring vertices)

52. This means that each vertex position Vi influences the total energy only through the terms and
Each vertex has m possible moves

53. We now define n variables (i=2,3,…,n)
Each contains the lowest total energy for the first k-1 vertices of the snake for a given
value of . Thus the minimum energy E of the whole snake is equal to

54. The globally best position for the snake is therefore computed by first computing all the , so we determine at each node the optimal position of its predecessor for each possible location of the node.
When we have computed we can find the optimal position for by minimizing the expression Once we know the position of the last node we look up the optimal position for the second last node and so
on until we have determined the optimal position for all the nodes.

55. states 1 2 … m
sites
Complexity:

56. Now, β≠0
If β≠0 then the decomposition of the energy function into local terms becomes:
This means that each vertex position Vi influences the total energy only through the terms and

57. And now we will define again n variables (i=2,3,…,n)
Note that we have to compute a table of values at each node instead of only m values per node for the simpler snakes with β=0.
Complexity:

58. Now, the snake is closed Solution:
1. Can use Viterbi to optimize snake energy in case
v1 = c is fixed. (in this case the energy above
effectively has no loop)
2. Use Viterbi to optimize snake for all possible
values of c and choose the best of the obtained
m solutions.
Complexity:

60. Snakes - Pros Vs. Cons Pros:
Snakes optimize globally.
Snakes are 1-dimensional which means that we can reduce 2-dimensional optimization
problems to 1-dimensional optimization problems.
Active contour models provide a unified solution to several image processing problems such as the detection of light and dark lines and edges.
Useful to track and fit non-rigid shapes.
Flexibility in how energy function is defined.

61. Cons:
Must have decent initialization near true boundary, may get stuck in local minimum.
Parameters of energy function must be set well based on prior information.

62. Extensions We will review two main applications of snakes:
-  Snake splitting
-  Snake expending

63. Snake Splitting Problem:
Snakes including multiple objects can not always extract each of them individually (when initialize the contour wrong)
Solution:
Using the split-and-merge contour model based on detecting self- and mutual-crossings of the contour.
The method:
There is a single contour which surrounding all objects and then iteratively splits into multiple contours by cutting at self-crossing points for extracting multiple objects individually

65. Snake Expanding
There is some cases when our snakes would need to expand in order to get close to the contour.

66. Problems:
If there is no image force (F=0), the curve shrinks on itself and vanishes to a point.
Often, due to noise, some isolated points are gradient maxima and can stop the curve when it passes by.
We can get stuck inside a shape (if our image force=0) So we need the define new force for this cases:

67. Solution:
we add another force which makes the contour have a more dynamic behavior.
You can think about our curve as a “Balloon” Force which inflate our snake
The new Force

68. Cool demonstrations

70. Summary Snakes include: - Two energy functions (Internal and External)
- N points who represents a function V
Main problem:
Finding an optimal function V that minimize the Energy function
Solved by:
- Greedy algorithm
- Viterbi algorithm

71. References
M. Kass, A. Witkin, and D. Terzopoulos, "Snakes:
Active contour models.“, International Journal of
Computer Vision. v. 1, n. 4, pp , 1987. 1. 3.
Wikipedia – Active contour models (Snakes) 2.
ormableContours.pdf 4.
handouts/snakes.pdf 5. 6.

72. Thank you