1. Active Contour Models (Snakes)
By: Sajjad Fouladvand
Professor : Dr Alireza Ossareh
December 16th, 2011
Shahid Chamran University Of Ahvaz

2. Overview Introduction Modeling Simple Demo Weakness
An Active Contour Balloon Model
Gradient Vector Flow (GVF)
Applications of snakes
References 1

3. Introduction What is the objective? What is segmentation?
To perform the task of Image Segmentation.
What is segmentation?
Subdividing or partitioning an image into its constituent regions or objects. 2

4. Introduction
First introduced in 1987 by Kass et al, and gained popularity since then.
Represents an object boundary or some other salient image feature as a parametric curve.
An energy functional E is associated with the curve.
The problem of finding object boundary is cast as an energy minimization problem. 3

5. Introduction
A higher level process or a user initializes any curve close to the object boundary.
The snake then starts deforming and moving towards the desired object boundary.
In the end it completely “shrink-wraps” around the object. 4

6. Simple modeling
The snake deforming and moving towards the desired object boundary. 5

7. Simple modeling Represent the curve with a set of n points :
𝑣 𝑖 = 𝑥 𝑖 , 𝑦 𝑖 𝑖=0 , …, 𝑛−1 6

8. Simple modeling The total energy of the snake is defined as
𝐸 𝑡𝑜𝑡𝑎𝑙 = 𝐸 𝑖𝑛𝑡𝑒𝑟𝑛𝑎𝑙 + 𝐸 𝑒𝑥𝑡𝑒𝑟𝑛𝑎𝑙
Encourages smoothness or a particular shape
Encourages curve onto
image structures (e.g edges) 7

9. Simple modeling
The energy terms are defined cleverly in a way such that the final position of the contour will have a minimum energy ( 𝐸 𝑚𝑖𝑛 )
Therefore our problem of detecting objects reduces to an energy minimization problem. 8

10. Modeling - internal energy
The snake should not break apart!
Points on the snake must stay close to each other
Each point on the snake pulls its neighbors
The force is proportional to the distance |Pi – Pi-1| 9

11. Modeling- internal energy
For a curve represented as a set of points a
simple elastic energy term is :
𝐸 𝑖𝑛𝑡𝑒𝑟𝑛𝑎𝑙 =𝛼 𝑖=0 𝑛−1 𝐿 𝑖 2
=𝛼 𝑖=0 𝑛−1 ( 𝑥 𝑖+1 − 𝑥 𝑖 ) 2 +( 𝑦 𝑖+1 − 𝑦 𝑖 ) 2
This encourages the closed curve to shrink to a
point (like a very small elastic band) 10

12. Modeling- external energy
Edge pixels must “pull” the snake points.
The stronger the edge, the stronger the pull.
• Numerous forms can be used, attracting the
curve toward different image features 11

13. Modeling - external energy
Suppose we have an image 𝐼 𝑥,𝑦
Generate gradient images 𝐺 𝑥 𝑥,𝑦 , 𝐺 𝑦 𝑥,𝑦
The absolute edge strength at a point is then
𝐺 𝑥 (𝑥,𝑦) 𝐺 𝑦 (𝑥,𝑦) 2
An external energy term for a snake is thus:
𝐸 𝑒𝑥𝑡𝑒𝑟𝑛𝑎𝑙 =− 𝑖=0 𝑛−1 𝐺 𝑥 (𝑥,𝑦) 𝐺 𝑦 (𝑥,𝑦) 2
(Negative in order that minimizing it forces the curve toward large edges) 12

14. Modeling - Simple Elastic Snake
A simple elastic snake is thus defined by
a set of n points
An internal elastic energy term
An external edge based energy term
• To use this to locate the outline of an object
Initialize in the vicinity of the object
Modify the points to minimize the total energy 13

15. Snake Energy Functional
Given a snake with N points 𝑣1, 𝑣2, …, 𝑣𝑁
Define the following Energy Functional:
𝐸= 𝑖=1 𝑁 𝑎 𝑖 𝐸 𝑐 ( 𝑣 𝑖 )+ 𝑏 𝑖 𝐸 𝑠 𝑣 𝑖 + 𝑐 𝑖 𝐸 𝑔 ( 𝑣 𝑖 )
Where:
“Continuity”
𝐸 𝑐
𝐸 𝑠
“Smoothness”
𝐸 𝑔
“Edgeness”
ai, bi, ci are “weights” to control influence 14

16. Snake Forces
Edge attraction
Continuity
Smoothness 15

17. Distance between points should be kept close to average
Continuity Term
Given a snake with 𝑁 points 𝑣1, 𝑣2, …, 𝑣𝑁
Let d be the average distance between points
Define the continuity term of the Energy Functional:
𝐸 𝑐 𝑣 𝑖 = (𝑑− 𝑣 𝑖 − 𝑣 𝑖−1 ) 2
Distance between points should be kept close to average 16

18. Smoothness Term 𝐸 𝑠 𝑣 𝑖 = 𝑣 𝑖−1 −2 𝑣 𝑖 + 𝑣 𝑖+1 2
Given a snake with N points 𝑣1, 𝑣2, …, 𝑣𝑁
Curvature should be kept small
Define the smoothness term of the Energy Functional:
𝐸 𝑠 𝑣 𝑖 = 𝑣 𝑖−1 −2 𝑣 𝑖 + 𝑣 𝑖+1 2
Second derivative
This term prevent Local minimum 17

19. Smoothness Term- Example
(2,5)
(2,2)
(3,1)
(1,1)
(3,1)
(1,1)
3− −
3− −
= −8 2=64
= −2 2=4 18

20. Edgeness Term 𝐸 𝑔 𝑣 𝑖 =− 𝛻𝐼( 𝑣 𝑖 )
Given a snake with N points 𝑣1, 𝑣2, …, 𝑣𝑁
Define the edgeness term of the Energy Functional:
𝐸 𝑔 𝑣 𝑖 =− 𝛻𝐼( 𝑣 𝑖 )
Magnitude of the gradient should be LARGE 19

21. Energy minimization
Several algorithms have been proposed to fit deformable contours:
Greedy search
Dynamic programming 20

22. Energy minimization- greedy
For each point, search window around it and move to where energy function is minimal
Typical window size, e.g., 5 x 5 pixels
Stop when predefined number of points have not changed in last iteration, or after max number of iterations
Note:
Convergence not guaranteed
Need decent initialization 21

23. Model
The most basic way of representing a curve in two dimensions is the explicit form: 𝑦 = 𝑓 𝑥
Nearly all simple curves can be represented using the explicit form, but for more complex curves with two or more y-values for a single corresponding x-value the explicit form fails: 𝑓 𝑥; 𝑦 = 0
Unfortunately the inherent form of the implicit curve equation
Unfortunately the inherent form of the implicit curve equation does not allow us to compute points on the curve directly, often requiring the solution of a non-linear equation for each point.
𝑉(𝑠)=(𝑥(𝑠),𝑦(𝑠)) 𝑢𝑠𝑢𝑎𝑙𝑙𝑦 𝑤𝑖𝑡ℎ 𝑠∈[0,1] 22

24. Model
𝐸= 𝑠 𝛼(𝑠) 𝑣 𝑠 2 +𝛽(𝑠) 𝑣 𝑠𝑠 2 + 𝐸 𝑒𝑥𝑡 𝑣 𝑠 𝑑𝑠 𝐸 𝑒𝑥𝑡 = 𝑠 𝐸 𝑖𝑚𝑎𝑔𝑒 𝑣 𝑠 𝑑𝑠
Total internal energy of the snake
𝐸 𝑒𝑥𝑡 =− 𝛻𝐼(𝑥,𝑦) 2
𝐸 𝑒𝑥𝑡 =− 𝛻( 𝐺 𝜎 𝑥,𝑦 ∗𝐼(𝑥,𝑦)) 2 23

25. Model
(a) The original image 𝐼(𝑥; 𝑦), showing a black square on a white background. (b) The image energy derived from the function 𝐸 𝑒𝑥𝑡 =− 𝛻 𝐺 𝜎 𝑥,𝑦 ∗𝐼 𝑥,𝑦 with σ= 9, shown as a 3D surface. (c) The same image energy as seen in. 24

26. Model This can be viewed as a force balance equation
A snake that minimizes E must satisfy the Euler equation:
𝜎 𝑣 𝑠𝑠 −𝛽 𝑣 𝑠𝑠𝑠 −𝛻 𝐸 𝑖𝑚𝑎𝑔𝑒 =0
This can be viewed as a force balance equation
𝐹 𝑖𝑛𝑡 + 𝐹 𝑒𝑥𝑡 1 =0
Each term corresponds to a force produced by the respective energy terms. The contour deforms under the action of these forces. 25

27. Simple Demo There is a simple demo for explain snake energy 26

28. Weakness
Fails to detect concave boundaries. External force cant pull control points into boundary concavity.
the initial contour must, in general, be close to the true boundary or else it will likely converge to the wrong result. 27

29. An Active Contour Balloon Model
The active contour model that Kass et al. introduced was further developed by Laurent D. Cohen in the paper “On Active Contour Models and Balloons“ [Cohen-91].
Works on the same principles as the Kass et al. snake.
where the Kass et al. snake would shrink when not under the influence of image forces the Cohen snake expands. 28

30. An Active Contour Balloon Model
Introduces makes it possible to find the contours of an object by placing the initial snake inside the object instead of outside, while also providing more stable results. 29

31. Gradient Vector Flow (GVF)
A new external force for snakes
A snake with GVF external forces moves into the concave boundary region. 30

32. Gradient Vector Flow (GVF)
The contour can also be initialized across the boundary of object!!Something not possible with traditional snakes.
also extended GVF to three dimensions 31

33. Model for GVF snake The GVF field is defined to be a vector field
𝑉(𝑥,𝑦) = (𝑢(𝑥,𝑦),𝑣(𝑥,𝑦))
𝑉(𝑥,𝑦) is defined such that it minimizes the energy functional
Force equation of GVF snake
𝛼 𝑣 𝑠𝑠 −𝛽 𝑣 𝑠𝑠𝑠 +𝑣=0
f(x,y) is the edge map of the image. 32

34. Model for GVF snake
GVF field can be obtained by solving following equations
2 Is the Laplacian operator
The above equations are solved iteratively using time derivative of u and v. 33

35. Traditional external force field v/s GVF field
Traditional force
GVF force
Three general properties of edge maps are important in the present context. First, the gradient of an edge map has vectors pointing toward the edges, which are normal to the edges at the edges. Second, these vectors generally have large magnitudes only in the immediate vicinity of the edges. Third, in homogeneous regions, where is nearly constant, is nearly zero.
Because of the first property, a snake initialized close to the edge will converge to a stable configuration near the edge. This is a highly desirable property. Because of the second property, however, the capture range will be very small, in general. Because of the third property, homogeneous regions will have no external forces whatsoever. These last two properties are undesirable 34

36. Traditional external force field v/s GVF field
Although the external forces correctly point toward the object boundary, within the boundary concavity the forces point horizontally in opposite directions.
We see that the magnitude of the external forces die out quite rapidly away from the object boundary. 35

37. Results
Traditional snake
GVF snake 36

38. Results Notice that the image is poor quality with sampling artifacts
Magnetic resonance image of the left ventricle of human heart
Notice that the image is poor quality with sampling artifacts 37

39. Problem with GVF snake Very sensitive to parameters.
Slow. Finding GVF field is computationally expensive. 38

40. Applications of snakes
Image segmentation particularly medical imaging community (tremendous help).
Motion tracking.
Stereo matching .
Shape recognition. 39

41. References
M.KASS, A.WITKIN, D.TERZOPOULOS, “Snakes: Active Contour Models”, International Journal of Computer Vision, (1988)
Ch.Xu, J.L. Prince, “Gradient Vector Flow: A New External Force for Snakes”, The Johns Hopkins University
Ch.Xu, J.L. Prince, “Snakes, Shapes, and Gradient Vector Flow”, IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 7, NO. 3, MARCH 1998
N.P Tiilikainen, “A Comparative Study of Active
Contour Snakes”, computer science department copenhagen University, 2007 40

42. With special thanks

43. Any question? Any idea