0. Chapter 7 – Segmentation II
7.1. Mean Shift Segmentation
7.2. Active Contour Models – Snakes
7.3. Geometric Deformable Models – Level
Sets and Geodesic Active Contours
7.4. Fuzzy Connectivity
7.5. Towards 3D Graph-Based Image Segmentation
7.6. Graph Cut-Segmentation
7.7. Optimal Single and Multiple Surface
Segmentation

1. 7.1. Mean Shift Segmentation Idea of mean shift:
1. For each data point Fix a window around the data point. Compute the mean of data within the window. 2. Shift data points to their computed means simultaneously 3. Repeat till convergence.
7-1

2. Window
Center of
mass
Mean Shift
vector
7-2 2

3. 7-3 3

4. 7-4 4

5. 7-5 5

6. Data points Objective: Find clusters Find modes Density estimator
Density distribution

7. In practice, radially symmetric kernels are used, i.e.,
Density kernel K(x)
Uniform
Epanechnikov
Normal
In practice, radially symmetric kernels are used, i.e.,
where k : profile of kernel K
7-7

8. Profile of kernel K(x)
Uniform
Epanechnikov
Normal

9. Given n data points in d-D space , the density estimation at point x :
We are interested in locating x for which
The density gradient:
Use profile, i.e.,
(See next page)

10. Let
Then
7-10

11. Assume the derivative of k is
let

12. Let g(x) be the profile of kernel G(x), i.e.,
The first term is proportional to a density
estimator
computed with
kernel
7-12

13. The second term is the mean shift vector
The successive location of the kernel G are
7-13

15. Example: Mode Detection
Feature vector of pixel i,
(L u v)-
color
space
Color
image

16. Example: Image Segmentation
Feature vector of pixel i,
where : spatial-domain part, : range-domain part
e.g.,

17. Two steps of mean-shift image segmentation:
1) Discontinuity Preserving Filtering
-- Preserves discontinuities of images
7-17

18. Let : pixels of the original image : pixels of the filtered image
2) Mean Shift Clustering
-- Regularizes regions
Given
: the mode associated with
known from Step 1.
7-18

19. where : segmentation label of pixel i
: convergence points known from
Step 1
BOA : Basin Of Attraction, the set of
all locations that converge to
the same mode

20. Examples: (1) Image Segmentation
(2) Image Smoothing and Segmentation
Original image
Smoothed image
Segmentation image
7-20

21. Gary levels of original image The path of mean shift
Gary levels after smoothing
Gary levels after segmentation
7-21

22. 7.2. Active Contour Models – Snakes
-- An energy-minimization approach
-- (i) A snake is a deformable model whose energy
depends on its shape and location in the image
(ii) Local minima of the energy correspond to
desired image properties.
(iii) Initial snake position should be provided.

24. 7.2.1 Traditional Snakes and Balloons
The energy functional of the snake, which is to be
minimized is a weighted combination of internal
and external forces
The internal forces emanate from the shape of the snake
The external forces come from the image and
constraints

25. The image forces may come from lines, edges, and
terminations

26. The constraint forces come from user
specified properties to be imposed on the snake,
e.g., smmothness.

27. To minimize
, let
This leads to the Euler-Lagrange motion equation
Substitute
into the equation
written this equation in an evolution form
Stop when

28. A balloon, which can be inflated, is extended from the snake by including an additional pressure force so that it can overcome small isolated barriers
7-28

29. 7.2.3 Gradient Vector Flow Snakes
Difficulties of previous approaches:
(i) initialization , (ii) concaves of boundaries
GVF: An external force field points toward
boundaries when in their proximity

30. GVF is derived from
image by minimizing an energy functional
g can be obtained by solving the following Euler equations, which are diffusion equations
Rewrite the above equations in an evolution form

31. Let in Eqs. (7.15) and (7.22) forming the GVF snake equation
7-31

32. 3D GVF:

34. Deformable models, Fourier deformable model,
7.2.2 Extensions
Deformable models, Fourier deformable model,
Finite element snakes, B-snakes, united snakes.
7-34

35. 7.3. Geometric Deformable Models (GDM) –
Level Sets and Geodesic Active Contours
2 main groups of deformable models:
1. Parametric deformable model: Borders are
represented in a parametric form, e.g., snakes
2. Geometric deformable model : Borders are
represented by partial differential equations,
e.g., level sets, geodesic active contours
Advantages of GDM:
(i) Curves are evolved using only geometric
computations, independent of any
parameterization

36. (ii) Curves are represented as level sets of higher
dimensional functions yielding seamless
treatment of topological changes
(iii) Multiple object can be detected simultaneously
Let a curve be denoted by
or by
positional vector
The associated length function:
: speed function

37. Assume the curve moves only in a direction normal (N) to itself, i.e.
Assume also the speed function is a function of
curvature c,
7-37

38. Constant Deformation Equation
-- Deformation is similar to inflation balloon force and
may introduce singularities (e.g., sharp corners)
7-38

39. Curvature Deformation Equation
7-39

40. Combined constant and curvature deformation:
Let
Then, k -> 0 when the curve approaches edges. Strong edges will stop the curve propagation.
The above geometric deformable model starts with
an initial curve and evolves its shape using a speed
function. Such an evolution can be implemented
using level sets technique.
7-40

41. Level set function is a higher-dimensional
function, which represents a boundary as a level set.
The curve at time t is the set of image points, for
which the value of the level set function at time t
is equal to zero

42. Using the level set representation of a curve allows its evolution by updating the level set function.
7-42

43. A level set function with contour
as its zero-level set
Differentiate w.r.t. t

44. Experiment:
7-44 44

45. : the initial contour
Level set function:
: the edge magnitude
7-45 45

46. 7.4. Fuzzy Connectivity Uncertainties for image segmentation:
Noise, uneven illumination, limited spatial resolution,
partial occlusion, etc.
Fuzzy connectivity segmentation keeps considering
the likelihood (i.e., hanging togetherness) of whether
nearby pixels belong to the same object
Fuzzy affinity : a strength of hanging togetherness
of nearby pixels, which is a function of distance
between two adjacent pixels.
: fuzzy adjacency
e.g.,
f : image properties

47. Fuzzy adjacency :
Fuzzy
connectedness :
7-47

48. Connectedness map :
c : seed; : any pixel
7-48

49. 7-49

50. 7-50

51. Thresholding yields the segmentation result.
7-51

52. Relative fuzzy connectivity,
Extensions:
Relative fuzzy connectivity,
7-52

53. Multi-seeded fuzzy connectivity
Scale-based fuzzy connectivity
7-53

54. 7.6. Graph Cut-Segmentation
Idea:
Seeds may be
interactively or
automatically
identified.

55. Steps: Given an image of size n
(1) Construct a graph with n+2 nodes
(2) Each node has 4 n-links, connected
to neighbors. Each link is assigned
a cost derived from smoothness term
, where f: image value
(3) Each node has 2 t-links, connected
to source S and sink T. Each link is
assigned a cost derived from data
term
7-55

56. (4) Identify seed pixels and define “hard constraint” for their links
(5) Find the minimum cut that
minimizes
(6) Obtain the segmentation result.
7-56

57. 7-57

58. Example: Given an image
Let I : the set of pixels
N: a set of neighborhood pixel pairs
: the resulting labeling vector
L is obtained by finding the cut that minimizes the
cost where
: regional property term
: boundary property term
7-58

59. The costs of arcs are defined in the following table
e.g.,
7-59

60. 7-60

61. Example:
7-61 61