Active Contours, Level Sets, and Image Segmentation Chunming Li Institute of Imaging Science Vanderbilt University URL: www.vuiis.vanderbilt.edu/~licm E-mail: chunming.li@vanderbilt.edu

Outline Curve evolution and level set methods: 4/11/2017 Outline Curve evolution and level set methods: Curve Evolution: from snake to general curve evolution Level Set Methods: basic concepts and methods Numerical issues: Difference scheme: upwind scheme, ENO interpolation… Reinitialization, velocity extension … Introduction to calculus of variations and applications: Segmentation: variational level set, shape prior … Denoising: diffusion, TV, …

4/11/2017 2D Level Set Click to play the movie Click to play the movie

4/11/2017 3D Level Set Click to play the movie Click to play the movie

More Applications of Level Sets 4/11/2017 More Applications of Level Sets Fluid Dynamics Computer Graphics: Surface Rendering, Animation… Material Science Others…

Advantages of Active Contours and Level Sets 4/11/2017 Advantages of Active Contours and Level Sets Nice representation of object boundary: Smooth and closed, good for shape analysis and recognition and other applications. Sub-pixel accuracy. Can incorporate various information such as shape prior and motion. Mature mathematical tools can be used: calculus of variations, PDE, differential geometry …

Snake Models

Image Segmentation: Classic Methods 4/11/2017 Image Segmentation: Classic Methods An image of blood vessel Thresholding Edge detection

Parametric Active Contours (Kass et al 1987) 4/11/2017 Parametric Active Contours (Kass et al 1987) For a contour , define energy:

Evolution of Active Contours Steepest descent: Internal force: Regularizes the contour. External force: Drives the contour towards the desired object boundary. In the classic snake model, the external force field is:

External Force

Gradient Vector Flow (Xu and Prince, 1998) Find a vector field v(x,y) that minimize the energy functional: Solve two decoupled PDEs:

An Example Gradient vector flow

Advantages of GVF Snake Significantly increase capture range Be able to detect concave object boundary

General Curve Evolution

Differential Geometry of Curves A planar curve C is a function , with Tangent vector Normal vector Arc length between and

Curvature Use arc length as the parameter. The tangent vector is a unit vector The unit vector defines the direction of the curve Curvature describes how fast the curve changes its direction. Take derivative again: Note that

Curvature (Cont’d) Both and are orthogonal to is parallel to For general parameterization the curvature can be expressed as:

Dynamic Curves A dynamic curve is a time dependent curve The motion of the curve is governed by a curve evolution equation: The tangent vector and normal vector forms a basis of .

Geometric Curve Evolution Theorem: Let be an intrinsic quantity. If evolves according to Then, there exists another parameterization of such that is solution of where at the same point General geometric curve evolution:

Constant Speed Motion and Mean Curvature Motion 4/11/2017 Constant Speed Motion and Mean Curvature Motion Mean curvature motion (Length shortening flow) Constant Speed Motion (Area decreasing/increasing) Click to play the movie Click to play the movie

Mean curvature motion (Length shortening flow) 4/11/2017 Mean curvature motion (Length shortening flow) Constant Speed Motion (Area decreasing flow)

Why MCM Shortens Curve Length? 4/11/2017 Why MCM Shortens Curve Length? Mean curvature motion is the steepest descent flow (or gradient flow) that minimizes arc length of the contour: Click to play the movie

Geodesic Active Contour (Caselles et al, 1997) 4/11/2017 Geodesic Active Contour (Caselles et al, 1997) Euclidean metric: Minimize a weighted length of C where This is a Riemannian metric. How to find the geodesic? Solve the gradient flow equation:

Geodesic Active Contour 4/11/2017 Geodesic Active Contour How does the evolution stop the contour at the edges? External force The external force has two opposite directions on the two sides of the object boundary. The curvature term regularizes the curve: shorten and smooth. Problem: The contour cannot progress into concavity. Add a balloon/pressure force:

4/11/2017 Example Click to play the movie

Difficulties of Parameterized Curve Evolution Re-parameterization during evolution: very difficult for 3D surface Cannot handle topological changes

Solutions Some approaches within parametric active contour framework: McInerney and Terzopoulos, 1995 Ray and Acton, 2003 Li, Liu, and Fox, 2005 More natural and better approaches: Level set methods

Level Set Methods

Level Set Representation of Curves 4/11/2017 Level Set Representation of Curves zero level zero level

Advantages of Level Set Methods All computation is carried out on a fixed grid, no need for re-parameterization. Topological changes are handled naturally. More … Also have some disadvantages, will be discussed later.

Level Set Functions contour 4/11/2017 Level Set Functions contour Level set functions with the same zero level set

Signed Distance Function Contour C

Useful Calculus Facts in Level Set Formulation If a curve is a level set of a function then, the normal vector and the curvature can be computed from the embedding function The surface (or line) integral of a quantity p(x) over the zero level surface (or contour) of The volume (or area) integral of p(x) over

From Curve Evolution to Level Set Evolution 4/11/2017 From Curve Evolution to Level Set Evolution Curve evolution where F is the speed function, N is normal vector to the curve C Embed the dynamic curve as the zero level set of a time dependent function , i.e. Take derivative with respect to time Level set evolution:

Special Cases Mean curvature flow Constant Speed Motion 4/11/2017 Special Cases Mean curvature flow Constant Speed Motion Geodesic active contours

4/11/2017 Demo of GAC Click to play the movie

Unstable Evolution and Need for Reinitialization 4/11/2017 Unstable Evolution and Need for Reinitialization Zero level set of degraded level set function Degraded level set function, 50 iterations, time step=0.1

Procedure of Standard Level Set Methods 4/11/2017 Procedure of Standard Level Set Methods Reinitialization Initialization Evolve level set function converge? N Y stop

More Details Difference Scheme: finite difference, upwind scheme. 4/11/2017 More Details Difference Scheme: finite difference, upwind scheme. ENO interpolation Velocity Extension Narrow Band Implementation More… Next time

4/11/2017 Questions

Thank you