1. Johann Radon Institute for Computational and Applied Mathematics: www.ricam.oeaw.ac.at 1/28 Signal- und Bildverarbeitung, 323.014 Image Analysis and Processing Arjan Kuijper 07.12.2006 Johann Radon Institute for Computational and Applied Mathematics (RICAM) Austrian Academy of Sciences Altenbergerstraße 56 A-4040 Linz, Austria arjan.kuijper@oeaw.ac.at

2. Johann Radon Institute for Computational and Applied Mathematics: www.ricam.oeaw.ac.at 2/28 Last week There is a strong analogy between curve evolution and PDE based schemes. They can be related directly to one another. Euclidean shortening / Mean Curvature Motion involves the diffusion to be limited to the direction perpendicular to the gradient only. The divergence of the flow in the equation is equal to the second order gauge derivative L vv with respect to v, the direction tangential to the isophote. Implementation with Gaussian derivatives may allow larger time steps

3. Johann Radon Institute for Computational and Applied Mathematics: www.ricam.oeaw.ac.at 3/28 Today Morphology The Mumford – Shah Functional Active Contours / Snakes

4. Johann Radon Institute for Computational and Applied Mathematics: www.ricam.oeaw.ac.at 4/28 Mathematical Morphology Mathematical morphology is one of the oldest image processing and analysis techniques. The original idea is the application of a logical area- operator (called structuring element) on areas of the image in the same way as convolution filters. Logical “and” gives erosions, “or” gives dilations

5. Johann Radon Institute for Computational and Applied Mathematics: www.ricam.oeaw.ac.at 5/28 Erosion and dilation of a curve can be considered as a ball rolled over it at the outer and inner borders. The larger structuring element smoothes the curve more. The size of the ball is the scale of the smoothing process.

6. Johann Radon Institute for Computational and Applied Mathematics: www.ricam.oeaw.ac.at 6/28 The motion of the contour of the image is governed by the structuring element in exactly the same way as the level set is moved in the direction of the normal. This is only true for an isotropic convex (i.e. round) structuring element. One also says that the unit gradient vector | Ñ L| is the infinitesimal generator for the normal motion evolution equation. The sequence of erosion followed by dilation removes small structures selectively from the image:

7. Johann Radon Institute for Computational and Applied Mathematics: www.ricam.oeaw.ac.at 7/28 The intermediate steps Top row: three consecutive erosions of the text image. Bottom row: Three consecutive dilations of the eroded image.

8. Johann Radon Institute for Computational and Applied Mathematics: www.ricam.oeaw.ac.at 8/28 Opening / closing Closing = erosion of a dilation (org, s=3, s=5) Opening = dilation of an erosion (org, s=3, s=5)

9. Johann Radon Institute for Computational and Applied Mathematics: www.ricam.oeaw.ac.at 9/28 Subtracting the result of the operation of erosion (of the original image) from the result of the operation of dilation (of the original image) gives us the result of the morphological gradient operator:

10. Johann Radon Institute for Computational and Applied Mathematics: www.ricam.oeaw.ac.at 10/28 Mathematical morphology on gray-valued images The classical way to change the binary operators from mathematical morphology into operators for gray- valued images, is to replace the binary operators by maximum/minimum operators. Where B is the structuring element. Example: original, dilated, and eroded image

11. Johann Radon Institute for Computational and Applied Mathematics: www.ricam.oeaw.ac.at 11/28 Opening / closing Similarly opening and closing are defined:

12. Johann Radon Institute for Computational and Applied Mathematics: www.ricam.oeaw.ac.at 12/28 … as well as the morphological gradient for this image and structuring element as eroded -dilated:

13. Johann Radon Institute for Computational and Applied Mathematics: www.ricam.oeaw.ac.at 13/28 It can be shown that dilation or erosion with a ball is mathematically equivalent to constant motion flow, where the isophotes are considered as curves and they are moved in the gradient (or opposite) direction.

14. Johann Radon Institute for Computational and Applied Mathematics: www.ricam.oeaw.ac.at 14/28 A parabolic structuring element establishes an elegant equivalence between mathematical morphology and Gaussian scale-space. –Decomposition –Unique rotationally symmetric function –Closed w.r.t. convolution Boomgaard, R.v.d. and Dorst, L. 1997. The morphological equivalent of the Gaussian scale-space. In Gaussian Scale-Space Theory, volume 8 of Computational Imaging and Vision Series, chap. 15, pp. 203–220. Mathematical morphology and Gaussian scale-space are cases from a more general formulation:

15. Johann Radon Institute for Computational and Applied Mathematics: www.ricam.oeaw.ac.at 15/28 Vertical: scale s = 2 k/2, k=0,…,8 Horizontal: m= -8, -4, -2, -1, 0, 1, 2, 4, 8 m=0: Gaussian scale space. m  +: dilation m  -: erosion Pseudo-Linear Scale- Space Theory Florack, Maas and Niessen IJCV 31(2/3), 247-259, 1999

16. Johann Radon Institute for Computational and Applied Mathematics: www.ricam.oeaw.ac.at 16/28 Mumford Shah

17. Johann Radon Institute for Computational and Applied Mathematics: www.ricam.oeaw.ac.at 17/28 Methods of diffusion-reaction type Nordström [1990] has suggested to obtain a reconstruction u of a degraded image f by minimizing the energy functional

18. Johann Radon Institute for Computational and Applied Mathematics: www.ricam.oeaw.ac.at 18/28 The corresponding Euler equations to this energy functional are given by Equipped with a homogeneous Neumann boundary condition for u. Solving the second equation gives This is the PM term !

19. Johann Radon Institute for Computational and Applied Mathematics: www.ricam.oeaw.ac.at 19/28 So for the first equation is the steady state of This equation can also be obtained directly as the descent method of the functional

20. Johann Radon Institute for Computational and Applied Mathematics: www.ricam.oeaw.ac.at 20/28 PM with additional bias term No need to find the PM stopping time However, now b needs to be found We still have the ill-posedness problem as PM.

21. Johann Radon Institute for Computational and Applied Mathematics: www.ricam.oeaw.ac.at 21/28 Mumford Shah Mumford and Shah [1985] have proposed to obtain a segmented image u from f by minimizing the functional with a,b ≥ 0. The discontinuity set K consists of the edges, and its one­dimensional Hausdorff measure |K| gives the total edge length. Like the Nordström functional, this expression consists of three cost terms: –the first one is the deviation cost, –the second one gives the stabilizing cost, and –the third one represents the edge cost.

22. Johann Radon Institute for Computational and Applied Mathematics: www.ricam.oeaw.ac.at 22/28 Numerical complications arise from the fact that the Mumford-Shah functional has numerous local minima. Global minimizers such as the simulated annealing method are extremely slow. Hence, one searches for fast (suboptimal) deterministic strategies, e.g. pyramidal region growing algorithms. Another interesting class of numerical methods is based on the idea to approximate the discontinuity set K by a smooth function w, which is close to 0 near edges of u and which approximates 1 elsewhere. We may for instance study the functional with a parameter c>0 specifying the ``edge width'‘.

23. Johann Radon Institute for Computational and Applied Mathematics: www.ricam.oeaw.ac.at 23/28 Convergence Ambrosio and Tortorelli proved that this functional converges to the Mumford-Shah functional for c -> 0 (in the sense of G ­ convergence). Minimizing F f corresponds to the gradient descent equations with homogeneous Neumann boundary conditions. As this is very similar to the Nordström process, similar problems arise: –The functional F f is not jointly convex in u and v, so it may have many local minima and a gradient descent algorithm may get trapped in a poor local minimum. –Well­posedness results for this system have not been obtained up to now, but a maximum-minimum principle and a local stability proof have been established.

24. Johann Radon Institute for Computational and Applied Mathematics: www.ricam.oeaw.ac.at 24/28 Examples From: Inverse problems in Image processing and Image segmentation: some mathematical and numerical aspects A. Chambolle School on Mathematical Problems in Image Processing 4 - 22 September 2000, Trieste, Italy http://users.ictp.it/~pub_off/lectures/vol2.html http://users.ictp.it/~pub_off/lectures/vol2.html

25. Johann Radon Institute for Computational and Applied Mathematics: www.ricam.oeaw.ac.at 25/28 Active Contours / Snakes

26. Johann Radon Institute for Computational and Applied Mathematics: www.ricam.oeaw.ac.at 26/28 Active Contours / Snakes A very short introduction Exploit the energy formulation http://www.icaen.uiowa.edu/~dip/LECTURE/Understanding2.html A Mathematica demo

27. Johann Radon Institute for Computational and Applied Mathematics: www.ricam.oeaw.ac.at 27/28 Summary Normal motion flow is equivalent to the mathematical morphological erosion or dilation with a ball. –The dilation and erosion operators are shown to be convolution operators with boolean operations on the operands. –Morphology with a quadratic structuring element links to Gaussian scale space –There exists a “pseudo-linear” equation linking them. The Mumford-Shah functional is designed to generate edges while denoising –Not unique –Complicated Active contours / snakes are defined as an energy minimizing splines that are supposed to converge to edges.

28. Johann Radon Institute for Computational and Applied Mathematics: www.ricam.oeaw.ac.at 28/28 Next week Deep structure in Gaussian Scale Space –Critical points –Movement of critical points –Catastrophe points (singularity theory) Annihilations Creations –Scale space critical points –Iso-manifolds –Hierarchy –Topological segmentation