1. Mathematical Morphology in Image Processing Dr.K.V.Pramod Dept. of Computer Applications Cochin University of Sc. & Technology

2. What is Mathematical Morphology ? What is Mathematical Morphology ?

3. An (imprecise) Mathematical answer : A mathematical tool for investigating geometric structure in binary and grayscale images. A mathematical tool for investigating geometric structure in binary and grayscale images.

4. A (precise) Mathematical Answer Algebra Complete Lattices Operators Erosions-Dilations Mathematical Morphology Topology Hit-or-Miss Geometry Convexity - Connectivity Distance Applications Image Processing and Analysis A mathematical tool that studies operators on complete lattices A mathematical tool that studies operators on complete lattices

5. With in Biology, the term morphology is used for the study of the shape and structure of animals and plants. With in Biology, the term morphology is used for the study of the shape and structure of animals and plants. In image processing Mathematical morphology is theoretical framework for representation, description and pre-processing In image processing Mathematical morphology is theoretical framework for representation, description and pre-processing

6. Built on Minkowski set theory. Built on Minkowski set theory. Part of the theory of finite lattices. Part of the theory of finite lattices. A mathematical theory or methodology for nonlinear image processing. A mathematical theory or methodology for nonlinear image processing. A technique for image analysis based on shape. A technique for image analysis based on shape. Extremely useful, yet not often used Extremely useful, yet not often used

7. Morphology - advantages Preserves edge information Preserves edge information Works by using shape-based processing Works by using shape-based processing Can be designed to be idempotent Can be designed to be idempotent Computationally efficient Computationally efficient

8. Morphology - applications Image enhancement Image enhancement Image restoration (eg. Removing scratches from digital film) Image restoration (eg. Removing scratches from digital film) Edge detection Edge detection Texture analysis Texture analysis Noise reduction Noise reduction There are many more applications that morphology can be applied to. Morphology has been widely researched for use in image and video processing. There are many more applications that morphology can be applied to. Morphology has been widely researched for use in image and video processing.

9. Some History Mathematical morphology was developed in the 1970’s by George Matheron and Jean Serra - George Matheron (1975) Random Sets and Integral Geometry, John Wiley. –Jean Serra (1982) Image Analysis and Mathematical Morphology, Academic Press. –Petros Maragos (1985) A Unified Theory of Translations-Invariant Systems with Applications to Morphological Analysis and Coding of Images, Doctoral Thesis, Georgia Tech.

10. Why we use these techniques ?: Shape Processing and Analysis –Visual perception requires transformation of images so as to make explicit particular shape information. –Goal: Distinguish meaningful shape information from irrelevant one. –The vast majority of shape processing and analysis techniques are based on designing a shape operator which satisfies desirable properties.

11. Example Z Image analysis consists of obtaining measurements characteristic to images under consideration. Z Geometric measurements (e.g., object location, orientation, area, length of perimeter) Grayscale Images Binary Images

12. Mathematical Morphology Principles: Principles: –Such further processing is performed using one or a combination of several morphological transformations. –The transformations work in a certain local neighborhood of each pixel (similarly to convolution) defined by so called ‘ Structuring Element ‘. The structuring element can be square, cross-like or any other shape.

13. Mathematical Morphology: Binary Images Morphology uses Set Theory as foundation for many functions. The simplest functions to implement are Dilation and Erosion Dilation Dilation –Dilation replaces zeros neighboring to ones by ones. Erosion Erosion –Erosion replaces ones neighboring to zeros by zeros.

14. Two other basic operations are - Closing & Opening Closing & Opening Closing Closing Closing is dilation followed by erosion.Closing is dilation followed by erosion. Closing merges dense of ones together, fills smallClosing merges dense of ones together, fills small holes and smoothes boundaries. Closing smoothes objects by adding pixelsClosing smoothes objects by adding pixelsOpening Opening is erosion followed by dilation.Opening is erosion followed by dilation. Opening removes single ones, thin lines and divides objects connected with a narrow path (neck).Opening removes single ones, thin lines and divides objects connected with a narrow path (neck). Opening smoothes objects by removing pixelsOpening smoothes objects by removing pixels

15. Dilation (I) Brief Description Brief Description –One of the two basic operators –Basic effect Gradually enlarge the boundaries of regions of foreground pixels on a binary image. Gradually enlarge the boundaries of regions of foreground pixels on a binary image. Dilate, Grow, Expand Common names: Dilate, Grow, Expand

16. Dilation (II) How It Works How It Works –A: set of Euclidean coordinates corresponding to the input binary image –B: set of coordinates for the structuring element –B x : translation of B so that its origin is at x. –The dilation of A by B is simply the set of all points x such that the intersection of B x with A is non-empty.

17. Dilation in 1D is defined as : A + B = {x : (Ḃ) x (intersection) A ≠ Ф} = Ü xЄB A x (1) A + B = {x : (Ḃ) x (intersection) A ≠ Ф} = Ü xЄB A x (1) Where A and B are sets in Z. this definition is also known as ‘Minkowski Addition’. This equation simply means that B is moved over A and the intersection of B reflected and translated with A is found. Usually A will be the signal or image being operated on and B will be the Structuring Element’. Where A and B are sets in Z. this definition is also known as ‘Minkowski Addition’. This equation simply means that B is moved over A and the intersection of B reflected and translated with A is found. Usually A will be the signal or image being operated on and B will be the Structuring Element’.

18. . The following figure shows how dilation works on a 1D binary signal. The output is given by (1) and will be set to one unless the input is the inverse of the structuring element. For example, in the input it is ‘000’ would cause the output to be zero, if in the SE it ‘111’. The output is given by (1) and will be set to one unless the input is the inverse of the structuring element. For example, in the input it is ‘000’ would cause the output to be zero, if in the SE it ‘111’.

19. How dilation works

20. . Dilation has several interesting properties, which make it useful for image processing. These properties are: Translation invariant. Translation invariant. This means that the result of A dilated with B translated is the same as A translated dilated with B as given by: This means that the result of A dilated with B translated is the same as A translated dilated with B as given by: (A + B) x = A x + B(2) (A + B) x = A x + B(2) Order invariant Order invariant This simply means that if several dilations are to be done, then the order in which they are done is irrelevant. The result will be same irrespective. This simply means that if several dilations are to be done, then the order in which they are done is irrelevant. The result will be same irrespective. (A + B) + C = A + (B + C)(3) (A + B) + C = A + (B + C)(3)

21. Increasing operator Increasing operator This means that if a set A, is a subject of another set B, then the dilation of A by C is still a subset of B dilated by C: This means that if a set A, is a subject of another set B, then the dilation of A by C is still a subset of B dilated by C: (A contained in B) =› A + C contained in B + C(4) (A contained in B) =› A + C contained in B + C(4) Scale invariant Scale invariant This means that the input and structuring element can be scaled, then dilated and will give the same as scaling the dilated output: This means that the input and structuring element can be scaled, then dilated and will give the same as scaling the dilated output: rA + rB = r(A + B)(5) rA + rB = r(A + B)(5) Where r is a scale factor. Where r is a scale factor. These properties can be very useful in image processing and can result in some operations being simplified. These properties can be very useful in image processing and can result in some operations being simplified.

22. Dilation (III) Guideline for Use Guideline for Use Effect of dilation using a 3×3 square structuring element 3×3 square structuring element Set of coordinate points = { (-1, -1), (0, -1), (1, -1), (-1, 0), (0, 0), (1, 0), (-1, 1), (0, 1), (1, 1) }

23. Dilation (IV) –Example: Binary dilation –Note that the corners have been rounded off. When dilating by a disk shaped structuring element, convex boundaries will become rounded, and concave boundaries will be preserved as they are. When dilating by a disk shaped structuring element, convex boundaries will become rounded, and concave boundaries will be preserved as they are. Original image Thresholded image Result of two Dilation passes using a disk shaped structuring element of 11 pixels radius

24. Dilation (V) –Example: Binary dilation (edge detection) Dilation can also be used for edge detection by taking the dilation of an image and then subtracting away the original image. Dilation can also be used for edge detection by taking the dilation of an image and then subtracting away the original image. Original image Result of dilation Edge detection

25. Dilation (VI) –Example: Binary dilation (Region Filling) Dilation is also used as the basis for many other mathematical morphology operators, often in combination with some logical operators. Dilation is also used as the basis for many other mathematical morphology operators, often in combination with some logical operators. Original imageRegion filling

26. Dilation (VII) Conditional dilation Conditional dilation –Combination of the dilation operator and a logical operator –Region filling applies logical NOT, logical AND and dilation iteratively.

27. Dilation (VIII) X 0 One pixel which lies inside the region Dilate the left image Negative of the boundary Original image AND Step 1 Result

28. Dilation (IX) Dilate the left image Negative of the boundary AND Step 2 Result X1X1

29. Dilation (X) –Repeating dilation and with the inverted boundary until convergence, yields Final Result Step 3 Result Step 4 ResultStep 5 ResultStep 6 Result OR Original image Result of Region Filling

30. Dilation (XI) Grayscale Dilation Grayscale Dilation –Generally brighten the image Bright regions surrounded by dark regions grow in size, and dark regions surrounded by bright regions shrink in size. Bright regions surrounded by dark regions grow in size, and dark regions surrounded by bright regions shrink in size. –The effect of dilation using a disk shaped structuring element

31. Dilation (XII) –Example: Grayscale dilation (brighten the image) The highlights on the bulb surface have increased in size. The highlights on the bulb surface have increased in size. The dark body of the cube has shrunk in size since it is darker than its surroundings. The dark body of the cube has shrunk in size since it is darker than its surroundings. Original image Two passes Dilation by 3×3 flat square structuring element Five passes Dilation by 3×3 flat square structuring element

32. Dilation (XIII) Pepper noise imageDilation by 3×3 flat square structuring element

33. Morphological Erosion Pablo Picasso, Pass with the Cape, 1960 Structuring Element

34. Erosion (I) Brief Description Brief Description –Erosion is one of the basic operators in the area of mathematical morphology. –Basic effect Erode away the boundaries of regions of foreground pixels (i.e. white pixels, typically). Erode away the boundaries of regions of foreground pixels (i.e. white pixels, typically). Erode, Shrink, Reduce Common names: Erode, Shrink, Reduce

35. Erosion (II) How It Works How It Works –A: set of Euclidean coordinates corresponding to input binary image –B : set of coordinates for the structuring element –B x : translation of B so that its origin is at x –The erosion of A by B is simply the set of all points x such that B x is a subset of A

36. The Opposite of Dilation is known as Erosion This is defined as: This is defined as: A Ө B = {x : (B) x contained in A} = Π xЄB A x A Ө B = {x : (B) x contained in A} = Π xЄB A x (6) (6) This definition is also known as ‘Minkowski Substration’. The equation simply says, erosion of A by B is the set of points x such that B translated by x is contained in A. The following Figure shows how erosion works on a 1D binary signal. This works in exactly the same way as dilation. However (6) essentially says that for the output to be a one, all of the inputs must be the same as the structuring element. This definition is also known as ‘Minkowski Substration’. The equation simply says, erosion of A by B is the set of points x such that B translated by x is contained in A. The following Figure shows how erosion works on a 1D binary signal. This works in exactly the same way as dilation. However (6) essentially says that for the output to be a one, all of the inputs must be the same as the structuring element.

37. How Erosion Works

38. Erosion, like dilation also contains properties that are useful for image processing: Translation invariant. Translation invariant. This means that the result of A eroded with B translated is the same as A translated eroded with B as given by: This means that the result of A eroded with B translated is the same as A translated eroded with B as given by: (A Ө B)x = Ax Ө B(7) (A Ө B)x = Ax Ө B(7) Order invariant Order invariant This simply means that if several erosions are to be done, then the order in which they are done is irrelevant. The result will be same irrespective. This simply means that if several erosions are to be done, then the order in which they are done is irrelevant. The result will be same irrespective. (A Ө B) Ө C = A Ө (B Ө C)(8) (A Ө B) Ө C = A Ө (B Ө C)(8)

39. Increasing operator Increasing operator This means that if a set, A, is a subject of another set, B, then the erosion of A by C is still a subset of B eroded by C: This means that if a set, A, is a subject of another set, B, then the erosion of A by C is still a subset of B eroded by C: (A Ө B) Ө C = A Ө C contained in B Ө C(9) (A Ө B) Ө C = A Ө C contained in B Ө C(9) Scale invariant Scale invariant This means that the input and structuring element can eb scaled, then eroded and will give the same as scaling the eroded output: This means that the input and structuring element can eb scaled, then eroded and will give the same as scaling the eroded output: rA Ө rB = r(A Ө B)(10) rA Ө rB = r(A Ө B)(10) Where r is a scale factor. Where r is a scale factor.

40. Erosion (III) Guideline for Use Guideline for Use Effect of erosion using a 3×3 square structuring element A 3×3 square structuring element Set of coordinate points = { (-1, -1), (0, -1), (1, -1), (-1, 0), (0, 0), (1, 0), (-1, 1), (0, 1), (1, 1) }

41. Erosion (IV) –Example: Binary erosion It shows that the hole in the middle of the image increases in size as the border shrinks. It shows that the hole in the middle of the image increases in size as the border shrinks. Erosion using a disk shaped structuring element will tend to round concave boundaries, but will preserve the shape of convex boundaries. Erosion using a disk shaped structuring element will tend to round concave boundaries, but will preserve the shape of convex boundaries. Original thresholded imageResult of erosion four times with a disk shaped structuring element of 11 pixels in diameter

42. Erosion (V) –Example: Binary erosion (separate touching objects) Original image (a number of dark disks)Inverted image after thresholding The result of eroding twice using a disk shaped structuring element 11 pixels in diameter Using 9×9 square structuring element leads to distortion of the shapes

43. Erosion (VI) –Grayscale erosion Generally darken the image Generally darken the image –Bright regions surrounded by dark regions shrink in size, and dark regions surrounded by bright regions grow in size.  The effect of erosion using a disk shaped structuring element

44. Erosion (VII) –Example: Grayscale erosion (darken the image) The highlights have disappeared. The highlights have disappeared. The body of the cube has grown in size since it is darker than its surroundings. The body of the cube has grown in size since it is darker than its surroundings. Original image Two passes Erosion by 3×3 flat square structuring element Five passes Erosion by 3×3 flat square structuring element

45. Erosion (VIII) –Example: Grayscale erosion (Remove salt noise) The noise has been removed. The noise has been removed. The rest of the image has been degraded significantly. The rest of the image has been degraded significantly. Salt noise image Erosion by 3×3 flat square structuring element

46. Opening (I) Brief Description Brief Description –The Basic Effect Somewhat like erosion in that it tends to remove some of the foreground(bright) pixels from the edges of regions of foreground pixels. Somewhat like erosion in that it tends to remove some of the foreground(bright) pixels from the edges of regions of foreground pixels. To preserve foreground regions that have a similar shape to structuring element. To preserve foreground regions that have a similar shape to structuring element.

47. Opening (II) How It works How It works –Opening is defined as an erosion followed by a dilation. –Gray-level opening consists simply of a gray- level erosion followed by a gray-level dilation. –Opening is the dual of closing. Opening the foreground pixels with a particular structuring element is equivalent to closing the background pixels with the same element. Opening the foreground pixels with a particular structuring element is equivalent to closing the background pixels with the same element.

48. Both dilation and erosion have interesting and useful properties. However, it would be useful to have the properties of both in one function. This can be done in two ways. The first method, ‘Opening’ is defined as: Both dilation and erosion have interesting and useful properties. However, it would be useful to have the properties of both in one function. This can be done in two ways. The first method, ‘Opening’ is defined as: A ๐ B = (A Ө B) + B (11) A ๐ B = (A Ө B) + B (11) This simply erodes the signal and then dilates the result as shown in Figure 3. As can be seen, the zeros are opened up. Any ones that are shorter than the structuring element are removed, but the rest of the signal is left unchanged. This simply erodes the signal and then dilates the result as shown in Figure 3. As can be seen, the zeros are opened up. Any ones that are shorter than the structuring element are removed, but the rest of the signal is left unchanged. This is a very useful property as it means that if the filter is applied once, no more changes to the signal will result from repeated applications is known as Idempotent: This is a very useful property as it means that if the filter is applied once, no more changes to the signal will result from repeated applications is known as Idempotent: (A ๐ B) ๐ B = A ๐ B(12) (A ๐ B) ๐ B = A ๐ B(12)

49. Opening (III) Guidelines for Use Guidelines for Use – Idempotence After the opening has been carried out, the new boundaries of foreground regions will all be such that the structuring element fits inside them. After the opening has been carried out, the new boundaries of foreground regions will all be such that the structuring element fits inside them. So further openings with the same element have no effect So further openings with the same element have no effect Effect of opening using a 3×3 square structuring element Effect of opening using a 3×3 square structuring element

50. Opening (IV) Binary Opening Example Binary Opening Example –Separate out the circles from the lines The lines have been almost completely removed while the circles remain almost completely unaffected. The lines have been almost completely removed while the circles remain almost completely unaffected. A mixture of circle and lines Opening with a disk shaped structuring element with 11 pixels in diameter

51. Opening (V) Binary Opening Example Binary Opening Example –Extract the horizontal and vertical lines separately There are a few glitches in rightmost image where the diagonal lines cross vertical lines. There are a few glitches in rightmost image where the diagonal lines cross vertical lines. These could easily be eliminated, however, using a slightly longer structuring element. These could easily be eliminated, however, using a slightly longer structuring element. Original image The result of an Opening with 3×9 vertically oriented structuring element The result of an Opening with 9×3 horizontally oriented structuring element

52. Opening (VI) –Example: Binary opening Original image Inverted image after thresholding The result of an Opening with 11 pixel circular structuring element The result of an Opening with 7 pixel circular structuring element

53. Opening (VII) –Example: Grayscale opening The important thing to notice here is the way in which bright features smaller than the structuring element have been greatly reduced in intensity. The important thing to notice here is the way in which bright features smaller than the structuring element have been greatly reduced in intensity. The fine grained hair and whiskers have been much reduced in intensity. The fine grained hair and whiskers have been much reduced in intensity. Original image Opening with a flat 5 ×5 square structuring element

54. Opening (VIII) Compare Opening with Erosion Compare Opening with Erosion –Opening The noise has been entirely removed with relatively little degradation of the underlying image. The noise has been entirely removed with relatively little degradation of the underlying image. –Erosion The noise has been removed. The noise has been removed. The rest of the image has been degraded significantly. The rest of the image has been degraded significantly. Salt noise ImageOpening with 3×3 square structuring element Erosion by 3×3 flat square structuring element

55. Opening (IX) –Example: Grayscale opening (pepper noise) No noise has been removed. No noise has been removed. At some places where two nearby noise pixels have merged into one larger point, the noise level has even been increased. At some places where two nearby noise pixels have merged into one larger point, the noise level has even been increased. Pepper noise imageResult of Opening

56. Closing (I) Brief Description Brief Description –Closing is one of the two important operators from mathematical morphology. –Closing is similar in some ways to dilation in that it tends to enlarge the boundaries of foreground (bright) regions in an image. Closing Common names : Closing

57. Closing (II) How It works How It works –Closing is defined as a dilation followed by an erosion. Closing is the dual of opening. Closing the foreground pixels with a particular structuring element is equivalent to opening the background pixels with the same element. Closing the foreground pixels with a particular structuring element is equivalent to opening the background pixels with the same element.

58. Closing … the opposite of opening is ‘Closing’ defined by” the opposite of opening is ‘Closing’ defined by” A ● B = (A + B) Ө B(13) A ● B = (A + B) Ө B(13) It can be seen that this closes gaps in the signal in the same way as opening opened up gaps. Closing also has the property of being idempotent. It can be seen that this closes gaps in the signal in the same way as opening opened up gaps. Closing also has the property of being idempotent.

59. Closing (III) Guidelines for Use Guidelines for Use – Idempotence After the closing has been carried out, the background region will be such that the structuring element can be made to cover any point in the background without any part of it also covering a foreground point. After the closing has been carried out, the background region will be such that the structuring element can be made to cover any point in the background without any part of it also covering a foreground point. So further closings will have no effect. So further closings will have no effect. Effect of closing using a 3×3 square structuring element

60. Closing (IV) –Example: Binary closing If it is desired to remove the small holes while retaining the large holes, then we can simply perform a closing with a disk-shaped structuring element with a diameter larger than the smaller holes, but smaller than the larger holes. If it is desired to remove the small holes while retaining the large holes, then we can simply perform a closing with a disk-shaped structuring element with a diameter larger than the smaller holes, but smaller than the larger holes. Original image Result of a closing with a 22 pixel diameter disk

61. Closing (V) –Example: Binary closing Enhance binary images of objects obtained from thresholding. Enhance binary images of objects obtained from thresholding. We can see that skeleton (B) is less complex, and it better represents the shape of the object. We can see that skeleton (B) is less complex, and it better represents the shape of the object. Original imageThresholded image Result of closing with a circular structuring element of size 20 (B) The skeleton of the image produced by the closing operator (A) The skeleton of the image which was only thresholded

62. Closing (VI) –Example: Grayscale closing Gray-level closing can similarly be used to select and preserve particular intensity patterns while attenuating others. Gray-level closing can similarly be used to select and preserve particular intensity patterns while attenuating others. Notice how the dark specks in between the bright spots in the hair have been largely filled in to the same color as the bright spots. Notice how the dark specks in between the bright spots in the hair have been largely filled in to the same color as the bright spots. But, the more uniformly colored nose area is largely the same intensity as before. But, the more uniformly colored nose area is largely the same intensity as before. Original image Opening with a flat 5 ×5 square structuring element

63. Closing (VII) –Compare Closing with Dilation *Closing –The noise has been completely removed with only a little degradation to the image. Dilation Dilation –Note that although the noise has been effectively removed, the image has been degraded significantly. Dilation by 3×3 flat square structuring element Closing with 3×3 square structuring element Pepper noise image

64. Closing (VIII) –Example: Grayscale closing (salt noise) No noise has been removed. No noise has been removed. The noise has even been increased at locations where two nearby noise pixels have merged together into one larger spot. The noise has even been increased at locations where two nearby noise pixels have merged together into one larger spot. Salt noise imageResult of Closing

65. ‘Open-close’ & ‘Close-open’ filters Both ‘ Open’ and ‘C lose’ filters again have interesting properties that would be nice to have in one filter. The opening and closing can be combined to merge these properties. There are two ways of combining these, the first of which is known as an ‘Open-Close’ filter and is defined by: Both ‘ Open’ and ‘C lose’ filters again have interesting properties that would be nice to have in one filter. The opening and closing can be combined to merge these properties. There are two ways of combining these, the first of which is known as an ‘Open-Close’ filter and is defined by: A O● B = (A ๐ B) ● B(14) A O● B = (A ๐ B) ● B(14) The signal is first opened and the result is then closed. The opposite can also be done by closing and then opening. This is called a ‘Close-Open’ filter and is defined by: The signal is first opened and the result is then closed. The opposite can also be done by closing and then opening. This is called a ‘Close-Open’ filter and is defined by: A ●O B = (A ● B) ๐ B(15) A ●O B = (A ● B) ๐ B(15)

66. Extending to Grey scale and Extending to 2D & 3D For morphology to be of use in image processing, it needs to be extended to non-binary signals. There are various ways in which this can be done. The chosen method uses very simple functions, which allow them to be implemented in an efficient way. One method of implementing is Gray scale morphology and further extending to 2D & 3D. For morphology to be of use in image processing, it needs to be extended to non-binary signals. There are various ways in which this can be done. The chosen method uses very simple functions, which allow them to be implemented in an efficient way. One method of implementing is Gray scale morphology and further extending to 2D & 3D.

67.  THANK YOU