1. 1

2. Images as Functions 2 A Binary image is a function B(x,y) Є [0,1] Gray-tone image is a function: g(x,y) Є [0.,1,….L-1] A color image is represented by three functions f(x,y) =(f1(x,y), f2(x,y), f3(x,y)) or a vector-valued function: f(x,y) Multi-spectral Image: f(x,y) =(f1(x,y), f2(x,y),…, fn(x,y))

3. Gray-tone Image as Function 3

4. Image vs Matrix 4 There are many different file formats.

5. 5 Digital Image Terminology: 0 0 0 0 1 0 0 0 0 1 1 1 0 0 0 1 95 96 94 93 92 0 0 92 93 93 92 92 0 0 93 93 94 92 93 0 1 92 93 93 93 93 0 0 94 95 95 96 95 pixel (with value 94) its 3x3 neighborhood binary image gray-scale (or gray-tone) image color image multi-spectral image labeled image region of medium intensity resolution (7x7)

6. Image File Formats  PGM Portable Gray Map, older form  GIF was early commercial version Graphics Interchange Format  JPEG (JPG) is modern version Joint Photographic Experts Group  MPEG for motion Motion Picture Experts Group 6

7. How do you create an İmage file format?  Given image, 1. Find a compact mathematical representation 2. Code the mathematical representation 7

8.  Many formats exist: header plus data  Do they handle color?  Do they provide for compression?  Are there good packages that use them or at least convert between them? 8

9. PGM image with ASCII info.  P2 means ASCII gray  Comments  W=16; H=8  192 is max intensity  Can be made with editor  Large images are usually not stored as ASCII 9

10. PBM/PGM/PPM Codes 10 P1: ascii binary (PBM) P2: ascii grayscale (PGM) P3: ascii color (PPM) P4: byte binary (PBM) P5: byte grayscale (PGM) P6: byte color (PPM)

11. JPG current popular form  Public standard  Allows for image compression; often 10:1 or 30:1 are easily possible  8x8 intensity regions are fit with basis of cosines  Error in cosine fit coded as well  Parameters then compressed with Huffman coding  Common for most digital cameras 11

12. 12

13. From 3D Scenes to 2D Images 13 Object World Camera Real Image Pixel Image

14. 14

15. Binary Image B(r,c) 15 0 represents the background 1 represents the foreground 00010010001000 00011110001000 00010010001000

16. Binary Image Analysis  is used in a number of practical applications, e.g. 16 Part inspection Shape analysis Enhancement Document processing

17. What kinds of operations? 17 Separate objects from background and from one another Aggregate pixels for each object Compute features for each object

18. Example: red blood cell image  Many blood cells are separate objects  Many touch – bad!  Salt and pepper noise from thresholding  How useable is this data? 18

19. Results of analysis  63 separate objects detected  Single cells have area about 50  Noise spots  Gobs of cells 19

20. Binary Image Operations 20 1.Thresholding a gray-tone image 2.Convolution 3.Morphology 4.Feature extractions (area, centroid) 5.Connected components analysis

21. 1. Thresholding 21 Convert gray level or color image into binary image Use histogram Definition: The Histogram of a gray-level image I is defined as H(m) = { (r,c) : I(r,c) =m) } Where m spans the gray values

22. Histogram-Directed Thresholding 22 How can we use a histogram to separate an image into 2 (or several) different regions? Is there a single clear threshold? 2? 3?

23. Histogram  Background is black  Healthy cherry is bright  Bruise is medium dark  Histogram shows two cherry regions (black background has been removed) 23 gray-tone values pixel counts 0 256

24. Automatic Thresholding: Otsu’s Method 24 Assumption: the histogram is bimodal t Method: find the threshold t that minimizes the weighted sum of within-group variances for the two groups that result from separating the gray tones at value t. Grp 1 Grp 2

25. Thresholding Example 25 original gray tone imagebinary thresholded image

26. 2. Convolution  Given a gray level image I(r,c) and a mask m(r,c) convolution is I(r,c)*m(r,c)= ΣΣ I(k,l). m(r-k,m-l) 26 Masks

27. SMOOTHING MASKS

28. Convolution of an İmage with a Mask 28

29. Image Enhancement WITH SMOOTING MASKS

30. Averaging blurrs the image

32. Image Enhancement WITH AVERAGING AND THRESHOLDING Image Enhancement WITH AVERAGING AND THRESHOLDING

33. Restricted Averaging  Apply averaging to only pixels with brightness value outside a predefined interval. Mask m(i,j) = 1For g(k+i,l+j)€ [min, max] 0 otherwise

34. Median Filtering  Find a median value of a given neighborhood.  Removes sand like noise 021 222 332 021 212 332 0 1 1 2 2 2 2 3 3

36. EDEGE DETECTİON BY CONVOLUTION EDGE PROFILES EDEGE DETECTİON BY CONVOLUTION EDGE PROFILES Edges are the pixels where the brightness changes abrubtly. It is a vector variable with magnitude and direction

37. EDGES, GRADIENT AND LAPLACIAN

38. SMOOT EDGES, NOISY EDGES

39. Continuous world  Gradient  Δg(x,y) = ∂g/ ∂x + ∂g/ ∂y  Magnitude: |Δg(x,y) | = √ (∂g/ ∂x) 2 + (∂g/ ∂y) 2  Phase : Ψ = arg (∂g/ ∂x, ∂g/ ∂y) radians

40. Discrete world  Use difference in various directions  Δi g(i,j) = g(i,j) - g(i+1,j)  or  Δj g(i,j) = g(i,j) - g(i,j+1)  or  Δij g(i,j) = g(i,j)- g(i+1,j+1)  or  |Δ g(i,j) | = |g(i,j)- g(i+1,j+1) | + |g(i,j+1)- g(i+1,j) |

41. GRADIENT EDGE MASKS Approximation in discrete grid GRADIENT EDGE MASKS Approximation in discrete grid

42. GRADIENT EDGE MASKS

43. Horizontal Horizontal vertical diagonal edges

44. Vertical, Horisontal and diagonal edge detection

45. EdgesEdges

46. LAPLACIAN MASKS

47. LAPLACIAN of GAUSSIAN EDGE MASKS

48. EDGE DETECTION

50. 3. Mathematical Morphology  Morphology: Study of forms of animals and plants  Mathematical Morphology: Study of shapes  Similar to convolution  Arithmetic operations Set Operations 50

51. Need to define Image as a Set  Given a binary image I (r,c), assume 1 correspond to object 0 correspond to backround. Define a set with elements to the coordinates of the object  X = { (r1,c1), (r2,c2),….} 51

52. 111000 000000  X= 52

53. Set Operations

54. Set Operations on Images AND, OR Set Operations on Images AND, OR

55. TRANSLATION REFLECTION

56. Set Operations

57. Morphologic Operations 57 Binary mathematical morphology consists of two basic operations dilation and erosion and several composite relations closing and opening

58. Dilation: A+B ={ Z: (Bz)∩ A≠ Φ} 58 Dilation expands the connected sets of 1s of a binary image. It can be used for 1. growing features 2. filling holes and gaps

59. Structuring Elements 59 A structuring element is a shape mask used in the basic morphological operations. They can be any shape and size that is digitally representable, and each has an origin. box hexagon disk something box(length,width) disk(diameter)

60. Dilation with Structuring Elements 60 The arguments to dilation and erosion are 1.a binary image B 2.a structuring element S dilate(B,S) takes binary image B, places the origin of structuring element S over each 1-pixel, and ORs the structuring element S into the output image at the corresponding position. 0 0 0 1 1 0 0 0 1 0 1 1 0 0 1 1 1 0 0 origin B S dilate B  S

61. DILATION

63. Erosion 63 Erosion shrinks the connected sets of 1s of a binary image. It can be used for 1. shrinking features 2. Removing bridges, branches and small protrusions

64. Erosion with Structuring Elements 64 erode(B,S) takes a binary image B, places the origin of structuring element S over every pixel position, and ORs a binary 1 into that position of the output image only if every position of S (with a 1) covers a 1 in B. 0 0 1 1 0 1 1 1 1 1 111111 0 0 0 0 0 0 0 1 1 0 0 0 0 0 0 B S origin erode B S

65. EROSİON: A-B = {z: (Bz) A

66. IMAGE ENHANCEMENT WİTH MORPHOLOGY

67. Example to Try 67 0 0 1 0 0 1 0 0 0 0 1 1 1 1 1 0 1 1 1 1 1 1 0 0 1 1 1 1 0 0 1 1 1 1 0 0 1 1 1 erode dilate with same structuring element S B

68. Opening and Closing 68 Closing is the compound operation of dilation followed by erosion (with the same structuring element) Opening is the compound operation of erosion followed by dilation (with the same structuring element)

69. Use of Opening 69 Original Opening Corners 1.What kind of structuring element was used in the opening? 2.How did we get the corners?

70. Morphological Image Processing

72. OBJECT DETECTION

73. BOUNDARY EXTRACTİON

75. HOW DO YOU REMOVE THE HOLES

76. Morphological Image Processing

84. 84

85. Gear Tooth Inspection 85 original binary image detected defects How did they do it?

86. Some Details 86

87. Region Properties 87 Properties of the regions can be used to recognize objects. geometric properties (Ch 3) gray-tone properties color properties texture properties shape properties (a few in Ch 3) motion properties relationship properties (1 in Ch 3)

88. Geometric and Shape Properties 88 area centroid perimeter perimeter length circularity elongation mean and standard deviation of radial distance bounding box extremal axis length from bounding box second order moments (row, column, mixed) lengths and orientations of axes of best-fit ellipse Which are statistical? Which are structural?

89. 2. Connected Components Labeling 89 Once you have a binary image, you can identify and then analyze each connected set of pixels. The connected components operation takes in a binary image and produces a labeled image in which each pixel has the integer label of either the background (0) or a component. binary image after morphology connected components

90. Methods for CC Analysis 90 1. Recursive Tracking (almost never used) 2. Parallel Growing (needs parallel hardware) 3. Row-by-Row (most common) Classical Algorithm (see text) Efficient Run-Length Algorithm (developed for speed in real industrial applications)

91. Equivalent Labels 91 0 0 0 1 1 1 0 0 0 0 1 1 1 1 0 0 0 0 1 0 0 0 1 1 1 1 0 0 0 1 1 1 1 0 0 0 1 1 0 0 0 1 1 1 1 1 0 0 1 1 1 1 0 0 1 1 1 0 0 0 1 1 1 1 1 1 0 1 1 1 1 0 0 1 1 1 0 0 0 1 1 1 1 1 1 1 1 1 1 1 0 0 1 1 1 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 1 1 1 1 1 1 0 0 0 0 0 1 1 1 1 1 Original Binary Image

92. Equivalent Labels 92 0 0 0 1 1 1 0 0 0 0 2 2 2 2 0 0 0 0 3 0 0 0 1 1 1 1 0 0 0 2 2 2 2 0 0 0 3 3 0 0 0 1 1 1 1 1 0 0 2 2 2 2 0 0 3 3 3 0 0 0 1 1 1 1 1 1 0 2 2 2 2 0 0 3 3 3 0 0 0 1 1 1 1 1 1 1 1 1 1 1 0 0 3 3 3 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 1 1 1 1 1 1 0 0 0 0 0 1 1 1 1 1 The Labeling Process 1  2 1  3

93. Run-Length Data Structure 93 1 1 1 1 1 1 1 1 1 1 1 0 1 2 3 4 0123401234 U N U S E D 0 0010 0340 1010 1440 2020 2440 4140 row scol ecol label 0123456701234567 Rstart Rend 1234560077123456007707 0123401234 Runs Row Index Binary Image

94. Run-Length Algorithm 94 Procedure run_length_classical { initialize Run-Length and Union-Find data structures count <- 0 /* Pass 1 (by rows) */ for each current row and its previous row { move pointer P along the runs of current row move pointer Q along the runs of previous row

95. Case 1: No Overlap 95 |/////| |/////| |////| |///| |///| |/////| Q P Q P /* new label */ count <- count + 1 label(P) <- count P <- P + 1 /* check Q’s next run */ Q <- Q + 1

96. Case 2: Overlap 96 Subcase 1: P’s run has no label yet |///////| |/////| |/////////////| Subcase 2: P’s run has a label that is different from Q’s run |///////| |/////| |/////////////| Q Q P P label(P) <- label(Q) move pointer(s) union(label(P),label(Q)) move pointer(s) }

97. Pass 2 (by runs) 97 /* Relabel each run with the name of the equivalence class of its label */ For each run M { label(M) <- find(label(M)) } where union and find refer to the operations of the Union-Find data structure, which keeps track of sets of equivalent labels.

98. Labeling shown as Pseudo-Color 98 connected components of 1’s from thresholded image connected components of cluster labels

99. Region Adjacency Graph 99 A region adjacency graph (RAG) is a graph in which each node represents a region of the image and an edge connects two nodes if the regions are adjacent. 1 2 3 4 12 34

100. Morphological Image Processing