1. Nottingham Image Analysis School, 23 – 25 June 2011 1 2011 NITS Image Segmentation Guoping Qiu School of Computer Science, University of Nottingham qiu@cs.nott.ac.uk | www.cs.nott.ac.uk/~qiu

2. Nottingham Image Analysis School, 23 – 25 June 2011 2 Lecture Plan Image segmentation in image analysis (ImageJ demo) A look into the algorithms … Convert an image (grey scale or color) to binary Clean up the binary image Extract/identify individual objects (regions) Describe/measure the objects (regions) Discussion/prospective

3. Nottingham Image Analysis School, 23 – 25 June 2011 3 First, a demo … ImageJ Demo Load image (File->Open) Make binary (Process->Binary->Make Binary) Process->Binary->Erode Process->Binary->Dilate Analyze->Analyze Particle http://www.cs.nott.ac.uk/~qiu/Teaching/NITS/test_im.png

4. Nottingham Image Analysis School, 23 – 25 June 2011 4 Identify and marking region of interest Describe the objects: Shapes, sizes, populations …

5. Nottingham Image Analysis School, 23 – 25 June 2011 5 Identify and marking region of interest Describe the objects: Shapes, sizes, populations … Separating objects from the background and from each other Generally referred to as image segmentation A hard (unsolved) problem

6. Nottingham Image Analysis School, 23 – 25 June 2011 6 Identify and marking region of interest Describe the objects: Shapes, sizes, populations … Identify individual objects or regions (connected component labelling)

7. Nottingham Image Analysis School, 23 – 25 June 2011 7 Identify and marking region of interest Describe the objects: Shapes, sizes, populations … Measure the sizes, shapes, and other attributes of individual objects (regions)

8. Nottingham Image Analysis School, 23 – 25 June 2011 8 Image Segmentation A very hard (unsolved problem) Same organ, different signals Different organ, same signal

9. Nottingham Image Analysis School, 23 – 25 June 2011 9 Image Segmentation A very hard (unsolved problem) Background Or Foreground ? Even human find it hard/impossible

10. Nottingham Image Analysis School, 23 – 25 June 2011 10 Image Segmentation For simple cases, use thresholding

11. Nottingham Image Analysis School, 23 – 25 June 2011 11 Image Segmentation Thresholding strategies One sided Two sided Set membership

12. Nottingham Image Analysis School, 23 – 25 June 2011 12 Image Segmentation Clean up the thresholded binary image Thresholded binary image will have isolated points, objects will have holes etc, it is necessary to clean up the image, remove unwanted pixels.

13. Nottingham Image Analysis School, 23 – 25 June 2011 13 Image Segmentation Clean up the thresholded binary image Morphological Operators Two basic operators Dilation Erosion Two combinations Closing Opening Structure elements

14. Nottingham Image Analysis School, 23 – 25 June 2011 14 Image Segmentation Morphological Operators Dilation For each background pixel (which we will call the input pixel) we superimpose the structuring element on top of the input image so that the origin of the structuring element coincides with the input pixel position. If at least one pixel in the structuring element coincides with a foreground pixel in the image underneath, then the input pixel is set to the foreground value. If all the corresponding pixels in the image are background, however, the input pixel is left at the background value.

15. Nottingham Image Analysis School, 23 – 25 June 2011 15 Image Segmentation Morphological Operators Dilation 1 1 1

16. Nottingham Image Analysis School, 23 – 25 June 2011 16 Image Segmentation Morphological Operators Dilation

17. Nottingham Image Analysis School, 23 – 25 June 2011 17 Image Segmentation Morphological Operators Erosion For each foreground pixel (which we will call the input pixel) we superimpose the structuring element on top of the input image so that the origin of the structuring element coincides with the input pixel coordinates. If for every pixel in the structuring element, the corresponding pixel in the image underneath is a foreground pixel, then the input pixel is left as it is. If any of the corresponding pixels in the image are background, however, the input pixel is also set to background value.

18. Nottingham Image Analysis School, 23 – 25 June 2011 18 Image Segmentation Morphological Operators Erosion 1 1 1

19. Nottingham Image Analysis School, 23 – 25 June 2011 19 Image Segmentation Morphological Operators Erosion

20. Nottingham Image Analysis School, 23 – 25 June 2011 20 Image Segmentation Morphological Operators Opening: an erosion followed by a dilation using the same structuring element for both operations

21. Nottingham Image Analysis School, 23 – 25 June 2011 21 Image Segmentation Morphological Operators Opening

22. Nottingham Image Analysis School, 23 – 25 June 2011 22 Image Segmentation Morphological Operators Closing: opening performed in reverse

23. Nottingham Image Analysis School, 23 – 25 June 2011 23 Image Segmentation Morphological Operators Closing: opening performed in reverse

24. Nottingham Image Analysis School, 23 – 25 June 2011 24 Identify Individual Object Connected Component Labeling: Give pixels belonging to the same object the same label (grey level or color)

25. Nottingham Image Analysis School, 23 – 25 June 2011 25 Identify Individual Object Connected component labelling works by scanning an image, pixel-by-pixel (from top to bottom and left to right) in order to identify connected pixel regions, i.e. regions of adjacent pixels which share the same set of intensity values.

26. Nottingham Image Analysis School, 23 – 25 June 2011 26 Identify Individual Object Connected Pixels Two pixels p and q are said to be connected if there is a sequence of foreground (1) pixels Such that

27. Nottingham Image Analysis School, 23 – 25 June 2011 27 Identify Individual Object Pixel Neighbors 010 1p1 010 Four nearest neighbors 111 1p1 111 Eight nearest neighbors

28. Nottingham Image Analysis School, 23 – 25 June 2011 28 Identify Individual Object 4- and 8-Connected Pixels When only the four nearest neighbors are considered part of the neighborhood, then pixels p and q are said to be “4- connected” When the 8 nearest neighbors are considered part of the neighborhood, then pixels p and q are said to be “8- connected” 4-connected 8-connected

29. Nottingham Image Analysis School, 23 – 25 June 2011 29 Labelling Algorithm The connected components labeling operator scans the image by moving along a row until it comes to a point p (where p denotes the pixel to be labeled at any stage in the scanning process) for which V={1}. When this is true, it examines the four neighbors of p which have already been encountered in the scan (in the case of 8-connectivity, the neighbors (i) to the left of p, (ii) above it, and (iii and iv) the two upper diagonal terms). Based on this information, the labeling of p occurs as follows: If all four neighbors are 0, assign a new label to p, else if only one neighbor has V={1}, assign its label to p, else if more than one of the neighbors have V={1}, assign one of the labels to p and make a note of the equivalences. After completing the scan, the equivalent label pairs are sorted into equivalence classes and a unique label is assigned to each class. As a final step, a second scan is made through the image, during which each label is replaced by the label assigned to its equivalence classes. For display, the labels might be different graylevels or colors.

30. Nottingham Image Analysis School, 23 – 25 June 2011 30 Labelling Algorithm Example (8-connectivity) 1111 11 11 11 1 111 1122 11 11 31 3 333 Equivalent labels {1, 2} Original Binary ImageAfter first scan

31. Nottingham Image Analysis School, 23 – 25 June 2011 31 Labelling Algorithm Example (8-connectivity) 1122 11 11 31 3 333 Equivalent labels {1, 2} After 1 st scan 1111 11 11 21 2 222 Final label 1: {1, 2} 2: {3} After 2 nd scan (two individual objects/regions have been identified)

32. Nottingham Image Analysis School, 23 – 25 June 2011 32 Describe Objects Some useful features can be extracted once we have connected components, including Area Centroid Extremal points, bounding box Circularity Spatial moments

33. Nottingham Image Analysis School, 23 – 25 June 2011 33 Describe Objects Area Centroid

34. Nottingham Image Analysis School, 23 – 25 June 2011 34 Summary Identify and marking region of interest will normally involve the following steps Describe the objects: Shapes, sizes, populations …

35. Nottingham Image Analysis School, 23 – 25 June 2011 35 Summary Identify and marking region of interest will normally involve the following steps Convert an image (grey scale or color) to binary (also referred to as image segmentation). Clean up the binary image Extract individual objects (regions) Describe the objects (regions)

36. Nottingham Image Analysis School, 23 – 25 June 2011 36 Discussion (Automatic) Image segmentation is one of the most critical phases in image analysis (but it is still very hard!) Simple thresholding After some morphological processing

37. Nottingham Image Analysis School, 23 – 25 June 2011 37 Discussion Semi-automatic and more sophisticated segmentation algorithms exist User specify pixel classes

38. Nottingham Image Analysis School, 23 – 25 June 2011 38 Discussion Looking into the future - computer aided diagnosis … Describe the objects: Shapes, sizes, populations … Machine learning Mathematic Models AI Algorithms Object/region descriptions Diagnosis Output