1. Image Topology ( Part 3 ) Dr. Bai-ling Zhang School of Computer Science & Mathematics Victoria University of Technology SCM3511 Image Processing 2, Week 5

2. Skeletonization: Example Application Example: hand gesture recognition Skeleton of a binary object: a collection of lines and curves which encapsulate the size/shape of the object.

3. Skeletonization: Example Application Example: handwriting digits recognition

4. Skeletonization: Example Application: human motion analysis video imagemotion detection skeleton

5. What is a skeleton? A skeleton can be defined by medial axis of an object: a pixel is on the medial axis if is equidistant from at least two pixels on the boundary of the object. Methods of approaching the medial axis  Imagine the object to be burning up by a fire which advances at a constant rate from the boundary. The places where two lines of fire meet form the medial axis.  Consider the set of all circles lying within the object which touch at least two points on the boundary. The centres of all such circles form the medial axis.

6. Topological methods We can directly define those pixels which are to be deleted to obtain the final skeleton. In general, we want to delete pixels which can be deleted without changing the connectivity of an object:  i.e., the number of components, the number of holes, or the relationship of objects and holes unchanged Example 1 A non-deletable pixel: creates a hole

7. Example 2 A non-deletable pixel: removes a hole Example 3 A non-deletable pixel: disconnects an object

8. Example 4 A non-deletable pixel: 4-connectivity 8-connectivity 4-connectivity A pixel which can be deleted without changing the 4- connectivity of the object is called 4-simple; A pixel which can be deleted without changing the 8- connectivity of the object is called 8-simple. Neither 4-simple Nor 8-simple 8-simple Not 4-simple

9. Check for deletability for a pixel A pixel’s deletability can be tested by checking its 3x3 neighbourhood. —The top two pixels and the bottom two pixels become separated, thus breaking up the object. —The top two pixels and the bottom two pixels are joined by a chain of pixels outside the neighbourhood, i.e, all pixels will encircle a hole, and removing the central pixel will remove the hole For this example, checking the deletability of the central pixel:

10. Check for deletability for a pixel To check whether a pixel is 4-simple or 8-simple, introduce some numbers associated with the neighbourhood of a foreground pixel Define N p : the 3x3 neighbourhood of p, N p * : the 3x3 neighbourhood excluding p A(p): the number of 4-components in N p * C(p): the number of 8-components in N p * B(p): the number of foreground pixels in N p * A(p) = 2 C(p) = 2 B(p) = 4 A(p) = 2 C(p) = 1 B(p) = 5

11. The importance of simple points for deletion  Since C(p)=1 the central pixel is 8-simple and so can be deleted without affecting the 8-connectivity of the object.  But since A(p)~=1, the central pixel is not 4-simple and so cannot be deleted without affecting the 4-connectivity of the object. A(p): the number of 4-components in N p * C(p): the number of 8-components in N p * B(p): the number of foreground pixels in N p * A foreground pixel p is 4-simple iff A(p)=1, and is 8-simple iff C(p)=1 o o o A(p)=2 C(p)=1 o o o o o o

12. Calculating A(p) and C(p)  For A(p) we are only interested in the case where A(p)=1 and this can be determined by calculating the crossing number X(p) of a foreground pixel  The crossing number X( p ) of a foreground pixel p is defined to be the number of times a 0 is followed by a 1 as we traverse the 8-neighbours of p in a clockwise direction  If X(p) =1, then A(p)=1 and so p is 4-simple p 1 p 2 p 3 p 8 p p 4 p 7 p 6 p 5 p 1, p 2, p 3, p 4, p 5, p 6, p 7, p 8, p 1 11 0 0 p 0 0 1 1 1, 1, 0, 0, 1, 1, 0, 0, 1 X(p)=2

13. Calculating A(p) 11 0 1 P 1 0 0 1 1, 1, 0, 1, 1, 0, 0, 1, 1X(p)=2 11 1 1 P 1 1 0 0 1, 1, 1, 1, 0, 0, 1, 1, 1 X(p)=1 11 1 0 P 0 1 0 1 1, 1, 0, 0, 1, 0, 1, 0, 1 X(p)=3

14. Calculating crossing number in Matlab A matrix 3x3 matrix a in Matlab can be indexed using single indexing as: Create two sequences: (1). The neighbouring pixels in clockwise order: (2) The neighbouring pixels starting at a(4): a(1) a(4) a(7) a(2) a(5) a(8) a(3) a(6) a(9) >>p=[a(1) a(4) a(7) a(8) a(9) a(6) a(3) a(2)]; >> pp =[p(2:8) p(1)];

15. Calculating crossing number in Matlab pp(i) =p(i+1) with pp(8)=p(1). A “0,1” is counted if for any i with, we have p(i)=0 and pp(i)=1. This will occur if and only if is equal to 1. >> crossnum = sum((1-p).*pp); (1-p(i))*pp(i) So

16. Calculating C(p) in Matlab p 1 p 2 p 3 p 8 p p 4 p 7 p 6 p 5 o o o

17. How not to do skeletonization In general, a skeletonization algorithm works by an iteration process: at each step identifying deletable pixels, and deleting them. The algorithm will continue until no further deletions are possible. One way to remove pixels: At each step, find all foreground pixels which are 4-simple, and delete them all. BUT…. They are all 4-simple ! Deleting them all will thus remove the object completely. 0 0 0 0 1 1 1 1 0 0 0 0

18. Zhang-Suen skeletonization algorithm An extra test for deletability needed to avoid deleting too many pixels. Two options : 1.Provide a step-wise algorithm, and change the test for deletability at each step 2.Apply a different test for deletability according to where the pixel lies on the image grid. Z-S algorithm For odd iterations, delete only pixels which are on the right hand side, or bottom of an object, or on a north-west corner. For even iterations, delete only pixels which are on the left hand side, or top of an object, or on a south-east corner.

19. Zhang-Suen algorithm Delete all fagged pixels. Continue until there are no more deletable pixels in two successive iterations. Step N Flag a foreground pixel p=1 to be deletable if 1. 2  B(p)  6 2. X(p)=1 3.If N is odd, then p2 p4 p6 = 0 p4 p6 p8 = 0 4. If N is even, then p2 p4 p8 = 0 P2 p6 p8 = 0 If N is odd, then p 4 =0, or p 6 =0, or p 2 =p 8 =0 If N is even, then p 2 =0, or p 8 =0, or p 4 =p 6 =0

20. Item 1 ensures that we don't delete pixels which have only one neighbour, or have seven or more. The boxed pixels show those which will be deleted by steps 1 If N is odd, delete only pixels which are (1) on the right hand side, or (2) bottom of an object, or (3) on a north-west corner Example: Step 1

21. Step 2: If N is even, delete only pixels which are (1) on the left hand side, or (2) top of an object, or (3) on a south-east corner.

22. Skeleton: the unboxed foreground pixels in the right hand diagram

23. Matlab Implementation of Z-S algorithm Set up lookup tables: Apply these lookup tables alternately until there is no change in the image for two successive iterations. Manage this by keeping three images at any given time: the current image, the previous image, and the last. If the current and last images are equal, we stop. Otherwise, push the images back: one for the odd iterations, one for the even. last previous current applylut(current,lut)

24. Matlab Implementation of Z-S algorithm

27. Example: First create a binary image L= zeros(12,10); L(2:11,2:6)=1; L(7:11,7:9)=1; Then apply the zs algorithm: LS=zs(L)

28. Example